Remove360: Benchmarking Residuals After Object Removal in 3D Gaussian Splatting

## Abstract

Abstract Understanding what semantic information persists after object removal is critical for privacy-preserving 3D reconstruction and editable scene representations.
In this work, we introduce a novel benchmark and evaluation framework to measure semantic residuals—the unintended semantic traces left behind—after object removal in 3D Gaussian Splatting.
We conduct experiments across a diverse set of indoor and outdoor scenes, showing that current methods can preserve semantic information despite the absence of visual geometry.
We also release Remove360, a dataset of pre/post-removal RGB images and object-level masks captured in real-world environments.
While prior datasets have focused on isolated object instances, Remove360 covers a broader and more complex range of indoor and outdoor scenes, enabling evaluation of object removal in the context of full-scene representations.
Given ground truth images of a scene before and after object removal, we assess whether we can truly eliminate semantic presence, and if downstream models can still infer what was removed.
Our findings reveal critical limitations in current 3D object removal techniques and underscore the need for more robust solutions capable of handling real-world complexity.
The evaluation framework is available at https://github.com/spatial-intelligence-ai/Remove360.git . Data are available at https://huggingface.co/datasets/simkoc/Remove360 .

## 1 Introduction

Trainable scene representations, such as neural radiance fields (NeRFs) Mildenhall et al. (2020); Barron et al. (2022); Reiser et al. (2021); Müller et al. (2022); Chen et al. (2024a); Kulhanek and Sattler (2023); Martin-Brualla et al. (2021) or 3D Gaussians Kerbl et al. (2023); Lin et al. (2024); Yu et al. (2024); Zhang et al. (2024); Kulhanek et al. (2024); Chen et al. (2024c); Wang et al. (2024) enable photorealistic 3D reconstructions from images.
These learnable scene representations can be easily extended to include features that have semantic meaning Kerr et al. (2023); Shi et al. (2024); Ye et al. (2024a); Wu et al. (2024a); Zhou et al. (2024); Hu et al. (2024); Jain et al. (2024).
This, in turn, creates a link between parts of the scene and human-understandable concepts in general and human language in particular.
This link can then be used for intuitive search in 3D scene representation by prompting Peng et al. (2023); Huang et al. (2024); Takmaz et al. (2025); Liang et al. (2024); Koch et al. (2024), *e.g*., asking ’find the remote control’.
Similarly, it enables intuitive editing operations Ye et al. (2024a); Zhou et al. (2024); Chen et al. (2024b); Gu et al. (2024); Choi et al. (2024a), *e.g*., by asking to ’remove the red armchair in the living room’.

The growing availability of learning-based 3D reconstruction software accessible to non-expert users sca; Tancik et al. (2023); Yu et al. (2022); RealityCapture2023 (2023); pol; Ye et al. (2024b); pos; lum, coupled with intuitive edit operations based on natural language Radford et al. (2021); Achiam et al. (2023); Schuhmann et al. (2022), opens up exciting possibilities:
Using data casually captured by a smart phone sca; pol; rea; lum, a user can easily create and edit photorealistic 3D models.
They can remove parts of the scene that they consider private, *e.g*., photos, documents, types of decoration, *etc*., before sharing these maps, *e.g*., before uploading them to online services for refurbishing(^1^11As an example, IKEA provides an app that allows users to scan rooms. The captured data is uploaded to IKEA’s servers. Hence, IKEA recommends to physically remove private parts before scanning. In contrast, the systems envisioned in this paper allow to perform the removal virtually after the scan, which is more practical.). The privacy implications of sharing 3D maps naturally leads to the question whether current approaches are able to remove objects completely or whether they leave traces that can be used to determine which objects were removed.

This paper is dedicated to this privacy aspect of (mask and language-based) editing of trainable scene representations.
We focus on removing objects from scenes and investigating whether state-of-the-art removal methods leave residuals that enable us to reason about what content was removed.
Contrary to previous works Cen et al. (2023a); Choi et al. (2024b); Mirzaei et al. (2023) that focus on foreground / background segmentation, we are interested in whether the residuals of the objects remain in the scene after removal and whether the residuals can be reasoned over (see Fig. [1](https://arxiv.org/html/2508.11431v1#S1.F1)).
To the best of our knowledge, we are the first to consider this aspect of trainable scene representations.
Hence, we introduce an evaluation framework
to quantitatively measure how well current state-of-the-art methods remove objects from scenes.
We propose metrics that measure how well an object is recognized after removal using off-the-shelf semantic segmentation at various granularity levels (object semantics ([1](https://arxiv.org/html/2508.11431v1#S3.E1)), object parts ([2](https://arxiv.org/html/2508.11431v1#S3.E2)), and instances ([3](https://arxiv.org/html/2508.11431v1#S3.E3))) and using depth maps ([4](https://arxiv.org/html/2508.11431v1#S3.E4)). Experiments on indoor and outdoor scenes show that the proposed metrics are consistent in their ranking of the evaluated methods.

To complement our evaluation, we introduce a new dataset, Remove360, comprising diverse scenes in which objects were deliberately placed and subsequently removed.
Remove360 includes real images of each scene both before and after object removal, with the post-removal images and corresponding object masks serving as ground truth.
Unlike existing datasets such as 360-USID Wu et al. (2025), which is designed around single object-centric removals, with carefully staged captures, Remove360 focuses on more naturalistic, multi-object, and interaction-rich scenes that better reflect real-world challenges.
This setup enables a more direct and realistic assessment of removal performance.
Initial experiments show that current state-of-the-art 3D removal methods often fail to generalize to Remove360, underscoring the open challenges and the relevance of this benchmark for future research.

In summary, our contributions are as follows:
i) We propose an evaluation that measures how well scene removal operations remove objects in the context of privacy. To the best of our knowledge, this is the first work to explore this aspect;
ii) We define quantitative metrics that support this evaluation and demonstrate their consistency and reliability across state-of-the-art methods for trainable 3D scenes;
iii) We introduce a new dataset of real-world indoor and outdoor scenes with pre-/post-removal images, and masks of the removed objects.
The dataset reveals failure cases in state-of-the-art methods—such as residual artifacts, incomplete removal, and over-smoothing—that are not exposed in existing benchmarks.
These challenges make it a valuable resource for advancing research on robust, privacy-aware scene editing.

Figure: Figure 1: Detecting (semantic) traces left behind after removing an object from a 3DGS reconstruction. When there remain residuals of the object, and they can be reasoned over, the object removal is imperfect. We measure the presence of residuals with off-the-shelf semantic models and in with depth. Top-Bottom: 3DGS Kerbl et al. (2023) scene before and after table removal. Left-Right: RGB renderings, SAM Kirillov et al. (2023) masks overlay with pseudo-ground-truth object outline, GroundedSAM Kirillov et al. (2023); Liu et al. (2023); Ren et al. (2024) overlay, depth renderings.
Refer to caption: x1.png

## 2 Related Work

3D reconstruction from images  builds 3D models that capture the scene’s geometry and appearance.
Popular scene representations are point clouds Schonberger and Frahm (2016); Yunhan Yang and Liu (2023); Huang et al. (2024); Peng et al. (2023); Liu et al. (2024a); Yin et al. (2024), meshes Schönberger et al. (2016); Furukawa and Ponce (2009); Lazebnik et al. (2001); Kundu et al. (2020), and, more recently, Neural Radiance Fields (NeRFs) Mildenhall et al. (2020); Barron et al. (2022); Reiser et al. (2021); Müller et al. (2022); Chen et al. (2024a); Kulhanek and Sattler (2023); Martin-Brualla et al. (2021) and 3D Gaussian Splatting (3DGS) Kerbl et al. (2023); Lin et al. (2024); Yu et al. (2024); Zhang et al. (2024); Kulhanek et al. (2024); Chen et al. (2024c); Wang et al. (2024).

NeRFs represent the scene implicitly with a colored volumetric field:
for each 3D point in the space, an MLP outputs a volumetric density and a view-dependent color value.
NeRFs are optimized for novel-view synthesis, *i.e*., so that a rendering of the field from a given view is similar to the original image from that view.
In 3DGS Kerbl et al. (2023), the scene is represented explicitly with a set of 3D Gaussians with learnable parameters (positions, orientation, scale, opacity, view-dependent color).
Again, the parameters are optimized so that splatting, *i.e*. rendering, the Gaussians generates a rendering that is similar to the original view.

Linking 3D reconstructions and semantics.
NeRFs and 3DGS can easily be extended to embed semantic features that can be prompted via natural language Kerr et al. (2023); Tschernezki et al. (2022), pixel locations Kerr et al. (2023); Tschernezki et al. (2022) or semantic labels Ye et al. (2024a).
The prompt
allows humans to search for elements in the 3D reconstruction then edit that location.
The type of prompt and the edition operation depend on the 3D representation and the embedded semantic features.

In NeRFs, the MLP is extended to output semantic features that are rendered in a differential manner (as for the color) and supervised with off-the-shelf 2D semantic features.
Examples are NeRFs extended with text features Wang et al. (2022); Mirzaei et al. (2022) like CLIP Radford et al. (2021), text and semantic features Kerr et al. (2023); Kobayashi et al. (2022), and unsupervised features Tschernezki et al. (2022) like DINO Caron et al. (2021); Oquab et al. (2024).
Prompting for an element in the reconstruction amounts to finding the features in the NeRF field that are the most similar to the prompt’s features.
The feature field can also be trained to render segmentation masks consistent with 2D masks Zhi et al. (2021); Vora* et al. (2022); Cen et al. (2023b); Liu et al. (2024b) derived with semantic models Chen et al. (2017); Graham and Van der Maaten (2017); Li et al. (2022), panoptic models Siddiqui et al. (2023); Bhalgat et al. (2024), foundation models Ravi et al. (2024); Kirillov et al. (2023); Zou et al. (2023); Li et al. (2024); Liu et al. (2022), or user annotations Mirzaei et al. (2023).
The prompt search then looks for the features associated with a given semantic label.

Although these methods perform well in locating prompted elements in the reconstruction, the implicit nature of NeRFs makes it complex to associate a location in the reconstruction with a set of abstract MLP parameters to edit.
Hence, editing the reconstruction is not as straightforward as in 3DGS Kerbl et al. (2023) representations that are explicit: removing a prompted location amounts to deleting the 3D Gaussians at that location.
3DGS can also be embedded with semantic features as easily as NeRFs with semantic models Cen et al. (2023a); Choi et al. (2024b); Jain et al. (2024); Wu et al. (2024b); Gu et al. (2024); Zhou et al. (2024); Wu et al. (2025), text features Shi et al. (2024); Liao et al. (2024); Qin et al. (2024), and unsupervised features Zuo et al. (2024).
This motivates this paper to focus on 3D reconstructions represented with 3DGS Kerbl et al. (2023).

Evaluating 3D reconstruction approaches.
3D reconstructions are evaluated based on the accuracy of the 3D geometry and whether the color renderings and the semantic (features) renderings are similar to those in the training views.

Often, scene operations, such as editing and removal, are reported only as illustrative examples with qualitative results Ye et al. (2024a); Zhou et al. (2024); Gu et al. (2024); Wu et al. (2024b); Jain et al. (2024); Choi et al. (2024b).
Some works Cen et al. (2023a); Choi et al. (2024b); Mirzaei et al. (2023) report a quantitative evaluation of ‘background’ removal by comparing the renderings of the searched element against its appearance in the original view.
This paper addresses a different problem, *i.e*., are there residuals of the removed object in the scene and if so, can they still be recognized or reasoned over?
We propose an evaluation framework to answer this question in a quantitative and automatic manner.
To the best of our knowledge, there is no previous work that addresses such a question.

Privacy challenges.
The use of extensive public data in the recent scientific breakthroughs Rombach et al. (2022); Saharia et al. (2022); Brooks et al. (2024); Achiam et al. (2023) has drawn attention to the protection of user data in the research community Raina et al. (2023); Speciale et al. (2019); Pittaluga et al. (2019); Chelani et al. (2023); Moon et al. (2024); Nasr et al. (2023), in companies Rubinstein and Good (2013); Grynbaum and Mac (2023), and governments Illman and Temple (2019); Voigt and Von dem Bussche (2017).

This challenge will grow as the deployment in households of new types of sensors, *e.g*., Augmented Reality / Virtual Reality (AR / VR) glasses spe; ari; xre, and autonomous systems will become the norm.
An efficient way to make systems privacy-preserving is to consume data that has already been made privacy-preserving by the user.
One relevant line of work proposes to anonymize the images Liu et al. (2024c); Weder et al. (2023) with inpainting before the scene reconstruction rather than editing the reconstruction later.

However, this method is more computationally complex: it involves editing many images as opposed to a single reconstruction and can introduce artifacts in the image that reduce the quality of the reconstruction.
Hence, operating on the 3D model offers privacy at a reasonable computational cost and better reconstruction quality.

## 3 Metrics Definition

We evaluate whether removal operations in 3DGS Kerbl et al. (2023) leave information about the removed content.
The proposed evaluation is conducted in the context of privacy considerations raised when sharing the scene representations.
Within this context, an element is considered private if it can not be identified Illman and Temple (2019); Voigt and Von dem Bussche (2017); Raina et al. (2023).
This section defines the metrics that evaluate whether elements can be identified after removal.
Since our evaluation metrics rely on the target object region, we assume the ground truth mask is available.
We specifically aim to assess changes occurring within this region.

### 3.1 Semantic Object Recognition

Semantic segmentation identifies objects in the scene by classifying each pixel into semantic categories Chen et al. (2017); Graham and Van der Maaten (2017); Li et al. (2022).
Here, it is used to assess whether an object can be identified by rendering the scene from multiple views and measuring the performance of the segmentation model on those renderings (see Fig. [2](https://arxiv.org/html/2508.11431v1#S3.F2)).
Comparing the segmentation performance before and after removal provides information on the removal quality.
A drop in performance indicates that the object is removed.
We thus define the semantic recognition metric as the segmentation’s performance gap on the renderings before and after removal.
The semantic segmentation is evaluated with the standard Intersection over Union (IoU) Chen et al. (2017); Badrinarayanan et al. (2017) that measures how well the estimated semantic mask overlaps with the ground-truth mask.

We compute IoU${}_{\text{pre}}$ and IoU${}_{\text{post}}$ on the semantic segmentation masks estimated on the renderings before and after removal, respectively.
To reduce the impact of false positives from the off-the-shelf segmentation model, we consider only predicted masks that overlap with the ground truth object mask.
The IoU drop IoU${}_{\text{drop}}$ is defined as:

$$ $\text{IoU}_{\text{drop}}=\text{IoU}_{\text{pre}}-\text{IoU}_{\text{post}},$ (1) $$

ranging from -1 to 1 and the higher, the better the object is removed, as indicated by the $\uparrow$.

Low absolute value of the $\text{IoU}_{\text{drop}}$ implies that $\text{IoU}_{\text{post}}=\text{IoU}_{\text{pre}}$, which can be interpreted in two ways.
(1) Both $\text{IoU}_{\text{post,pre}}$ are high, *i.e*., the segmentation model identifies the object both before and after removal, so the removal failed.
(2) When both IoUs are low, no conclusion can be drawn since the segmentation model could not segment the object even in the original image.
Thus, there is no performance drop to measure.

To address the ambiguity between these two interpretations, we complete this metric with another semantic metric defined in the next section.
Still, $\text{IoU}_{\text{drop}}$ is relevant on its own to signal an issue with the removal or the evaluation, which can be useful in systems with a man in the loop, *e.g*., active labeling.
In the experiments, we also report a more intuitive metric, the performance of the segmentation after removal, and analyse its correlation $\text{IoU}_{\text{drop}}$.
We define the accuracy $\text{acc}_{\text{seg}}$ as the ratio of images after removal in which the semantic element is not recognized anymore.
The element is not recognized if IoU${}_{\text{post}}$ is smaller than a given threshold $\xi_{\text{IoU}}$, and we report this metric over multiple thresholds (see Supp. Tab. [6(b)](https://arxiv.org/html/2508.11431v1#A2.T6.st2), [9](https://arxiv.org/html/2508.11431v1#A3.T9)).

$\text{acc}_{\text{seg}}$ ranges from 0 to $1$ and the higher $\text{acc}_{\text{seg}}$, the better the object removal, as indicated by the $\uparrow$.

$$ $\text{acc}_{\text{seg},\xi_{\text{IoU}}}=\frac{\|\text{\# images with }\text{IoU}_{\text{post}}<\xi_{\text{IoU}}\|}{\|\text{\# images }\|}$ (2) $$

Figure: Figure 2: Semantic segmentation changes before and after removal, Remove360 dataset. Left-right: GroundedSAM2 Kirillov et al. (2023); Liu et al. (2023); Ren et al. (2024) overlay on the rendering before removal, rendering after removal, overlay after removal. These semantic masks are used to calculate change in semantic segmentation in ([1](https://arxiv.org/html/2508.11431v1#S3.E1)) and its accuracy ([2](https://arxiv.org/html/2508.11431v1#S3.E2)). Rows: Different object removals in dataset Remove360. Even though the object can not be recognized by a human, the segmentation model still finds it. One explanation can be that the pixel distribution on the edited area still exhibits patterns characteristic of the object, similar to what occurs in adversarial attacks.
Refer to caption: x2.png

### 3.2 Anything Recognition

Figure: Figure 3: SAM Kirillov et al. (2023); Ravi et al. (2024) masks comparison. When the object is removed, the SAM masks change, which is the signal we exploit to evaluate the removal. When comparing SAM masks with ground truth SAM masks after removal smaller changes indicate better removal. SAM masks are used for evaluation, to calculate the similarity score ([3](https://arxiv.org/html/2508.11431v1#S3.E3)). Left-right: SAM Kirillov et al. (2023); Ravi et al. (2024) overlay on the rendering before removal, after removal, and ground truth novel view with ground truth object mask (green outline) from Remove360 dataset.
Refer to caption: x3.png

The previous sections defined the $\text{IoU}_{\text{drop}}$ that can be interpreted in two ways when it is low.
To address such ambiguity, we complement the IoU${}_{\text{drop}}$ with a second semantic metric based on finer segmentations, *i.e*., object parts or instances instead of semantic categories.
These segmentations are derived with the foundational SegmentAnything (SAM) Kirillov et al. (2023); Ravi et al. (2024), a prompt-based segmentation model that can be prompted with image locations, bounding boxes, or texts.
The model can also operate without prompts, which results in a set of masks that cover all the semantic elements in the image (see Fig. [3](https://arxiv.org/html/2508.11431v1#S3.F3)).

In this evaluation, we assess whether an element has been removed by comparing the SAM Kirillov et al. (2023); Ravi et al. (2024) masks of the scene renderings before and after removal (in case we do not have ground truth after removal), and the scene renderings with ground truth after removal.
When the object is removed or partially removed, SAM segments what is behind the object so the masks should change (see rows in Fig. [3](https://arxiv.org/html/2508.11431v1#S3.F3)).

We next define $\text{sim}_{\text{SAM}}$ that measures the similarity between two sets of SAM Kirillov et al. (2023); Ravi et al. (2024) masks based on how well they spatially overlap.
We first match the masks that overlap the most between the two sets.
Then $\text{sim}_{\text{SAM}}$ is the average overlap between the mask matches.
We enforce a 1-to-1 matching, *i.e*., a mask in one set is matched to at most one mask in the other.
We do so by defining that two masks match if they overlay, and if one mask gets matched to more than one, we keep the match that leads to the maximum overlay over all matches.
This is derived by solving an assignment problem that maximizes the overlay over all matches with the Hungarian algorithm Munkres (1957).

More formally, let $A=(a_{i})_{i\in[1,N]}$ and $B=(b_{j})_{j\in[1,M]}$ be the sets of comparing SAM masks,and $(a_{k},b_{k})_{k=1,K}$ be the $K$ matching masks.
The similarity between the two sets of SAM masks $A$ and $B$ is:

$$ $\text{sim}_{\text{SAM}}=\frac{\sum_{k=1}^{K}\text{IoU}(a_{k},b_{k})}{\text{max}(N,M)}$ (3) $$

$\text{sim}_{\text{SAM}}$ lies in $[0,1]$.
Based on set of comparing mask, we aim for higher $\text{sim}_{\text{SAM}}$ score, expecting masks to be more similar, or lower score, expecting masks to be less similar.
Having ground truth after removal, comparison between after removal mask with the ground truth mask should yield high score, indicating no visual difference in the rendering, as indicated by the $\uparrow$.
When comparing masks of renderings before and after removal, the score should be be lower, the less similar the masks are, hence the better the removal, as indicated by the $\downarrow$.

Note that we normalize the score with the highest number of masks $\text{max}(N,M)$ instead of the number of mask matches $K$.
We do so to account not only for the difference in overlay (in the numerator) but also for the difference in the number of masks.
To reflect the changes related to the removed object, the metric is derived only over the masks that overlay with the object with an IoU of at least $0.1$.

### 3.3 Spatial Recognition

We complete the previous metrics with one that depends only on the 3D scene before and after removal, hence increasing the robustness of the evaluation against possible errors in the segmentations.

Inspired by recent works in scene change detection Adam et al. (2022), we measure how well an object is removed based on the changes in the rendered depths before and after removal: a strong change in the depth maps indicates a change in the scene.

Hence, if the depth of the object changes enough, then the object is well removed.

More formally, we report the ratio of the object’s pixels which depth changes by more than a threshold $\xi_{\text{depth}}$.
The threshold $\xi_{\text{depth}}$ is derived automatically with Generalized Histogram Thresholding Barron (2020) on the histogram of depth differences over the whole image (see Fig. [4](https://arxiv.org/html/2508.11431v1#S3.F4)).

The depth maps are derived from the scene’s rendering before and after removal so they have consistent scales.
The defined ratio can be interpreted as the accuracy in depth change and is noted $\text{acc}_{\Delta\text{depth}}$:

$$ $\text{acc}_{\Delta\text{depth}}=\frac{\#\text{object pixel with depth change}>\xi_{\text{depth}}}{\#\text{object pixels}}$ (4) $$

The object’s pixel locations are specified by the object’s mask in the image that we assume is available.

Figure: Figure 4: Depth changes before and after removal. Left to right: rendered depth before removal, rendered depth after removal, thresholded depth difference and ground-truth outline of the object to be removed in green. This depth difference is used for evaluation in ([4](https://arxiv.org/html/2508.11431v1#S3.E4)). Results on Remove 360 showing localized changes in the depth maps, indicating not fully inaccurate object removal, somewhere over removed other under removed.
Refer to caption: x4.png

## 4 Object Removal Dataset Remove360

To facilitate the evaluation of object removal methods, we introduce Remove360, a new dataset featuring RGB images of scenes both before and after object removal, along with accurate object masks.
Unlike 360-USID Wu et al. (2025), which focuses on single-object, carefully aligned captures with one reference view, Remove360 targets complex, real-world scenarios with multiple interacting objects and rich scene context.
This makes it a more challenging and realistic benchmark for object removal.

Dataset Composition.
Remove360 contains 11 scenes: 6 indoor and 5 outdoor (see Fig. [5](https://arxiv.org/html/2508.11431v1#S4.F5)). Each scene includes:
(1.) Training views: RGB images and object masks before removal.
(2.) Testing views: RGB images of the same scene after removal, providing ground truth for novel view synthesis and residual detection.
Each scene contains between 150 and 300 training views, with a comparable number of testing views.
The number of removed objects per scene ranges a single item (e.g., backpack, bicycle), to pairs (e.g., white plastic chairs), and up to several objects (e.g., pillows or multiple chairs in a conference room).
The removed objects cover a wide range of physical characteristics, varying in size—from small (e.g., toy truck, backpack) to large (e.g., sofa, table)—and shape, from compact (e.g., backpack, playhouse) to complex shapes (e.g., deckchair, bicycle).
Object masks were initially generated using SAM Kirillov et al. (2023) and subsequently refined by manual annotation.
To measure the impact of inaccuracies in the ground truth masks on the results of the evaluation process, we performed a mask erosion and dilation analysis.
The analysis validated that the evaluation truly focuses on removal fidelity and is robust to small boundary inaccuracies (see Supp. [A.3](https://arxiv.org/html/2508.11431v1#A1.SS3)).

Dataset Collection Protocol.
We recorded 4K 60fps videos using the Insta360 AcePro camera. Each scene was captured over 4–6 minutes.
We selected the sharpest frame per second using the variance of the Laplacian method.
Camera poses were recovered using the hloc structure-from-motion pipeline, with SuperPoint and LightGlue for feature matching (see Supp. [A.2](https://arxiv.org/html/2508.11431v1#A1.SS2)).

Figure: Figure 5: Overview of the Remove360 dataset. Sample images from each of the 11 scenes in Remove360, comprising six indoor and five outdoor environments. Scenes vary in object count, spatial layout, and the degree of interaction between objects and their surroundings. Removed objects are highlighted with bounding boxes.
Refer to caption: x5.png

## 5 Experiments

Methods.
We evaluate five publicly available methods for object removal.
To ensure fair comparison, no additional inpainting or refinement is applied after removal.
We focus on Gaussian Splatting due to its explicit and interpretable 3D representation, which allows direct manipulation and evaluation of individual scene components.
However, our evaluation framework is not limited to Gaussian Splatting—any 3D representation can be evaluated as long as renderings and depth maps before and after removal are available.

Feature3DGS (FGS) Zhou et al. (2024) distills the LSEG Li et al. (2022) semantic features aligned with CLIP’s text features Radford et al. (2021).
FGS is prompted with a tuple of text entries: one positive query is associated with the object of interest and the others are negative queries.
The search compares the Gaussians’ feature with the features of each text entry, and their similarity is normalized.

GaussianGrouping (GG) Ye et al. (2024a) distills SAM Kirillov et al. (2023) features that operate at a finer granularity than LSEG Li et al. (2024).
Also, GG enforces spatial consistency between semantically similar Gaussians so that close-by Gaussians have similar features.
A Gaussian is removed if its feature is associated with the prompted instance label.
Post-processing then removes all Gaussians within the convex hull of the removed Gaussians.

SAGS Hu et al. (2024) is a training- and feature-free method that removes Gaussians based on their projection overlap with 2D object masks across views.
It estimates a removal probability for each Gaussian.
The 3D center of the Gaussian is projected on the images and the removal probability is the ratio of images in which the projections land on the object’s location.
When assigning Gaussians to the object does not account for the Gaussian’s opacity, which may lead to over-removal.

GaussianCut (GC) Jain et al. (2024) leverages the spatial and color correlations between Gaussians.
It models a trained 3DGS Kerbl et al. (2023) scene as a graph of Gaussians and removes them through graph-cut optimization using 2D object mask prompts, without features or training.
The removal probability value is initialized by lifting the 2D prompt mask to 3D and refined via graph-cut optimization where the unary term represents the likelihood of the Gaussian to be removed and the binary term measures the color similarity and spatial distance between two Gaussians.

AuraFusion360 (AF) Wu et al. (2025) is a training-based method using multi-view RGB images and object masks.
It removes objects using depth-aware mask generation to handle occlusions.

Datasets.
Methods are evaluated on our Remove360 dataset and the Mip-NeRF360 dataset Barron et al. (2022).
Since Mip-NeRF360 Barron et al. (2022) lacks semantic masks, we generate pseudo-ground-truth masks using SAM Kirillov et al. (2023) and human annotations (see Supp. [A.3](https://arxiv.org/html/2508.11431v1#A1.SS3)).
We use the Mip-NeRF360 dataset as it has been commonly used by the approaches we consider in this work.

Implementation details.
Due to memory constraints, we use 100–150 images per used scene from Mip-NeRF360 (kitchen, counter, room, garden), which is sufficient per prior studies Jain et al. (2024); Zhou et al. (2024).
All methods are trained on the same image subsets and 3D point clouds reconstructed using Remove360 scenes are used in full.
For evaluation, we apply GroundedSAM2 Kirillov et al. (2023); Liu et al. (2023); Ren et al. (2024) to compute metrics based on semantic segmentation, and SAM Kirillov et al. (2023) for instance segmentation metrics.

**(a) Remove360 dataset evaluation results.**
| Scene- | IoU${}_{\text{drop}}\uparrow$ | acc${}_{\text{seg, IoU${}_{post}<0.5$}}\uparrow$ | acc${}_{\Delta\text{depth}}\uparrow$ | sim${}_{\text{SAM}}\uparrow$ |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Object | FGS | GG | SAGS | GC | AF | FGS | GG | SAGS | GC | AF | FGS | GG | SAGS | GC | AF | FGS | GG | SAGS | GC | AF |
| Backyard- Deckchair | * | * | * | 0.85 | 0.84 | * | * | * | 0.99 | 0.99 | * | * | * | 0.67 | 0.65 | * | * | * | 0.56 | 0.54 |
| Backyard- Chairs | * | * | * | 0.85 | 0.87 | * | * | * | 1.00 | 1.00 | * | * | * | 0.76 | 0.67 | * | * | * | 0.83 | 0.62 |
| Backyard- Stroller | * | * | * | 0.92 | 0.91 | * | * | * | 1.00 | 1.00 | * | * | * | 0.89 | 0.73 | * | * | * | 0.85 | 0.72 |
| Backyard- Playhouse | * | * | * | 0.95 | 0.97 | * | * | * | 1.00 | 1.00 | * | * | * | 0.92 | 0.87 | * | * | * | 0.50 | 0.49 |
| Backyard- Toy Truck | * | * | * | 0.95 | 0.93 | * | * | * | 0.99 | 0.98 | * | * | * | 0.73 | 0.64 | * | * | * | 0.22 | 0.20 |
| Bedroom- Table | * | * | * | 0.91 | 0.91 | * | * | * | 0.98 | 1.00 | * | * | * | 0.57 | 0.58 | * | * | * | 0.48 | 0.44 |
| Living Room- Pillows | * | * | * | 0.62 | 0.76 | * | * | * | 0.77 | 0.88 | * | * | * | 0.53 | 0.51 | * | * | * | 0.19 | 0.18 |
| Living Room- Sofa | * | * | * | 0.57 | 0.62 | * | * | * | 0.50 | 0.64 | * | * | * | 0.62 | 0.62 | * | * | * | 0.17 | 0.13 |
| Office- Chairs | * | * | * | 0.69 | 0.64 | * | * | * | 0.85 | 0.76 | * | * | * | 0.91 | 0.82 | * | * | * | 0.34 | 0.33 |
| Park- Bicycle | * | * | * | 0.95 | 0.95 | * | * | * | 0.99 | 1.00 | * | * | * | 0.91 | 0.80 | * | * | * | 0.68 | 0.48 |
| Stairwell- Backpack | * | * | * | 0.89 | 0.82 | * | * | * | 0.93 | 0.85 | * | * | * | 0.73 | 0.65 | * | * | * | 0.37 | 0.37 |

### 5.1 Results

Methods comparison.
Tab. [1](https://arxiv.org/html/2508.11431v1#S5.T1) reports the considered approaches under our proposed metrics.
While methods perform well overall, closer inspection and visualizations show persistent semantic residuals, indicating that current removal methods remain imperfect (see Supp. [A.1](https://arxiv.org/html/2508.11431v1#A1.SS1)).

On Remove360, GaussianCut (GC) Jain et al. (2024) outperforms AuraFusion (AF)Wu et al. (2025), particularly in instance segmentation similarity (simSAM), suggesting more accurate and complete object removal.
AF benefits from training on multi-view masks, however, this characteristic did not translate to ideal performance, where removing objects using AF resulted in less similar instance segmentations after the removal, lower simSAM.
IoUdrop, simSAM, and accseg appear to correlate, confirming that greater semantic change and segmentation similarity with ground truth novel view after removal, results in better removal.
However, some scenes (e.g., living room, office) still show residual traces, and depth accuracy (acc${}_{\Delta\text{depth}}$) remains low—indicating limited depth modification.
Notably, several methods Ye et al. (2024a); Zhou et al. (2024); Hu et al. (2024) fail on Remove360, unable to detect removed objects, showcasing its difficulty.
This suggests that these methods might not be altering the depth information of the removed objects as effectively as they are altering the semantic and instance segmentation.
On the Remove360, methods such as Jain et al. (2024); Wu et al. (2025) are trained and evaluated on the full Remove360 dataset.
In contrast, other methods Ye et al. (2024a); Zhou et al. (2024); Hu et al. (2024) could not be evaluated on Remove360, as they failed to learn meaningful object representations and were thus unable to detect removed objects.
As a result, their evaluation metrics remain identical before and after object removal, indicating failure to detect changes in Tab. [1](https://arxiv.org/html/2508.11431v1#S5.T1).

For the Mip-Nerf360 Barron et al. (2022), the methods’ ranks remain stable across the three metrics:
AF Wu et al. (2025), GC Jain et al. (2024), and GG Ye et al. (2024a) lead across metrics.
In contrast, FGS Zhou et al. (2024) under performs, likely due to prompt sensitivity, and SAGS Hu et al. (2024) shows high variance, performing better in spatially distinct objects-centric scenes (garden, kitchen).
Mip-NeRF360 Barron et al. (2022) lacks post-removal ground truth, so only sim${}_{\text{SAM}}$ between pre- and post-removal renderings is available, therefore we expect to see lower similarity.

Importantly, Remove360 introduces real-world challenges with paired pre/post-removal images and masks, enabling direct measurement of semantic residuals and post-removal segmentation.
Unlike MipNeRF360, it supports ground-truth-based evaluation and reveals generalization gaps, offering a more rigorous benchmark for future methods.

Qualitative Results.
Figs. [2](https://arxiv.org/html/2508.11431v1#S3.F2), [3](https://arxiv.org/html/2508.11431v1#S3.F3), and [4](https://arxiv.org/html/2508.11431v1#S3.F4) show renderings of the evaluated methods before and after removal.
Fig. [2](https://arxiv.org/html/2508.11431v1#S3.F2) illustrates an interesting case where the object is not visible to humans anymore, yet GroundedSAM2 Kirillov et al. (2023); Liu et al. (2023); Ren et al. (2024) finds the object.
Our dataset was not available until now, and thus is not part of GroundedSAM2’s training data. This suggests that barely visible information about the object can remain in the scene, even when the removal is successful to the human eye, and that the proposed metrics can detect such scenarios.
This opens interesting future directions on whether a network could be trained to invert the object removal from invisible pixel information and how to prevent it.

Fig. [3](https://arxiv.org/html/2508.11431v1#S3.F3) shows the distribution of SAM Kirillov et al. (2023) masks on the rendering before and after removal, compared to ground truth instance segmentation after removal.
It provides a visual intuition on how sim${}_{\text{SAM}}$ behaves: successful ’sofa’ removal should reveal new segments behind it.
Fig. [4](https://arxiv.org/html/2508.11431v1#S3.F4) provides a visual intuition for acc${}_{\Delta\text{depth}}$.
Successful removal causes depth differences localized in the object area (object mask outlined in green).
More visualizations and quantitative results are provided in Supplementary (for Remove360 see [B](https://arxiv.org/html/2508.11431v1#A2), and for Mip-NERF360 see [C](https://arxiv.org/html/2508.11431v1#A3)).

Limitations.
The metrics rely on off-the-shelf semantic segmentation models that can introduce errors.
Although introducing redundancy between the metrics alleviates this issue, it does not fully address it, which calls for further research on robust metrics for the evaluation of object removal.

## 6 Conclusion

We introduce a novel evaluation framework for assessing object removal in 3D Gaussian Splatting, targeting privacy-preserving scene representations.
Our metrics combine off-the-shelf semantic models and depth reasoning to quantify whether removed objects leave detectable residuals.
Experiments on state-of-the-art methods reveal persistent semantic traces, underscoring key limitations in current approaches.
To enable rigorous, ground-truth-based evaluation, we release Remove360, a challenging real-world dataset with paired pre- and post-removal images and object masks.

We hope this work lays the foundation for future research in privacy-preserving 3D scene manipulation, where removal operations leave no recoverable trace.

## Acknowledgements

This work was supported by
the Czech Science Foundation (GACR) EXPRO (grant no. 23-07973X),
the Ministry of Education, Youth and Sports of the Czech Republic through the e-INFRA CZ (ID:90254),
the Grant Agency of the Czech Technical University in Prague (grant no. SGS25/060/OHK3/1T/13).

Supplementary Material

The supplementary material provides additional implementation details, evaluation metrics, and extended experimental results supporting the main paper.

Section A describes the implementation details.
Subsection A.1 presents the parameter settings for the compared methods: Feature3DGS Zhou et al. (2024), SAGS Hu et al. (2024), GaussianGrouping Ye et al. (2024a), Gaussian Cut Jain et al. (2024), and Aura Fusion Wu et al. (2025).
Subsection A.2 describes dataset processing protocol used for Remove360.
Subsection A.3 describes how the pseudo-groung-truth masks for Mip-NERF360 were obtained.
Subsection A.4 describes the evaluation metrics.

Section B presents additional results for the Remove360 Dataset.
Subsection B.1 presents quantitative results.
Subsection B.2 presents qualitative results.

Section C presents additional results for the Mip-NERF360 Dataset.
Subsection C.1 presents quantitative results.
Subsection C.2 presents qualitative results.

## Appendix A Implementation Details

### A.1 Compared Methods

Official implementations provided by the respective authors are used with the following settings.

Feature3DGS (FGS) Zhou et al. (2024)
is prompted with a tuple of text entries: one positive text prompt is associated with the object of interest and the others are negative text prompts.
The search compares the Gaussians’ feature with the features of each text entry, and their similarity is normalized with softmax.
A Gaussian is removed if the similarity between the Gaussian’s feature and the prompt feature is higher than a threshold.
We set this similarity threshold to $0.4$ for all scenes and objects.

The following negative prompts are used in Mip-NERF360, per scene:
Garden: {grass, sidewalk, tree, house},
Room: {sofa, rug, television, floor},
Kitchen: {rug, table, chair},
Counter: {oranges, wooden rolling pin, coconut oil}.

The following negative prompts are used in Remove360, per scene:
Backyard- Deckchair : {tree, grass, sky}
Backyard- Chairs : {house, sidewalk, grass, plant, sky}
Backyard- Stroller : {sidewalk, grass, plant, bench}
Backyard- Playhouse : {tree, grass, plant, sky}
Backyard- Toy Truck : {fence, grass, plant, dirt}
Bedroom- Table : {cabinet, floor, bed, wall}
Living Room- Pillows : {sofa, armchair, wall, rug, curtain}
Living Room- Sofa : {plant, wall, floor, curtain, shelf}
Office- Chairs : {table, floor, window, wall}
Park- Bicycle : {sidewalk, road, plant, sky, wall}
Stairwell- Backpack : {stairs, wall, ceiling}

SAGS Hu et al. (2024)
is a training-free and feature-free method, taking object masks as prompts.
It estimates a removal likelihood for each Gaussian based on projective geometry.
The 3D center of the Gaussian is projected on the images and the removal probability is the fraction of images in which the projections land in the object mask.
The object masks therefore need to be available.
A Gaussian is removed if its removal likelihood is higher than $0.7$.

Gaussian Grouping Ye et al. (2024a)
takes SAM Kirillov et al. (2023) features and use them to assign a label for each Gaussian.
After training, Gaussian is removed if its label is equal to selected label in config file for each scene.
For the label training, SAM IoU prediction threshold is set to $0.8$.
For Gaussian training, the default settings are used, $\text{densify_until_iter}=10000$, $\text{num_classes}=256$, $\text{reg3d_interval}=5$, $\text{reg3d_k}=5$, $\text{reg3d_lambda_val}=2$, $\text{reg3d_max_points}=200000$, $\text{reg3d_sample_size}=1000$.
For the object removal setting, the default number of classes of $256$ and the removal threshold of $0.3$ are used.

Gaussian Cut Jain et al. (2024)
is a feature-free and training-free method that leverages the spatial and color correlations between Gaussians.
Given a trained 3DGS Kerbl et al. (2023) scene, it models the scene as a graph and determines which Gaussians should be removed via graph optimization.
As for SAGS, the prompt is a set of object masks.
The Gaussians define the nodes of the graph and are extended with a single parameter representing the probability of the Gaussian to be removed.
The parameter is initialized by lifting the 2D prompt mask to 3D and refined via graph-cut optimization where the unary term represents the likelihood of the Gaussian to be removed and the binary term measures the color similarity and spatial distance between two Gaussians.
The graph is built with the following parameters:
each Gaussian is connected to its 10 nearest neighbors in 3D space- ’number of edges’ per node is 10).
The ’terminal clusters’ define how foreground (source) and background (sink) labels are seeded in the graph-cut.
Specifically, setting the ’terminal cluster source’ = 5 and ’terminal cluster sink’ = 5 mean that 5 clusters of Gaussians (likely foreground and background respectively) are selected to initialize the optimization.
The ’leaf size’ = 40 controls the granularity of the spatial clustering used to construct the graph efficiently.
A ’foreground threshold’ of 0.9 is applied when Gaussians connected the the object mask are visible in at least 90% of the masked views are considered for removal.
The prompt is a set of multi-view masks associated with the object to be removed.

Aura Fusion Wu et al. (2025)
jointly fuses 2D semantic masks with the 3D Gaussian representation.
During training, it is supervised by either ground-truth masks or pseudo-ground-truth masks depending on the dataset.
Training runs for 20,000 iterations and the object masks used as supervision are dilated with a kernel size of 10.
The model learns to predict an object removal confidence for each Gaussian.
To better handle occlusions and capture shape priors, a diffusion depth module is used to propagate 2D mask information into the 3D scene along view-dependent depth directions.
At inference time, a Gaussian is removed if its predicted removal confidence exceeds a threshold of $0.6$.
An additional unseen object threshold of $0.0$ is used to control background filtering, ensuring that only Gaussians with non-zero predicted relevance to the object are considered for removal.

### A.2 Remove360 Dataset Processing Protocol

Camera Pose Estimation.
Camera poses and sparse scene geometry are reconstructed using the Hierarchical Localization Sarlin et al. (2019, 2020) (hLoc) pipeline.
The steps are as follows:

1. Global Feature Extraction: Global descriptors are extracted using NetVLAD Arandjelović et al. (2016).

2. Local Feature Extraction and Matching: Local features are extracted using SuperPoint DeTone et al. (2018) (Aachen configuration) and matched using LightGlue Lindenberger et al. (2023) under the superpoint+lightglue configuration.

3. Image Pair Selection: Sequential image pairs are generated with a temporal overlap of 10 frames, with enabled quadratic overlap to match frames at exponentially increasing intervals.
Loop closure detection is performed every 5th frame by retrieving the top 20 most similar images based on NetVLAD Arandjelović et al. (2016) descriptors.

4. Structure-from-Motion (SfM) Reconstruction: Sparse reconstruction is performed using COLMAP Schönberger and Frahm (2016); Schönberger et al. (2016) via the pycolmap interface.
The RADIAL camera model is used.
Camera parameters are used to undistort the input images, and both distorted and undistorted reconstructions are retained.

### A.3 Object Masks

Pseudo-Ground-Truth Object Masks for Mip-NERF360.
Mip-NeRF360 Barron et al. (2022) does not provide ground-truth semantic masks necessary for our evaluation.
To address this, pseudo-ground-truth masks are generated by applying SAM Kirillov et al. (2023) to each image, segmenting all objects, and selecting the masks corresponding to the target objects.
When an object is covered by multiple overlapping segments, all relevant segments are combined to fully capture the object.
These masks are sufficiently accurate for evaluation purposes.
To ensure reliability, cases with incomplete segmentation from SAM are excluded from evaluation.
However, such cases are rare.

Ground-Truth Object Masks for Remove360.
Were initialized semantic masks using SAM, and then manually verified all masks, merging oversegmented regions.
When needed, manually verified and refined by adding/removing pixels.
Fewer than 10 images per scene (of up to 300) required edits.
We estimate that the segmentations are accurate up to a pixel or two at the boundaries.
No parts are missing and no unrelated parts are included in the masks that are used as ground truth.
To measure the impact of inaccuracies in the ground truth masks on the results of the evaluation process, we performed a mask erosion and dilation analysis, see Tab. [2](https://arxiv.org/html/2508.11431v1#A1.T2), [3](https://arxiv.org/html/2508.11431v1#A1.T3), [4](https://arxiv.org/html/2508.11431v1#A1.T4)
We used OpenCV’s Bradski (2000) morphological operation with an elliptical kernel.
This expands or shrinks the masks by ±5, ±10, or ±15 pixels.
The elliptical kernel avoids unrealistic boxy boundaries and approximates object contours more faithfully than a rectangular structuring element.
Note that in our experience, the masks are more accurate than 5 pixels, i.e. the inaccuracies we observed are below 5 pixels on the boundary.

Dilation simulates potential over-segmentation or imprecise labeling, and allows us to measure how our metrics behave when neighboring context is included. As our results show, the metrics (IoU${}_{\text{drop}}$ Tab. [2](https://arxiv.org/html/2508.11431v1#A1.T2), acc${}_{\Delta\text{depth}}$ Tab. [3](https://arxiv.org/html/2508.11431v1#A1.T3), sim${}_{\text{SAM}}$ Tab. [4](https://arxiv.org/html/2508.11431v1#A1.T4)) gradually degrade under dilation, indicating that they are sensitive to spatial precision but not overly affected by nearby irrelevant regions.

Conversely, erosion (shrinking the mask inward with the same elliptical kernel) helps isolate the object core, where removal is most likely to be clean.
The consistent improvement in scores under erosion across multiple scenes validates that our evaluation truly focuses on removal fidelity and is robust to small boundary inaccuracies.

**Table 2: IoU${}_{\text{drop}}$ results for mask erosion (E) and dilation (D) analysis, Remove360 dataset. Results show, how evaluation metric IoU${}_{\text{drop}}$ changes when the ground truth mask is eroded or dilated by 5-15 pixels compared to the original mask used for Remove360 dataset. The best value between methods is highlighted for each object metric and mask state. Erosion consistently improves scores (Deckchair: 0.85 → 0.89), suggesting residuals remain near boundaries for both methods (which are then not taken into account when eroding the ground truth masks). Dilation lowers scores (Deckchair: 0.85 → 0.66), indicating inclusion of nearby artifacts like shadows. Thin objects (chairs, bike) are more sensitive, while larger ones (sofa, playhouse) are less affected. This confirms that our evaluation is not only robust, but object-aware, capturing residual traces at a fine-grained level.**
| Scene | Object | Method | E 15 | E 10 | E 5 | Original | D 5 | D 10 | D 15 |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Backyard | Deckchair | GC | 0.89 | 0.89 | 0.88 | 0.85 | 0.81 | 0.73 | 0.66 |
|  |  | AF | 0.88 | 0.88 | 0.87 | 0.84 | 0.79 | 0.71 | 0.65 |
|  | Chairs | GC | 0.88 | 0.88 | 0.88 | 0.85 | 0.80 | 0.72 | 0.66 |
|  |  | AF | 0.89 | 0.89 | 0.89 | 0.87 | 0.82 | 0.75 | 0.69 |
|  | Stroller | GC | 0.94 | 0.94 | 0.94 | 0.92 | 0.88 | 0.84 | 0.80 |
|  |  | AF | 0.93 | 0.93 | 0.93 | 0.91 | 0.87 | 0.83 | 0.79 |
|  | Playhouse | GC | 0.95 | 0.95 | 0.95 | 0.95 | 0.93 | 0.92 | 0.90 |
|  |  | AF | 0.98 | 0.98 | 0.98 | 0.97 | 0.96 | 0.94 | 0.92 |
|  | Toy Truck | GC | 0.96 | 0.96 | 0.96 | 0.95 | 0.91 | 0.87 | 0.83 |
|  |  | AF | 0.94 | 0.94 | 0.94 | 0.93 | 0.89 | 0.85 | 0.81 |
| Bedroom | Table | GC | 0.94 | 0.93 | 0.93 | 0.91 | 0.85 | 0.82 | 0.76 |
|  |  | AF | 0.94 | 0.93 | 0.93 | 0.91 | 0.85 | 0.82 | 0.76 |
| Living Room | Pillows | GC | 0.63 | 0.63 | 0.63 | 0.62 | 0.60 | 0.57 | 0.55 |
|  |  | AF | 0.78 | 0.78 | 0.77 | 0.76 | 0.73 | 0.70 | 0.67 |
|  | Sofa | GC | 0.58 | 0.58 | 0.58 | 0.57 | 0.55 | 0.53 | 0.51 |
|  |  | AF | 0.63 | 0.63 | 0.63 | 0.62 | 0.59 | 0.57 | 0.54 |
| Office | Chairs | GC | 0.75 | 0.74 | 0.72 | 0.69 | 0.64 | 0.58 | 0.54 |
|  |  | AF | 0.71 | 0.69 | 0.67 | 0.64 | 0.59 | 0.54 | 0.50 |
| Park | Bicycle | GC | 0.97 | 0.97 | 0.96 | 0.95 | 0.91 | 0.86 | 0.82 |
|  |  | AF | 0.97 | 0.97 | 0.96 | 0.95 | 0.91 | 0.86 | 0.82 |
| Stairwell | Backpack | GC | 0.90 | 0.90 | 0.90 | 0.89 | 0.83 | 0.77 | 0.72 |
|  |  | AF | 0.84 | 0.84 | 0.84 | 0.82 | 0.77 | 0.70 | 0.65 |

**Table 3: acc${{}_{\Delta\text{depth}}}$ results for mask erosion (E) and dilation (D) analysis, Remove360 dataset. Results show, how evaluation metric acc$\Delta$depth changes when the ground truth mask is eroded or dilated by 5-15 pixels compared to the original mask used for Remove360 dataset. The best value between methods is highlighted for each object metric and mask state. Erosion slightly improves scores by focusing on core object geometry. Dilation degrades accuracy by including unmodified context. This shows our depth-based metric isolates removal-induced geometry changes and is robust to boundary noise.**
| Scene | Object | Method | E 15 | E 10 | E 5 | Original | D 5 | D 10 | D 15 |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Backyard | Deckchair | GC | 0.69 | 0.69 | 0.67 | 0.67 | 0.65 | 0.65 | 0.64 |
|  |  | AF | 0.67 | 0.66 | 0.65 | 0.65 | 0.65 | 0.59 | 0.55 |
|  | Chairs | GC | 0.77 | 0.77 | 0.76 | 0.76 | 0.76 | 0.75 | 0.75 |
|  |  | AF | 0.67 | 0.67 | 0.67 | 0.67 | 0.65 | 0.62 | 0.58 |
|  | Stroller | GC | 0.89 | 0.89 | 0.89 | 0.89 | 0.88 | 0.87 | 0.86 |
|  |  | AF | 0.72 | 0.73 | 0.73 | 0.73 | 0.72 | 0.70 | 0.67 |
|  | Playhouse | GC | 0.92 | 0.92 | 0.92 | 0.92 | 0.92 | 0.91 | 0.91 |
|  |  | AF | 0.87 | 0.87 | 0.87 | 0.87 | 0.86 | 0.85 | 0.84 |
|  | Toy Truck | GC | 0.73 | 0.73 | 0.73 | 0.73 | 0.73 | 0.72 | 0.71 |
|  |  | AF | 0.65 | 0.64 | 0.64 | 0.64 | 0.63 | 0.61 | 0.59 |
| Bedroom | Table | GC | 0.58 | 0.58 | 0.57 | 0.57 | 0.56 | 0.56 | 0.55 |
|  |  | AF | 0.58 | 0.58 | 0.58 | 0.58 | 0.57 | 0.56 | 0.55 |
| Living Room | Pillows | GC | 0.53 | 0.53 | 0.53 | 0.53 | 0.52 | 0.52 | 0.52 |
|  |  | AF | 0.51 | 0.51 | 0.51 | 0.51 | 0.51 | 0.51 | 0.50 |
|  | Sofa | GC | 0.63 | 0.62 | 0.62 | 0.62 | 0.62 | 0.62 | 0.62 |
|  |  | AF | 0.62 | 0.62 | 0.62 | 0.62 | 0.62 | 0.62 | 0.61 |
| Office | Chairs | GC | 0.94 | 0.94 | 0.93 | 0.91 | 0.90 | 0.88 | 0.87 |
|  |  | AF | 0.83 | 0.83 | 0.82 | 0.82 | 0.81 | 0.79 | 0.78 |
| Park | Bicycle | GC | 0.91 | 0.91 | 0.91 | 0.91 | 0.90 | 0.90 | 0.89 |
|  |  | AF | 0.80 | 0.80 | 0.80 | 0.80 | 0.80 | 0.78 | 0.76 |
| Stairwell | Backpack | GC | 0.73 | 0.73 | 0.73 | 0.73 | 0.72 | 0.71 | 0.71 |
|  |  | AF | 0.66 | 0.66 | 0.66 | 0.65 | 0.65 | 0.62 | 0.60 |

**Table 4: sim${}_{\text{SAM}}$ results for mask erosion (E) and dilation (D) analysis, Remove360 dataset. Results show, how evaluation metric sim${}_{\text{SAM}}$ changes when the ground truth mask is eroded or dilated by 5-15 pixels compared to the original mask used for Remove360 dataset. The best value between methods is highlighted for each object metric and mask state. Erosion sometimes improves scores (Deckchair: 0.56 → 0.65), while dilation has small, inconsistent effects. The metric is robust to mask variation, and variability across scenes highlights the need for multiple complementary metrics.**
| Scene | Object | Method | E 15 | E 10 | E 5 | Original | D 5 | D 10 | D 15 |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Backyard | Deckchair | GC | 0.65 | 0.63 | 0.60 | 0.56 | 0.52 | 0.50 | 0.49 |
|  |  | AF | 0.65 | 0.64 | 0.60 | 0.54 | 0.51 | 0.51 | 0.51 |
|  | Chairs | GC | 0.84 | 0.83 | 0.83 | 0.83 | 0.83 | 0.82 | 0.82 |
|  |  | AF | 0.67 | 0.66 | 0.65 | 0.62 | 0.61 | 0.60 | 0.58 |
|  | Stroller | GC | 0.85 | 0.85 | 0.85 | 0.85 | 0.86 | 0.86 | 0.86 |
|  |  | AF | 0.72 | 0.72 | 0.72 | 0.72 | 0.72 | 0.72 | 0.73 |
|  | Playhouse | GC | 0.50 | 0.50 | 0.50 | 0.50 | 0.50 | 0.50 | 0.50 |
|  |  | AF | 0.48 | 0.48 | 0.48 | 0.49 | 0.49 | 0.48 | 0.48 |
|  | Toy Truck | GC | 0.21 | 0.21 | 0.22 | 0.22 | 0.22 | 0.23 | 0.23 |
|  |  | AF | 0.20 | 0.19 | 0.19 | 0.20 | 0.20 | 0.20 | 0.20 |
| Bedroom | Table | GC | 0.43 | 0.44 | 0.45 | 0.48 | 0.48 | 0.48 | 0.50 |
|  |  | AF | 0.38 | 0.39 | 0.40 | 0.44 | 0.45 | 0.45 | 0.46 |
| Living Room | Pillows | GC | 0.19 | 0.19 | 0.19 | 0.19 | 0.19 | 0.19 | 0.20 |
|  |  | AF | 0.18 | 0.18 | 0.18 | 0.18 | 0.18 | 0.18 | 0.19 |
|  | Sofa | GC | 0.17 | 0.17 | 0.17 | 0.17 | 0.18 | 0.18 | 0.18 |
|  |  | AF | 0.13 | 0.13 | 0.13 | 0.13 | 0.13 | 0.13 | 0.13 |
| Office | Chairs | GC | 0.34 | 0.34 | 0.34 | 0.34 | 0.35 | 0.36 | 0.37 |
|  |  | AF | 0.32 | 0.32 | 0.33 | 0.33 | 0.33 | 0.32 | 0.32 |
| Park | Bicycle | GC | 0.68 | 0.68 | 0.68 | 0.68 | 0.68 | 0.68 | 0.68 |
|  |  | AF | 0.46 | 0.47 | 0.48 | 0.48 | 0.47 | 0.46 | 0.46 |
| Stairwell | Backpack | GC | 0.37 | 0.37 | 0.37 | 0.37 | 0.38 | 0.38 | 0.38 |
|  |  | AF | 0.37 | 0.37 | 0.37 | 0.37 | 0.37 | 0.38 | 0.38 |

### A.4 Metrics

Semantic Recognition.
If GroundedSAM2 Kirillov et al. (2023); Liu et al. (2023); Ren et al. (2024) fails to detect an object for a given prompt, no semantic mask is produced, and the semantic IoU is 0.

Complementarity Analysis of the Metrics.
The ranking of the methods between the different metrics is mostly consistent, which is expected since they are all designed to measure the removal quality (Tab. [5](https://arxiv.org/html/2508.11431v1#A2.T5), [6](https://arxiv.org/html/2508.11431v1#A2.T6), [8](https://arxiv.org/html/2508.11431v1#A3.T8), [9](https://arxiv.org/html/2508.11431v1#A3.T9)).
The presence of redundancy in the metrics makes the proposed evaluation robust to potential errors in the semantic models used in the derivation.

When a method achieves good results on all three metrics (IOU${}_{\text{drop}}$, acc${}_{\Delta\text{depth}}$, sim${}_{\text{SAM}}$), then it is very likely that the removal succeeded and that the metrics are reliable.
However, a mix of good and bad metric scores indicates that either the removal quality is low or that the segmentation used to derive IOU${}_{\text{drop}}$ and sim${}_{\text{SAM}}$ are incorrect, thus, the metric is not reliable.
For instance, low drop in object detection IOUdrop can mean either a failed removal or poor segmentation output from the segmentation model.
The other metrics help disambiguate between the two interpretations.
Cross-checking with depth difference accuracy acc${{}_{\Delta\text{depth}}}$ helps resolve this ambiguity.
This metric tends to overestimate removal quality, making lower values a stronger indicator of failure—e.g., Aura Fusion Wu et al. (2025) on ’Pillows’ (Tab. [5](https://arxiv.org/html/2508.11431v1#A2.T5), Fig. [10(a)](https://arxiv.org/html/2508.11431v1#A2.F10.sf1))
However, a higher acc${{}_{\Delta\text{depth}}}$ does not guarantee success.
For example, in Fig. [9(a)](https://arxiv.org/html/2508.11431v1#A2.F9.sf1) is shown that Gaussian Cut Jain et al. (2024) achieves 0.62 acc${}_{\Delta\text{depth}}$ on ‘Sofa’, but IOU${}_{\text{drop}}$ is low (0.57).
The third metric can disambiguate such a case: in this example of ’Sofa’ removal, the sim${}_{\text{SAM}}$ at 0.17 suggests that the removal does not perform well, as the SAM Kirillov et al. (2023) segmentation is not similar with ground truth SAM segmentation after removal.
Therefore the high acc${}_{\Delta\text{depth}}$ does indicate a high-quality removal but instead just some local editing, what we can visually confirm.
Visual inspection confirms the local edits.

This analysis demonstrates the complementary nature of the metrics.
Together, they provide robust, interpretable evaluation, especially in the absence of ground-truth post-removal data.
This is critical on datasets like Mip-NeRF360, where only pre-removal ground truth is available; in such cases, sim${}_{\text{SAM}}$ between before and after renders serves as a proxy, with lower values indicating better removal.

More quantitative and qualitative results follow in the next section.

## Appendix B Removal Results for Remove360 Dataset

### B.1 Quantitative Results After Removal

Tab. [5](https://arxiv.org/html/2508.11431v1#A2.T5), [6](https://arxiv.org/html/2508.11431v1#A2.T6) present quantitative results after removal on the Remove360 dataset.
The removal methods Gaussian Cut Jain et al. (2024) and Aura Fusion Wu et al. (2025) perform relatively similar, with advantage of Gaussian Cut Jain et al. (2024) in the semantic similarity sim${}_{\text{SAM}}$, and depth difference accuracy acc${}_{\Delta\text{depth}}$.
However, Aura Fusion Wu et al. (2025) performs better in the semantic object segmentation after removal, achieving less detections compared to Gaussian Cut Jain et al. (2024).
Having ground-truth after removal, we are able to compute PSNR between the renders after removal and ground-truth novel views.
Having visually consistent background after object removal is wanted.
A higher PSNR means the removal method preserves the visual quality better.
Slightly better results are achieved by Gaussian Cut Jain et al. (2024).
Overall low PSNR values indicate poor quality after removal, and the need of additional processing, for example through in-painting (see Fig. [7(c)](https://arxiv.org/html/2508.11431v1#A2.F7.sf3), [8(a)](https://arxiv.org/html/2508.11431v1#A2.F8.sf1), [8(b)](https://arxiv.org/html/2508.11431v1#A2.F8.sf2), [8(c)](https://arxiv.org/html/2508.11431v1#A2.F8.sf3), [9(b)](https://arxiv.org/html/2508.11431v1#A2.F9.sf2).

### B.2 Qualitative Results After Removal

Qualitative results for each scene in the Remove360 dataset are presented in Fig. [7](https://arxiv.org/html/2508.11431v1#A2.F7), [8](https://arxiv.org/html/2508.11431v1#A2.F8), [9](https://arxiv.org/html/2508.11431v1#A2.F9), [10](https://arxiv.org/html/2508.11431v1#A2.F10).
Each visualization includes SAM Kirillov et al. (2023) segmentations, and depth differences computed before and after removal using the thresholding approach described in the spatial recognition subsection of the main paper.
The segmentation similarity between the SAM segments of the ground-truth and the render after removal is reported; higher similarity scores indicate a closer match to the ground-truth and thus more successful object removal.
Additionally, the accuracy of the depth difference within the ground-truth mask is reported; higher values suggest effective removal, as changes in depth at the object’s location are expected.
Gaussian Cut Jain et al. (2024) generally produces more visually coherent results, leaving fewer artifacts and preserving scene quality more effectively than competing methods.
Among the methods, Gaussian Cut Jain et al. (2024) often produces the most visually coherent results, with fewer rendering artifacts and more realistic background compared to Aura Fusion360 Wu et al. (2025) results.

However, these visual results are not always aligned with the quantitative semantic metrics (see Tab.[5](https://arxiv.org/html/2508.11431v1#A2.T5),[6](https://arxiv.org/html/2508.11431v1#A2.T6)).
In some cases, even though the object appears to be successfully removed in the image (Fig. [6](https://arxiv.org/html/2508.11431v1#A2.F6)), the semantic segmentation metric (IoU${}_{\text{post}}$) reports relatively high object detection after removal (office chairs after removal reach mean IoU${}_{\text{post}}$ of 0.18 and 0.19, see Tab [6](https://arxiv.org/html/2508.11431v1#A2.T6)), indicating that semantic features of the removed object are still present.
This discrepancy often points to invisible or occluded Gaussians that still carry semantic cues, which the SAM model can detect even when they’re not visually obvious, see Fig. [6](https://arxiv.org/html/2508.11431v1#A2.F6).

This highlights the importance of our grounded SAM-based metric, which serves as a proxy for semantic leakage:
It can detect residual traces of the removed object that are not apparent in RGB renderings but remain in the underlying 3D representation.
Thus, even if an image looks correct to a human observer, the scene may still reveal what was removed to a machine vision system—violating the goal of effective and irrecoverable object removal.

Our metric is particularly valuable in the absence of true post-removal ground-truth labels, as it leverages the consistency and sensitivity of a strong segmentation model to detect failures that would otherwise go unnoticed.

**Table 5: Remove360: Evaluation results. These five metrics measure changes in semantics and depth before and after removal, along with image quality after removal: IoU${}_{\text{drop}}$ measures the drop in semantic segmentation after removal, acc${}_{\text{seg},\xi_{\text{IoU}}}$ measures the ratio of images after removal in which the semantic element is not recognized anymore while having IoU${}_{\text{post}}<0.5$. Values acc${}_{\Delta\text{depth}}$ capture changes in the depth maps compared to object mask, sim${}_{\text{SAM}}$ quantifies similarity in the SAM Kirillov et al. (2023) masks between ground truth and renders after removal, and PSNR measures of the whole image quality after removal, comparing the whole ground-truth novel view after removal to check for visual consistency. The best value is highlighted for each object and metric. GaussianCut (GC) Jain et al. (2024) outperforms AuraFusion (AF) Wu et al. (2025), especially in the instance segmentation similarity sim${}_{\text{SAM}}$. Remaining methods, Gaussian Groupping Ye et al. (2024a), Feature3DGS Zhou et al. (2024) and SAGS Hu et al. (2024), were unable to run object removal, therefore they are not included in this table.**
| Scene | Object | Method | IoU${}_{\text{drop}}\uparrow$ | acc${}_{\text{seg, IoU${}_{post}<0.5$}}\uparrow$ | acc${}_{\Delta\text{depth}}\uparrow$ | sim${}_{\text{SAM}}\uparrow$ | PSNR |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Backyard | Deckchair | Gaussian Cut | 0.85 | 0.99 | 0.67 | 0.56 | 15.62 |
|  |  | Aura Fusion | 0.84 | 0.99 | 0.65 | 0.54 | 15.86 |
|  | Chairs | Gaussian Cut | 0.85 | 1.00 | 0.76 | 0.83 | 17.99 |
|  |  | Aura Fusion | 0.87 | 1.00 | 0.67 | 0.62 | 17.89 |
|  | Stroller | Gaussian Cut | 0.92 | 1.00 | 0.89 | 0.85 | 19.19 |
|  |  | Aura Fusion | 0.91 | 1.00 | 0.73 | 0.72 | 18.74 |
|  | Playhouse | Gaussian Cut | 0.95 | 1.00 | 0.92 | 0.50 | 18.05 |
|  |  | Aura Fusion | 0.97 | 1.00 | 0.87 | 0.49 | 18.07 |
|  | Toy Truck | Gaussian Cut | 0.95 | 0.99 | 0.73 | 0.22 | 15.62 |
|  |  | Aura Fusion | 0.93 | 0.98 | 0.64 | 0.20 | 15.66 |
| Bedroom | Table | Gaussian Cut | 0.91 | 0.98 | 0.57 | 0.48 | 21.76 |
|  |  | Aura Fusion | 0.91 | 1.00 | 0.58 | 0.44 | 21.92 |
| Living Room | Pillows | Gaussian Cut | 0.62 | 0.77 | 0.53 | 0.19 | 21.45 |
|  |  | Aura Fusion | 0.76 | 0.88 | 0.51 | 0.18 | 20.41 |
|  | Sofa | Gaussian Cut | 0.57 | 0.50 | 0.62 | 0.17 | 17.45 |
|  |  | Aura Fusion | 0.62 | 0.64 | 0.62 | 0.13 | 16.81 |
| Office | Chairs | Gaussian Cut | 0.69 | 0.85 | 0.91 | 0.34 | 17.27 |
|  |  | Aura Fusion | 0.64 | 0.76 | 0.82 | 0.33 | 15.93 |
| Park | Bicycle | Gaussian Cut | 0.95 | 0.99 | 0.91 | 0.68 | 17.00 |
|  |  | Aura Fusion | 0.95 | 1.00 | 0.80 | 0.48 | 16.61 |
| Stairwell | Backpack | Gaussian Cut | 0.89 | 0.93 | 0.73 | 0.37 | 19.71 |
|  |  | Aura Fusion | 0.82 | 0.85 | 0.65 | 0.37 | 19.49 |

**(a) Breakdown of the proposed semantic segmentation IoU${}_{\text{drop}}$ metric. IoU${}_{\text{drop}}$ = IoU${}_{\text{post}}$ - IoU${}_{\text{pre}}$ and the higher, the better the removal. The best-performing method per object is highlighted in bold. The mean individual segmentation IoUs before and after removal, mIoU${}_{\text{pre}}$ and mIoU${}_{\text{post}}$ respectively, are also reported.**
| Scene | Object | Method | mIoU${}_{\text{pre}}$ | mIoU${}_{\text{post}}$ | IoU${}_{\text{drop}}\uparrow$ |
| --- | --- | --- | --- | --- | --- |
| Backyard | Deckchair | Gaussian Cut | 0.90 | 0.05 | 0.85 |
|  |  | Aura Fusion | 0.88 | 0.04 | 0.84 |
|  | Chairs | Gaussian Cut | 0.89 | 0.04 | 0.85 |
|  |  | Aura Fusion | 0.89 | 0.02 | 0.87 |
|  | Stroller | Gaussian Cut | 0.92 | 0.00 | 0.92 |
|  |  | Aura Fusion | 0.91 | 0.00 | 0.91 |
|  | Playhouse | Gaussian Cut | 0.97 | 0.03 | 0.95 |
|  |  | Aura Fusion | 0.99 | 0.02 | 0.97 |
|  | Toy Truck | Gaussian Cut | 0.95 | 0.00 | 0.95 |
|  |  | Aura Fusion | 0.95 | 0.02 | 0.93 |
| Bedroom | Table | Gaussian Cut | 0.93 | 0.02 | 0.91 |
|  |  | Aura Fusion | 0.92 | 0.01 | 0.91 |
| Living Room | Pillows | Gaussian Cut | 0.90 | 0.28 | 0.62 |
|  |  | Aura Fusion | 0.89 | 0.13 | 0.76 |
|  | Sofa | Gaussian Cut | 0.96 | 0.39 | 0.57 |
|  |  | Aura Fusion | 0.95 | 0.33 | 0.62 |
| Office | Chairs | Gaussian Cut | 0.85 | 0.18 | 0.67 |
|  |  | Aura Fusion | 0.83 | 0.19 | 0.64 |
| Park | Bicycle | Gaussian Cut | 0.97 | 0.02 | 0.95 |
|  |  | Aura Fusion | 0.95 | 0.00 | 0.95 |
| Stairwell | Backpack | Gaussian Cut | 0.94 | 0.05 | 0.89 |
|  |  | Aura Fusion | 0.96 | 0.14 | 0.82 |

Figure: Figure 6: Semantic segmentation changes before and after removal, Remove360 dataset. Left-right: GroundedSAM2 Kirillov et al. (2023); Liu et al. (2023); Ren et al. (2024) overlay on the rendering before removal, rendering after removal, overlay after removal. These semantic masks are used to calculate change in semantic segmentation in IoU${}_{\text{drop}}$ and its accuracy $\text{acc}_{\text{seg},\xi_{\text{IoU}}}$. Rows: Different object removals in dataset Remove360. Even though the object can not be recognized by a human, the segmentation model still finds it. One explanation can be that the pixel distribution on the edited area still exhibits patterns characteristic of the object, similar to what occurs in adversarial attacks.
Refer to caption: x6.png

Figure: (a) Deckchair removal.
Refer to caption: x7.png

Figure: (a) Stroller removal.
Refer to caption: x10.png

Figure: (a) Sofa removal.
Refer to caption: x13.png

Figure: (a) Pillows removal.
Refer to caption: x16.png

### B.3 Additional analysis

We analyse the correlation between input view visibility and residual signal strength after object removal using the sim${}_{\text{SAM}}$ metric.
Visibility (IoU${}_{\text{before}}$) is defined as the IoU between pre-removal semantic masks, obtained with GroundedSAM Ren et al. (2024) from rendered views, and ground truth masks from the original images. The sim${}_{\text{SAM}}$ metric measures similarity between SAM segments of the removal ground truth image and the after-removal renderings, with higher values indicating better removal.
For both Gaussian Cut (GC) and Aura Fusion (AF), are images grouped by IoU${}_{\text{before}}$ ranges and computed the Pearson correlation coefficient $r$ between IoU${}_{\text{before}}$ and sim${}_{\text{SAM}}$.
All results are displayed in Tab.[7](https://arxiv.org/html/2508.11431v1#A2.T7).
The hypothesis is that higher visibility should yield better removal, however the results yield that strong correlations are rare and mostly occurred in bins with very few samples (N < 20), limiting statistical reliability.
A notable exception is Backyard Toy House, where for IoU${}_{\text{before}}$ 0.95–1.00 both methods, GC ($r$ = –0.569, N = 116) and AF ($r$ = –0.504, N = 120), showed clear negative correlations, indicating that high visibility did not guarantee effective removal.
In the highest-visibility ranges (0.90–1.00), covering most of the dataset, correlations were generally weak (–0.2 to +0.2) and inconsistent across scenes and methods.
These results suggest that per-image visibility alone is not a reliable predictor of removal quality, and residual behaviour is likely influenced by more complex multi-view factors.

**Table 7: Correlation analysis between object visibility ranges before removal and sim${}_{\text{SAM}}$ score after removal on the Remove360 dataset. Pearson correlation statistic noted as $r$. Positive $r$: higher visibility $\rightarrow$ higher sim${}_{\text{SAM}}$. Negative $r$: higher visibility $\rightarrow$ lower sim${}_{\text{SAM}}$. Close to 0: little or no correlation. Notation "–" means insufficient data. Weak and inconsistent correlations indicate that visibility per image alone is not a reliable predictor of removal quality. Strong correlations mostly occur in IoU${}_{\text{before}}$ bins with very few samples (N < 20, 10$\%$ of total views), limiting statistical reliability. The image distribution is skewed toward the high IoU ranges. This is not surprising, because the benchmark is designed to have high object visibility in most views, leaving only few low-IoU views.**
| Scene | Object | Method | 0.01–0.50 | 0.50–0.75 | 0.75–0.90 | 0.90–0.95 | 0.95–1.00 |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  |  |  | #img | $r$ | #img | $r$ | #img | $r$ | #img | $r$ | #img | $r$ |
| Backyard | Deckchair | GC | 1 | – | 21 | -0.215 | 68 | -0.007 | 50 | -0.012 | 83 | -0.099 |
|  |  | AF | 1 | – | 21 | -0.225 | 68 | -0.001 | 50 | -0.010 | 83 | -0.088 |
|  | White chairs | GC | 11 | -0.465 | 1 | – | 2 | 1.000 | 9 | -0.403 | 115 | 0.120 |
|  |  | AF | 17 | -0.196 | 1 | – | 3 | 0.883 | 8 | 0.163 | 152 | 0.054 |
|  | Stroller | GC | 2 | – | 0 | – | 1 | – | 5 | 0.579 | 193 | 0.061 |
|  |  | AF | 3 | – | 0 | – | 2 | 1.000 | 10 | -0.112 | 186 | 0.091 |
|  | Toy house | GC | 4 | – | 0 | – | 1 | – | 0 | – | 116 | -0.569 |
|  |  | AF | 0 | – | 0 | – | 1 | – | 0 | – | 120 | -0.504 |
|  | Toy truck | GC | 6 | 0.215 | 1 | – | 2 | – | 1 | – | 172 | -0.136 |
|  |  | AF | 6 | 0.305 | 0 | – | 2 | 1.000 | 4 | -0.979 | 170 | 0.012 |
| Bedroom | Table | GC | 6 | 0.008 | 0 | – | 4 | -0.645 | 18 | -0.117 | 119 | 0.020 |
|  |  | AF | 7 | 0.488 | 0 | – | 6 | -0.054 | 34 | -0.082 | 100 | 0.076 |
| Living room | Pillows | GC | 14 | -0.145 | 6 | -0.239 | 5 | 0.347 | 4 | -0.970 | 132 | -0.018 |
|  |  | AF | 15 | -0.111 | 8 | 0.440 | 3 | -0.995 | 7 | 0.161 | 128 | -0.182 |
| Office | Chairs | GC | 9 | -0.134 | 33 | 0.071 | 123 | 0.211 | 121 | 0.131 | 55 | -0.170 |
|  |  | AF | 12 | -0.143 | 32 | 0.015 | 197 | -0.002 | 92 | -0.168 | 8 | 0.214 |
| Park | Bicycle | GC | 0 | – | 0 | – | 17 | 0.218 | 16 | -0.523 | 149 | -0.031 |
|  |  | AF | 0 | – | 5 | -0.098 | 26 | 0.184 | 15 | -0.365 | 136 | 0.066 |
| Stairwell | Backpack | GC | 2 | – | 0 | – | 0 | – | 5 | 0.298 | 102 | -0.177 |
|  |  | AF | 0 | – | 0 | – | 4 | 0.441 | 6 | -0.156 | 106 | -0.120 |

## Appendix C Removal Results for Mip-NERF360 Dataset Barron et al. ( 2022 )

### C.1 Quantitative Results After Removal

Tab. [8](https://arxiv.org/html/2508.11431v1#A3.T8), [9](https://arxiv.org/html/2508.11431v1#A3.T9) present quantitative results after removal on the Mip-NERF360 dataset.
The removal methods Gaussian Cut Jain et al. (2024) and Aura Fusion Wu et al. (2025) achieve the best performance in reducing semantic segmentation and object detection presence.
However, some objects remain difficult to fully remove, such as ’Slippers’ (see Fig. [11(a)](https://arxiv.org/html/2508.11431v1#A3.F11.sf1)) and ’Blue Gloves’ (see Fig. [11(b)](https://arxiv.org/html/2508.11431v1#A3.F11.sf2)), which are still detected in up to 96% of the views, with at least 11% persistence across methods.
Note that both methods were designed and evaluated on Mip-NeRF360 Barron et al. (2022).
We don’t know whether the segmentation model used in our evaluation, GroundedSAM2 Kirillov et al. (2023); Liu et al. (2023); Ren et al. (2024), have been trained using this dataset.
We were unable to obtain confirmation either confirming or denying this possibility.
For this reason, results on the novel Remove360 dataset are considered more reliable.
Since ground-truth novel views after removal are not available, PSNR cannot be computed for this dataset.

Figure: (a) Slippers removal.
Refer to caption: x18.png

### C.2 Qualitative Results After Removal

Qualitative results for each used scene in the Mip-NERF360 dataset are presented in Fig. [13](https://arxiv.org/html/2508.11431v1#A3.F13), [14](https://arxiv.org/html/2508.11431v1#A3.F14), [15](https://arxiv.org/html/2508.11431v1#A3.F15).
Each visualization includes SAM Kirillov et al. (2023) segmentations, and depth differences computed before and after removal using a thresholding approach described in the spatial recognition subsection of the main paper.
The semantic similarity between the SAM segments of renders before and after removal is reported; as ground-truth segmentation after removal is not available.
Lower similarity scores indicate greater distinction between the before and after states, reflecting more successful object removal.
Additionally, the accuracy of the depth difference within the pseudo-ground-truth mask is shown; higher values suggest effective removal, as changes in depth at the object’s location are expected.

Gaussian Cut Jain et al. (2024) generally produces more visually coherent results, leaving fewer artifacts and preserving scene quality more effectively than competing methods.
These observations are not supported by all the quantitative results of the semantic segmentation (Tab. [5](https://arxiv.org/html/2508.11431v1#A2.T5), [6](https://arxiv.org/html/2508.11431v1#A2.T6)), which means invisible Gaussians with the semantic information, must be still present in the image, see Fig. [12](https://arxiv.org/html/2508.11431v1#A3.F12).

Figure: Figure 12: Semantic segmentation changes before and after object removal of Gaussian Cut Jain et al. (2024) method, Mip-NERF360 Barron et al. (2022) dataset. Left-right: GroundedSAM2 Kirillov et al. (2023); Liu et al. (2023); Ren et al. (2024) overlay on the rendering before removal, rendering after removal, overlay after removal. These semantic masks are used to calculate change in semantic segmentation in IoU${}_{\text{drop}}$ and its accuracy $\text{acc}_{\text{seg},\xi_{\text{IoU}}}$. Rows: Different objects removed by Gaussian Cut Jain et al. (2024) in different scenes from Mip-NERF360 Barron et al. (2022). Even though the object can not be recognized by a human, the segmentation model still finds it.
Refer to caption: x20.png

**Table 8: Mip-NERF360: Breakdown of the proposed semantic segmentation IoU${}_{\text{drop}}$ metric. IoU${}_{\text{drop}}$ = IoU${}_{\text{post}}$ - IoU${}_{\text{pre}}$ and the higher, the better the removal. The best-performing method is highlighted in bold, second-best underlined. The mean individual segmentation IoUs before and after removal, mIoU${}_{\text{pre}}$ and mIoU${}_{\text{post}}$ respectively, are also reported.**
| Scene | Object | Method | mIoU${}_{\text{pre}}$ | mIoU${}_{\text{post}}$ | IoU${}_{\text{drop}}\uparrow$ |
| --- | --- | --- | --- | --- | --- |
| Counter | Baking Tray | Gaussian Grouping | 0.61 | 0.08 | 0.53 |
|  |  | Feature 3DGS | 0.54 | 0.21 | 0.34 |
|  |  | SAGS | 0.62 | 0.52 | 0.10 |
|  |  | Gaussian Cut | 0.63 | 0.01 | 0.62 |
|  |  | Aura Fusion | 0.64 | 0.04 | 0.60 |
|  | Plant | Gaussian Grouping | 0.84 | 0.00 | 0.84 |
|  |  | Feature 3DGS | 0.75 | 0.00 | 0.75 |
|  |  | SAGS | 0.85 | 0.82 | 0.03 |
|  |  | Gaussian Cut | 0.86 | 0.00 | 0.86 |
|  |  | Aura Fusion | 0.87 | 0.00 | 0.87 |
|  | Blue Gloves | Gaussian Grouping | 0.75 | 0.15 | 0.60 |
|  |  | Feature 3DGS | 0.67 | 0.66 | 0.01 |
|  |  | SAGS | 0.74 | 0.64 | 0.10 |
|  |  | Gaussian Cut | 0.74 | 0.15 | 0.60 |
|  |  | Aura Fusion | 0.76 | 0.11 | 0.65 |
|  | Egg Box | Gaussian Grouping | 0.63 | 0.00 | 0.63 |
|  |  | Feature 3DGS | 0.78 | 0.70 | 0.08 |
|  |  | SAGS | 0.60 | 0.04 | 0.56 |
|  |  | Gaussian Cut | 0.63 | 0.01 | 0.62 |
|  |  | Aura Fusion | 0.64 | 0.01 | 0.63 |
| Room | Plant | Gaussian Grouping | 0.50 | 0.23 | 0.26 |
|  |  | Feature 3DGS | 0.53 | 0.00 | 0.53 |
|  |  | SAGS | 0.52 | 0.35 | 0.17 |
|  |  | Gaussian Cut | 0.53 | 0.00 | 0.53 |
|  |  | Aura Fusion | 0.49 | 0.26 | 0.23 |
|  | Slippers | Gaussian Grouping | 0.96 | 0.14 | 0.82 |
|  |  | Feature 3DGS | 0.96 | 0.96 | 0.00 |
|  |  | SAGS | 0.96 | 0.71 | 0.25 |
|  |  | Gaussian Cut | 0.97 | 0.48 | 0.48 |
|  |  | Aura Fusion | 0.43 | 0.37 | 0.06 |
|  | Coffee Table | Gaussian Grouping | 0.88 | 0.02 | 0.86 |
|  |  | Feature 3DGS | 0.86 | 0.29 | 0.57 |
|  |  | SAGS | 0.89 | 0.89 | 0.00 |
|  |  | Gaussian Cut | 0.89 | 0.03 | 0.86 |
|  |  | Aura Fusion | 0.58 | 0.03 | 0.55 |
| Kitchen | Truck | Gaussian Grouping | 0.67 | 0.06 | 0.61 |
|  |  | Feature 3DGS | 0.67 | 0.05 | 0.62 |
|  |  | SAGS | 0.67 | 0.00 | 0.67 |
|  |  | Gaussian Cut | 0.67 | 0.01 | 0.66 |
|  |  | Aura Fusion | 0.96 | 0.01 | 0.95 |
| Garden | Table | Gaussian Grouping | 0.89 | 0.41 | 0.48 |
|  |  | Feature 3DGS | 0.90 | 0.24 | 0.67 |
|  |  | SAGS | 0.90 | 0.09 | 0.81 |
|  |  | Gaussian Cut | 0.90 | 0.04 | 0.86 |
|  |  | Aura Fusion | 0.91 | 0.01 | 0.90 |
|  | Ball | Gaussian Grouping | 0.16 | 0.00 | 0.16 |
|  |  | Feature 3DGS | 0.06 | 0.06 | 0.00 |
|  |  | SAGS | 0.41 | 0.00 | 0.41 |
|  |  | Gaussian Cut | 0.42 | 0.00 | 0.42 |
|  |  | Aura Fusion | 0.42 | 0.00 | 0.42 |
|  | Vase | Gaussian Grouping | 0.85 | 0.22 | 0.64 |
|  |  | Feature 3DGS | 0.90 | 0.11 | 0.79 |
|  |  | SAGS | 0.97 | 0.01 | 0.96 |
|  |  | Gaussian Cut | 0.97 | 0.01 | 0.97 |
|  |  | Aura Fusion | 0.98 | 0.01 | 0.97 |

**Table 9: Mip-NERF360: Breakdown of the proposed metric of semantic recognition $\text{acc}_{\text{seg},\xi_{\text{IoU}}}$ based on the IoU${}_{\text{post}}$ threshold. This table presents the ratio of images in which the semantic element is not recognized. We define that the object is not segmented if the semantic segmentation IoU is lower than a threshold. The higher, the better the removal. Reported thresholds {$0.3,0.5,0.7,0.9$}.**
| Scene | Object | Method | acc${}_{\text{IoU${}_{\text{post}}<0.3$}}\uparrow$ | acc${}_{\text{IoU${}_{\text{post}}<0.5$}}\uparrow$ | acc${}_{\text{IoU${}_{\text{post}}<0.7$}}\uparrow$ | acc${}_{\text{IoU${}_{\text{post}}<0.9$}}\uparrow$ |
| --- | --- | --- | --- | --- | --- | --- |
| Counter | Baking Tray | Gaussian Grouping | 0.915 | 0.915 | 0.915 | 0.943 |
|  |  | Feature 3DGS | 0.783 | 0.783 | 0.802 | 0.821 |
|  |  | SAGS | 0.358 | 0.481 | 0.557 | 0.698 |
|  |  | Gaussian Cut | 0.991 | 0.991 | 0.991 | 0.991 |
|  |  | Aura Fusion | 0.953 | 0.953 | 0.953 | 0.962 |
|  | Plant | Gaussian Grouping | 1.000 | 1.000 | 1.000 | 1.000 |
|  |  | Feature 3DGS | 1.000 | 1.000 | 1.000 | 1.000 |
|  |  | SAGS | 0.167 | 0.167 | 0.167 | 0.167 |
|  |  | Gaussian Cut | 1.000 | 1.000 | 1.000 | 1.000 |
|  |  | Aura Fusion | 1.000 | 1.000 | 1.000 | 1.000 |
|  | Blue Gloves | Gaussian Grouping | 0.837 | 0.837 | 0.837 | 0.904 |
|  |  | Feature 3DGS | 0.240 | 0.279 | 0.337 | 0.471 |
|  |  | SAGS | 0.212 | 0.337 | 0.500 | 0.558 |
|  |  | Gaussian Cut | 0.827 | 0.827 | 0.827 | 0.962 |
|  |  | Aura Fusion | 0.875 | 0.885 | 0.885 | 0.962 |
|  | Egg Box | Gaussian Grouping | 1.000 | 1.000 | 1.000 | 1.000 |
|  |  | Feature 3DGS | 0.196 | 0.196 | 0.206 | 0.289 |
|  |  | SAGS | 0.959 | 0.959 | 0.959 | 0.959 |
|  |  | Gaussian Cut | 0.990 | 0.990 | 1.000 | 1.000 |
|  |  | Aura Fusion | 0.990 | 0.990 | 0.990 | 0.990 |
| Room | Plant | Gaussian Grouping | 0.640 | 0.800 | 0.880 | 0.920 |
|  |  | Feature 3DGS | 1.000 | 1.000 | 1.000 | 1.000 |
|  |  | SAGS | 0.440 | 0.720 | 0.800 | 0.920 |
|  |  | Gaussian Cut | 1.000 | 1.000 | 1.000 | 1.000 |
|  |  | Aura Fusion | 0.961 | 0.961 | 0.981 | 0.994 |
|  | Slippers | Gaussian Grouping | 0.853 | 0.853 | 0.853 | 0.868 |
|  |  | Feature 3DGS | 0.015 | 0.015 | 0.015 | 0.015 |
|  |  | SAGS | 0.029 | 0.279 | 0.324 | 0.779 |
|  |  | Gaussian Cut | 0.338 | 0.441 | 0.588 | 0.838 |
|  |  | Aura Fusion | 0.568 | 0.568 | 0.574 | 0.839 |
|  | Coffee Table | Gaussian Grouping | 0.990 | 0.990 | 0.990 | 0.990 |
|  |  | Feature 3DGS | 0.586 | 0.616 | 0.788 | 0.949 |
|  |  | SAGS | 0.091 | 0.091 | 0.101 | 0.101 |
|  |  | Gaussian Cut | 0.970 | 0.990 | 0.990 | 0.990 |
|  |  | Aura Fusion | 0.981 | 0.961 | 0.981 | 0.994 |
| Kitchen | Truck | Gaussian Grouping | 0.897 | 0.922 | 0.990 | 1.000 |
|  |  | Feature 3DGS | 0.941 | 0.951 | 0.951 | 0.956 |
|  |  | SAGS | 1.000 | 1.000 | 1.000 | 1.000 |
|  |  | Gaussian Cut | 0.985 | 0.995 | 1.000 | 1.000 |
|  |  | Aura Fusion | 0.993 | 1.000 | 1.000 | 1.000 |
| Garden | Table | Gaussian Grouping | 0.426 | 0.541 | 0.709 | 0.899 |
|  |  | Feature 3DGS | 0.676 | 0.703 | 0.818 | 1.000 |
|  |  | SAGS | 0.872 | 0.878 | 0.926 | 1.000 |
|  |  | Gaussian Cut | 0.953 | 0.953 | 0.966 | 1.000 |
|  |  | Aura Fusion | 1.000 | 1.000 | 1.000 | 1.000 |
|  | Ball | Gaussian Grouping | 1.000 | 1.000 | 1.000 | 1.000 |
|  |  | Feature 3DGS | 0.939 | 0.939 | 0.939 | 0.939 |
|  |  | SAGS | 1.000 | 1.000 | 1.000 | 1.000 |
|  |  | Gaussian Cut | 1.000 | 1.000 | 1.000 | 1.000 |
|  |  | Aura Fusion | 1.000 | 1.000 | 1.000 | 1.000 |
|  | Vase | Gaussian Grouping | 0.784 | 0.784 | 0.797 | 0.804 |
|  |  | Feature 3DGS | 0.885 | 0.892 | 0.905 | 0.926 |
|  |  | SAGS | 1.000 | 1.000 | 1.000 | 1.000 |
|  |  | Gaussian Cut | 1.000 | 1.000 | 1.000 | 1.000 |
|  |  | Aura Fusion | 1.000 | 1.000 | 1.000 | 1.000 |

Figure: (a) Plant removal.
Refer to caption: x21.png

Figure: (a) Baking Tray removal.
Refer to caption: x23.png

Figure: (a) Table removal.
Refer to caption: x25.png

## References

- (1)
Accelerating AR and AI from a human perspective.
https://www.projectaria.com/.
- (2)
Luma AI Interactive Scenes.
https://lumalabs.ai/interactive-scenes.
- (3)
Polycam.
https://poly.cam/spatial-capture.
- (4)
Postshot.
https://www.jawset.com/.
- (5)
Realityscan - 3d scanning app.
- (6)
Scaniverse. niantic, inc.
https://scaniverse.com/.
- (7)
Spectacles, AR glasses powered by Snap OS.
https://www.spectacles.com/.
- (8)
Xreal One, AR for All.
https://www.xreal.com/.
- Achiam et al. (2023)
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al.
Gpt-4 technical report.
*arXiv preprint arXiv:2303.08774*, 2023.
- Adam et al. (2022)
Aikaterini Adam, Torsten Sattler, Konstantinos Karantzalos, and Tomas Pajdla.
Objects can move: 3d change detection by geometric transformation consistency.
In *European Conference on Computer Vision*, pages 108–124. Springer, 2022.
- Arandjelović et al. (2016)
R. Arandjelović, P. Gronat, A. Torii, T. Pajdla, and J. Sivic.
NetVLAD: CNN architecture for weakly supervised place recognition.
In *IEEE Conference on Computer Vision and Pattern Recognition*, 2016.
- Badrinarayanan et al. (2017)
Vijay Badrinarayanan, Alex Kendall, and Roberto Cipolla.
Segnet: A deep convolutional encoder-decoder architecture for image segmentation.
*IEEE transactions on pattern analysis and machine intelligence*, 39(12):2481–2495, 2017.
- Barron (2020)
Jonathan T. Barron.
A generalization of otsu’s method and minimum error thresholding.
In *European conference on computer vision*, 2020.
- Barron et al. (2022)
Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman.
Mip-nerf 360: Unbounded anti-aliased neural radiance fields.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pages 5470–5479, 2022.
- Bhalgat et al. (2024)
Yash Bhalgat, Iro Laina, João F Henriques, Andrea Vedaldi, and Andrew Zisserman.
Contrastive lift: 3d object instance segmentation by slow-fast contrastive fusion.
*Advances in Neural Information Processing Systems*, 36, 2024.
- Bradski (2000)
G. Bradski.
The OpenCV Library.
*Dr. Dobb’s Journal of Software Tools*, 2000.
- Brooks et al. (2024)
Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, et al.
Video generation models as world simulators.
*OpenAI Blog*, 1:8, 2024.
- Caron et al. (2021)
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin.
Emerging properties in self-supervised vision transformers.
In *Proceedings of the IEEE/CVF international conference on computer vision*, pages 9650–9660, 2021.
- Cen et al. (2023a)
Jiazhong Cen, Jiemin Fang, Chen Yang, Lingxi Xie, Xiaopeng Zhang, Wei Shen, and Qi Tian.
Segment any 3d gaussians.
*arXiv preprint arXiv:2312.00860*, 2023a.
- Cen et al. (2023b)
Jiazhong Cen, Zanwei Zhou, Jiemin Fang, Wei Shen, Lingxi Xie, Dongsheng Jiang, Xiaopeng Zhang, Qi Tian, et al.
Segment anything in 3d with nerfs.
*Advances in Neural Information Processing Systems*, 36:25971–25990, 2023b.
- Chelani et al. (2023)
Kunal Chelani, Torsten Sattler, Fredrik Kahl, and Zuzana Kukelova.
Privacy-preserving representations are not enough: Recovering scene content from camera poses.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 13132–13141, 2023.
- Chen et al. (2024a)
Anpei Chen, Haofei Xu, Stefano Esposito, Siyu Tang, and Andreas Geiger.
Lara: Efficient large-baseline radiance fields.
In *European Conference on Computer Vision*, pages 338–355. Springer, 2024a.
- Chen et al. (2017)
Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille.
Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs.
*IEEE transactions on pattern analysis and machine intelligence*, 40(4):834–848, 2017.
- Chen et al. (2024b)
Minghao Chen, Iro Laina, and Andrea Vedaldi.
Dge: Direct gaussian 3d editing by consistent multi-view editing.
In *European Conference on Computer Vision*, pages 74–92. Springer, 2024b.
- Chen et al. (2024c)
Yuedong Chen, Haofei Xu, Chuanxia Zheng, Bohan Zhuang, Marc Pollefeys, Andreas Geiger, Tat-Jen Cham, and Jianfei Cai.
Mvsplat: Efficient 3d gaussian splatting from sparse multi-view images.
In *European Conference on Computer Vision*, pages 370–386. Springer, 2024c.
- Choi et al. (2024a)
Seokhun Choi, Hyeonseop Song, Jaechul Kim, Taehyeong Kim, and Hoseok Do.
Click-gaussian: Interactive segmentation to any 3d gaussians.
In *European Conference on Computer Vision*, pages 289–305. Springer, 2024a.
- Choi et al. (2024b)
Seokhun Choi, Hyeonseop Song, Jaechul Kim, Taehyeong Kim, and Hoseok Do.
Click-gaussian: Interactive segmentation to any 3d gaussians.
In *European Conference on Computer Vision*, pages 289–305. Springer, 2024b.
- DeTone et al. (2018)
Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich.
Superpoint: Self-supervised interest point detection and description.
In *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops*, pages 224–236, 2018.
- Furukawa and Ponce (2009)
Yasutaka Furukawa and Jean Ponce.
Accurate, dense, and robust multiview stereopsis.
*IEEE transactions on pattern analysis and machine intelligence*, 32(8):1362–1376, 2009.
- Graham and Van der Maaten (2017)
Benjamin Graham and Laurens Van der Maaten.
Submanifold sparse convolutional networks.
*arXiv preprint arXiv:1706.01307*, 2017.
- Grynbaum and Mac (2023)
Michael M Grynbaum and Ryan Mac.
The times sues openai and microsoft over ai use of copyrighted work.
*The New York Times*, 27, 2023.
- Gu et al. (2024)
Qiao Gu, Zhaoyang Lv, Duncan Frost, Simon Green, Julian Straub, and Chris Sweeney.
Egolifter: Open-world 3d segmentation for egocentric perception.
In *European Conference on Computer Vision*, pages 382–400. Springer, 2024.
- Hu et al. (2024)
Xu Hu, Yuxi Wang, Lue Fan, Junsong Fan, Junran Peng, Zhen Lei, Qing Li, and Zhaoxiang Zhang.
Semantic anything in 3d gaussians.
*arXiv e-prints*, pages arXiv–2401, 2024.
- Huang et al. (2024)
Rui Huang, Songyou Peng, Ayca Takmaz, Federico Tombari, Marc Pollefeys, Shiji Song, Gao Huang, and Francis Engelmann.
Segment3d: Learning fine-grained class-agnostic 3d segmentation without manual labels.
In *European Conference on Computer Vision*, pages 278–295. Springer, 2024.
- Illman and Temple (2019)
Erin Illman and Paul Temple.
California consumer privacy act.
*The Business Lawyer*, 75(1):1637–1646, 2019.
- Jain et al. (2024)
Umangi Jain, Ashkan Mirzaei, and Igor Gilitschenski.
Gaussiancut: Interactive segmentation via graph cut for 3d gaussian splatting.
In *The Thirty-eighth Annual Conference on Neural Information Processing Systems*, 2024.
- Kerbl et al. (2023)
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis.
3d gaussian splatting for real-time radiance field rendering.
*ACM Transactions on Graphics*, 42(4), 2023.
- Kerr et al. (2023)
Justin Kerr, Chung Min Kim, Ken Goldberg, Angjoo Kanazawa, and Matthew Tancik.
Lerf: Language embedded radiance fields.
In *Proceedings of the IEEE/CVF International Conference on Computer Vision*, pages 19729–19739, 2023.
- Kirillov et al. (2023)
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollár, and Ross Girshick.
Segment anything.
*arXiv:2304.02643*, 2023.
- Kobayashi et al. (2022)
Sosuke Kobayashi, Eiichi Matsumoto, and Vincent Sitzmann.
Decomposing nerf for editing via feature field distillation.
*Advances in neural information processing systems*, 35:23311–23330, 2022.
- Koch et al. (2024)
Sebastian Koch, Johanna Wald, Mirco Colosi, Narunas Vaskevicius, Pedro Hermosilla, Federico Tombari, and Timo Ropinski.
Relationfield: Relate anything in radiance fields.
*arXiv preprint arXiv:2412.13652*, 2024.
- Kulhanek and Sattler (2023)
Jonas Kulhanek and Torsten Sattler.
Tetra-nerf: Representing neural radiance fields using tetrahedra.
In *Proceedings of the IEEE/CVF International Conference on Computer Vision*, pages 18458–18469, 2023.
- Kulhanek et al. (2024)
Jonas Kulhanek, Songyou Peng, Zuzana Kukelova, Marc Pollefeys, and Torsten Sattler.
Wildgaussians: 3d gaussian splatting in the wild.
In *The Thirty-Eighth Annual Conference on Neural Information Processing Systems (NeurIPS 2024)*, 2024.
- Kundu et al. (2020)
Abhijit Kundu, Xiaoqi Yin, Alireza Fathi, David Ross, Brian Brewington, Thomas Funkhouser, and Caroline Pantofaru.
Virtual multi-view fusion for 3d semantic segmentation.
In *Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXIV 16*, pages 518–535. Springer, 2020.
- Landrieu and Obozinski (2017)
Loic Landrieu and Guillaume Obozinski.
Cut pursuit: Fast algorithms to learn piecewise constant functions on general weighted graphs.
*SIAM Journal on Imaging Sciences*, 10(4):1724–1766, 2017.
- Lazebnik et al. (2001)
Svetlana Lazebnik, Edmond Boyer, and Jean Ponce.
On computing exact visual hulls of solids bounded by smooth surfaces.
In *Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001*, pages I–I. IEEE, 2001.
- Li et al. (2022)
Boyi Li, Kilian Q Weinberger, Serge Belongie, Vladlen Koltun, and Rene Ranftl.
Language-driven semantic segmentation.
In *International Conference on Learning Representations*, 2022.
- Li et al. (2024)
Feng Li, Hao Zhang, Peize Sun, Xueyan Zou, Shilong Liu, Chunyuan Li, Jianwei Yang, Lei Zhang, and Jianfeng Gao.
Segment and recognize anything at any granularity.
In *European Conference on Computer Vision*, pages 467–484. Springer, 2024.
- Liang et al. (2024)
Siyun Liang, Sen Wang, Kunyi Li, Michael Niemeyer, Stefano Gasperini, Nassir Navab, and Federico Tombari.
Supergseg: Open-vocabulary 3d segmentation with structured super-gaussians.
*arXiv preprint arXiv:2412.10231*, 2024.
- Liao et al. (2024)
Guibiao Liao, Jiankun Li, Zhenyu Bao, Xiaoqing Ye, Jingdong Wang, Qing Li, and Kanglin Liu.
Clip-gs: Clip-informed gaussian splatting for real-time and view-consistent 3d semantic understanding.
*arXiv preprint arXiv:2404.14249*, 2024.
- Lin et al. (2024)
Jiaqi Lin, Zhihao Li, Xiao Tang, Jianzhuang Liu, Shiyong Liu, Jiayue Liu, Yangdi Lu, Xiaofei Wu, Songcen Xu, Youliang Yan, et al.
Vastgaussian: Vast 3d gaussians for large scene reconstruction.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 5166–5175, 2024.
- Lindenberger et al. (2023)
Philipp Lindenberger, Paul-Edouard Sarlin, and Marc Pollefeys.
LightGlue: Local Feature Matching at Light Speed.
In *ICCV*, 2023.
- Liu et al. (2024a)
Dingning Liu, Xiaoshui Huang, Yuenan Hou, Zhihui Wang, Zhenfei Yin, Yongshun Gong, Peng Gao, and Wanli Ouyang.
Uni3d-llm: Unifying point cloud perception, generation and editing with large language models.
*arXiv preprint arXiv:2402.03327*, 2024a.
- Liu et al. (2022)
Quande Liu, Youpeng Wen, Jianhua Han, Chunjing Xu, Hang Xu, and Xiaodan Liang.
Open-world semantic segmentation via contrasting and clustering vision-language embedding.
In *European Conference on Computer Vision*, pages 275–292. Springer, 2022.
- Liu et al. (2023)
Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, et al.
Grounding dino: Marrying dino with grounded pre-training for open-set object detection.
*arXiv preprint arXiv:2303.05499*, 2023.
- Liu et al. (2024b)
Yichen Liu, Benran Hu, Chi-Keung Tang, and Yu-Wing Tai.
Sanerf-hq: Segment anything for nerf in high quality.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 3216–3226, 2024b.
- Liu et al. (2024c)
Zhiheng Liu, Hao Ouyang, Qiuyu Wang, Ka Leong Cheng, Jie Xiao, Kai Zhu, Nan Xue, Yu Liu, Yujun Shen, and Yang Cao.
Infusion: Inpainting 3d gaussians via learning depth completion from diffusion prior.
*arXiv preprint arXiv:2404.11613*, 2024c.
- Martin-Brualla et al. (2021)
Ricardo Martin-Brualla, Noha Radwan, Mehdi SM Sajjadi, Jonathan T Barron, Alexey Dosovitskiy, and Daniel Duckworth.
Nerf in the wild: Neural radiance fields for unconstrained photo collections.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pages 7210–7219, 2021.
- Mildenhall et al. (2020)
B Mildenhall, PP Srinivasan, M Tancik, JT Barron, R Ramamoorthi, and R Ng.
Nerf: Representing scenes as neural radiance fields for view synthesis.
In *European conference on computer vision*, 2020.
- Mirzaei et al. (2022)
Ashkan Mirzaei, Yash Kant, Jonathan Kelly, and Igor Gilitschenski.
Laterf: Label and text driven object radiance fields.
In *European Conference on Computer Vision*, pages 20–36. Springer, 2022.
- Mirzaei et al. (2023)
Ashkan Mirzaei, Tristan Aumentado-Armstrong, Konstantinos G Derpanis, Jonathan Kelly, Marcus A Brubaker, Igor Gilitschenski, and Alex Levinshtein.
Spin-nerf: Multiview segmentation and perceptual inpainting with neural radiance fields.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 20669–20679, 2023.
- Moon et al. (2024)
Heejoon Moon, Chunghwan Lee, and Je Hyeong Hong.
Efficient privacy-preserving visual localization using 3d ray clouds.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)*, pages 9773–9783, 2024.
- Müller et al. (2022)
Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller.
Instant neural graphics primitives with a multiresolution hash encoding.
*ACM transactions on graphics (TOG)*, 41(4):1–15, 2022.
- Munkres (1957)
James Munkres.
Algorithms for the assignment and transportation problems.
*Journal of the society for industrial and applied mathematics*, 5(1):32–38, 1957.
- Nasr et al. (2023)
Milad Nasr, Saeed Mahloujifar, Xinyu Tang, Prateek Mittal, and Amir Houmansadr.
Effectively using public data in privacy preserving machine learning.
In *International Conference on Machine Learning*, pages 25718–25732. PMLR, 2023.
- Oquab et al. (2024)
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al.
Dinov2: Learning robust visual features without supervision.
*Transactions on Machine Learning Research Journal*, pages 1–31, 2024.
- Peng et al. (2023)
Songyou Peng, Kyle Genova, Chiyu Jiang, Andrea Tagliasacchi, Marc Pollefeys, Thomas Funkhouser, et al.
Openscene: 3d scene understanding with open vocabularies.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pages 815–824, 2023.
- Pittaluga et al. (2019)
Francesco Pittaluga, Sanjeev J Koppal, Sing Bing Kang, and Sudipta N Sinha.
Revealing scenes by inverting structure from motion reconstructions.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 145–154, 2019.
- Qin et al. (2024)
Minghan Qin, Wanhua Li, Jiawei Zhou, Haoqian Wang, and Hanspeter Pfister.
Langsplat: 3d language gaussian splatting.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 20051–20060, 2024.
- Radford et al. (2021)
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.
Learning transferable visual models from natural language supervision.
In *International conference on machine learning*, pages 8748–8763. PmLR, 2021.
- Raina et al. (2023)
Nikhil Raina, Guruprasad Somasundaram, Kang Zheng, Sagar Miglani, Steve Saarinen, Jeff Meissner, Mark Schwesinger, Luis Pesqueira, Ishita Prasad, Edward Miller, Prince Gupta, Mingfei Yan, Richard Newcombe, Carl Ren, and Omkar M Parkhi.
Egoblur: Responsible innovation in aria, 2023.
- Ravi et al. (2024)
Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junting Pan, Kalyan Vasudev Alwala, Nicolas Carion, Chao-Yuan Wu, Ross Girshick, Piotr Dollár, and Christoph Feichtenhofer.
Sam 2: Segment anything in images and videos.
*arXiv preprint arXiv:2408.00714*, 2024.
- RealityCapture2023 (2023)
RealityCapture2023.
RealityCapture, 2023.
- Reiser et al. (2021)
Christian Reiser, Songyou Peng, Yiyi Liao, and Andreas Geiger.
Kilonerf: Speeding up neural radiance fields with thousands of tiny mlps.
In *Proceedings of the IEEE/CVF international conference on computer vision*, pages 14335–14345, 2021.
- Ren et al. (2024)
Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, Zhaoyang Zeng, Hao Zhang, Feng Li, Jie Yang, Hongyang Li, Qing Jiang, and Lei Zhang.
Grounded sam: Assembling open-world models for diverse visual tasks, 2024.
- Rombach et al. (2022)
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer.
High-resolution image synthesis with latent diffusion models.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pages 10684–10695, 2022.
- Rubinstein and Good (2013)
Ira S Rubinstein and Nathaniel Good.
Privacy by design: A counterfactual analysis of google and facebook privacy incidents.
*Berkeley Tech. LJ*, 28:1333, 2013.
- Saharia et al. (2022)
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al.
Photorealistic text-to-image diffusion models with deep language understanding.
*Advances in neural information processing systems*, 35:36479–36494, 2022.
- Sarlin et al. (2019)
Paul-Edouard Sarlin, Cesar Cadena, Roland Siegwart, and Marcin Dymczyk.
From coarse to fine: Robust hierarchical localization at large scale.
In *CVPR*, 2019.
- Sarlin et al. (2020)
Paul-Edouard Sarlin, Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich.
SuperGlue: Learning feature matching with graph neural networks.
In *CVPR*, 2020.
- Schönberger and Frahm (2016)
Johannes Lutz Schönberger and Jan-Michael Frahm.
Structure-from-motion revisited.
In *Conference on Computer Vision and Pattern Recognition (CVPR)*, 2016.
- Schonberger and Frahm (2016)
Johannes L Schonberger and Jan-Michael Frahm.
Structure-from-motion revisited.
In *Proceedings of the IEEE conference on computer vision and pattern recognition*, pages 4104–4113, 2016.
- Schönberger et al. (2016)
Johannes Lutz Schönberger, Enliang Zheng, Marc Pollefeys, and Jan-Michael Frahm.
Pixelwise view selection for unstructured multi-view stereo.
In *European Conference on Computer Vision (ECCV)*, 2016.
- Schuhmann et al. (2022)
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al.
Laion-5b: An open large-scale dataset for training next generation image-text models.
*Advances in neural information processing systems*, 35:25278–25294, 2022.
- Shi et al. (2024)
Jin-Chuan Shi, Miao Wang, Hao-Bin Duan, and Shao-Hua Guan.
Language embedded 3d gaussians for open-vocabulary scene understanding.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 5333–5343, 2024.
- Siddiqui et al. (2023)
Yawar Siddiqui, Lorenzo Porzi, Samuel Rota Buló, Norman Müller, Matthias Nießner, Angela Dai, and Peter Kontschieder.
Panoptic lifting for 3d scene understanding with neural fields.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 9043–9052, 2023.
- Speciale et al. (2019)
Pablo Speciale, Johannes L Schonberger, Sing Bing Kang, Sudipta N Sinha, and Marc Pollefeys.
Privacy preserving image-based localization.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 5493–5503, 2019.
- Takmaz et al. (2025)
Ayca Takmaz, Alexandros Delitzas, Robert W Sumner, Francis Engelmann, Johanna Wald, and Federico Tombari.
Search3d: Hierarchical open-vocabulary 3d segmentation.
*IEEE Robotics and Automation Letters*, 2025.
- Tancik et al. (2023)
Matthew Tancik, Ethan Weber, Evonne Ng, Ruilong Li, Brent Yi, Justin Kerr, Terrance Wang, Alexander Kristoffersen, Jake Austin, Kamyar Salahi, Abhik Ahuja, David McAllister, and Angjoo Kanazawa.
Nerfstudio: A modular framework for neural radiance field development.
In *ACM SIGGRAPH 2023 Conference Proceedings*, 2023.
- Tschernezki et al. (2022)
Vadim Tschernezki, Iro Laina, Diane Larlus, and Andrea Vedaldi.
Neural feature fusion fields: 3d distillation of self-supervised 2d image representations.
In *2022 International Conference on 3D Vision (3DV)*, pages 443–453. IEEE, 2022.
- Voigt and Von dem Bussche (2017)
Paul Voigt and Axel Von dem Bussche.
The eu general data protection regulation (gdpr).
*A practical guide, 1st ed., Cham: Springer International Publishing*, 10(3152676):10–5555, 2017.
- Vora* et al. (2022)
Suhani Vora*, Noha Radwan*, Klaus Greff, Henning Meyer, Kyle Genova, Mehdi S. M. Sajjadi, Etienne Pot, Andrea Tagliasacchi, and Daniel Duckworth.
Nesf: Neural semantic fields for generalizable semantic segmentation of 3d scenes.
*Transactions on Machine Learning Research*, 2022.
- Wang et al. (2022)
Can Wang, Menglei Chai, Mingming He, Dongdong Chen, and Jing Liao.
Clip-nerf: Text-and-image driven manipulation of neural radiance fields.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pages 3835–3844, 2022.
- Wang et al. (2024)
Yunsong Wang, Tianxin Huang, Hanlin Chen, and Gim Hee Lee.
Freesplat: Generalizable 3d gaussian splatting towards free view synthesis of indoor scenes.
*Advances in Neural Information Processing Systems*, 37:107326–107349, 2024.
- Weder et al. (2023)
Silvan Weder, Guillermo Garcia-Hernando, Aron Monszpart, Marc Pollefeys, Gabriel J Brostow, Michael Firman, and Sara Vicente.
Removing objects from neural radiance fields.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 16528–16538, 2023.
- Wu et al. (2025)
Chung-Ho Wu, Yang-Jung Chen, Ying-Huan Chen, Jie-Ying Lee, Bo-Hsu Ke, Chun-Wei Tuan Mu, Yi-Chuan Huang, Chin-Yang Lin, Min-Hung Chen, Yen-Yu Lin, and Yu-Lun Liu.
Aurafusion360: Augmented unseen region alignment for reference-based 360${\deg}$ unbounded scene inpainting.
In *CVPR*, 2025.
- Wu et al. (2024a)
Yanmin Wu, Jiarui Meng, LI Haijie, Chenming Wu, Yahao Shi, Xinhua Cheng, Chen Zhao, Haocheng Feng, Errui Ding, Jingdong Wang, et al.
Opengaussian: Towards point-level 3d gaussian-based open vocabulary understanding.
In *The Thirty-Eighth Annual Conference on Neural Information Processing Systems (NeurIPS 2024)*, 2024a.
- Wu et al. (2024b)
Yanmin Wu, Jiarui Meng, Haijie Li, Chenming Wu, Yahao Shi, Xinhua Cheng, Chen Zhao, Haocheng Feng, Errui Ding, Jingdong Wang, et al.
Opengaussian: Towards point-level 3d gaussian-based open vocabulary understanding.
2024b.
- Ye et al. (2024a)
Mingqiao Ye, Martin Danelljan, Fisher Yu, and Lei Ke.
Gaussian grouping: Segment and edit anything in 3d scenes.
In *ECCV*, 2024a.
- Ye et al. (2024b)
Vickie Ye, Ruilong Li, Justin Kerr, Matias Turkulainen, Brent Yi, Zhuoyang Pan, Otto Seiskari, Jianbo Ye, Jeffrey Hu, Matthew Tancik, and Angjoo Kanazawa.
gsplat: An open-source library for Gaussian splatting.
*arXiv preprint arXiv:2409.06765*, 2024b.
- Yin et al. (2024)
Yingda Yin, Yuzheng Liu, Yang Xiao, Daniel Cohen-Or, Jingwei Huang, and Baoquan Chen.
Sai3d: Segment any instance in 3d scenes.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 3292–3302, 2024.
- Yu et al. (2022)
Zehao Yu, Anpei Chen, Bozidar Antic, Songyou Peng, Apratim Bhattacharyya, Michael Niemeyer, Siyu Tang, Torsten Sattler, and Andreas Geiger.
Sdfstudio: A unified framework for surface reconstruction, 2022.
- Yu et al. (2024)
Zehao Yu, Torsten Sattler, and Andreas Geiger.
Gaussian opacity fields: Efficient adaptive surface reconstruction in unbounded scenes.
*ACM Transactions on Graphics (TOG)*, 43(6):1–13, 2024.
- Yunhan Yang and Liu (2023)
Tong He Hengshuang Zhao Yunhan Yang, Xiaoyang Wu and Xihui Liu.
Sam3d: Segment anything in 3d scenes.
In *ICCVW*, 2023.
- Zhang et al. (2024)
Dongbin Zhang, Chuming Wang, Weitao Wang, Peihao Li, Minghan Qin, and Haoqian Wang.
Gaussian in the wild: 3d gaussian splatting for unconstrained image collections.
In *European Conference on Computer Vision*, pages 341–359. Springer, 2024.
- Zhi et al. (2021)
Shuaifeng Zhi, Tristan Laidlow, Stefan Leutenegger, and Andrew J Davison.
In-place scene labelling and understanding with implicit scene representation.
In *Proceedings of the IEEE/CVF International Conference on Computer Vision*, pages 15838–15847, 2021.
- Zhou et al. (2024)
Shijie Zhou, Haoran Chang, Sicheng Jiang, Zhiwen Fan, Zehao Zhu, Dejia Xu, Pradyumna Chari, Suya You, Zhangyang Wang, and Achuta Kadambi.
Feature 3dgs: Supercharging 3d gaussian splatting to enable distilled feature fields.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 21676–21685, 2024.
- Zou et al. (2023)
Xueyan Zou, Jianwei Yang, Hao Zhang, Feng Li, Linjie Li, Jianfeng Wang, Lijuan Wang, Jianfeng Gao, and Yong Jae Lee.
Segment everything everywhere all at once.
*Advances in neural information processing systems*, 36:19769–19782, 2023.
- Zuo et al. (2024)
Xingxing Zuo, Pouya Samangouei, Yunwen Zhou, Yan Di, and Mingyang Li.
Fmgs: Foundation model embedded 3d gaussian splatting for holistic 3d scene understanding.
*International Journal of Computer Vision*, pages 1–17, 2024.

## Peer Reviews

confidence: 4
contribution: 3
initial_rating: 4
presentation: 3
questions: Overall the paper comes with a nice collection of metrics and dataset for 3D object removal benchmarking. I have some questions to be answered:

- How do the authors plan to handle shadows, reflections, and indirect lighting effects that remain outside the object mask after removal?
- Have the authors considered extending their metrics to include contextual regions beyond the mask to better capture these residual cues?
- How does 3D inpainting or neural field completion approaches perform in the benchmark?
- Have the authors conducted any human studies to assess whether semantic residuals are perceptually noticeable in practice?
soundness: 2
strengths: - Remove360

The paper comes with a real world dataset for object removal, i.e., Remove360. The dataset provides a diverse collection of both indoor and outdoor scenes under different lightings with objects pre- and post- removal. It also comes with GT annotations of objects being removed. Remove360 focuses on the multi-object removal scenario which is a better match for its everyday usage. The dataset could serve as a baseline benchmarking dataset for future works.

- Low and high level metrics

The paper proposes several metrics for object removal evaluation. These metrics cover a wide range of quality assessments, from low level depth consistency to high level semantic labelings. Specifically, it evaluates the segmentation accuracy, semantic accuracy, and depth accuracy. Together, it provides a comprehensive, multi-level evaluation framework for quantifying the quality of 3D object removal.
summary: This paper introduces a new dataset and a suite of metrics for evaluating the quality of object removal in 3D Gaussian Splatting. It emphasizes the importance of achieving high-quality, privacy-preserving object removal by examining whether a removed object is truly eliminated or leaves behind semantic residuals that reveal its presence. The proposed metrics jointly assess both low-level visual signals (depth) and high-level semantic consistency, providing a holistic evaluation of removal fidelity. The newly released Remove360 dataset, comprising 11 diverse indoor and outdoor scenes, serves as a comprehensive benchmark for assessing and comparing existing 3DGS-based object removal methods.
weaknesses: - Regions Outside the Object

A common issue in object removal is that shadows, reflections, or indirect effects of the removed objects often remain in the scene. These residual pixels lie outside the object’s mask, yet can still reveal visual or semantic cues about what was removed. The paper’s proposed metrics primarily focus on evaluating changes within the masked region, without explicitly considering these contextual regions outside the mask. Such unaddressed residuals may still compromise privacy or expose identifiable traces. For instance, as shown in Figure 2, leftover shadows remain visible but are not penalized by the current evaluation framework.

- Focus Limited to 3DGS-Based Methods

The paper’s evaluation framework and experiments concentrate exclusively on 3DGS–based object removal techniques. While this focus provides depth and clarity, it overlooks alternative paradigms such as 3D inpainting or neural field completion methods, which also play an important role in 3D scene editing and privacy-preserving reconstruction. 3D inpainting method, e.g., Inpaint3D, takes advantage of diffusion-based priors to fill in missing regions and handle long-range dependencies such as shadows or reflections. Although 3D consistency is not guaranteed, these two types of methods are on the different ends of the spectrum and should both be evaluated.

---

confidence: 3
contribution: 2
initial_rating: 4
presentation: 2
questions: My main questions concern the reliability of IoU_drop and sim_SAM, and the comparison with previous pixel-level metrics like PSNR, SSIM, and LPIPS. Please refer to the weaknesses for details.
soundness: 2
strengths: 1. The paper focuses on the critical issue of invisible "semantic residuals" that may persist in scene data after 3D removal. To this end, it proposes a comprehensive evaluation from multiple dimensions, including semantics and segmentation.   
2. The paper releases the Remove360 dataset, which includes pre-/post-removal RGB images and object-level masks from real-world scenes. Providing these scenes after physical object removal to serve as ground truth offers a solid basis for evaluating removal capabilities. It is also noteworthy that the dataset has garnered significant attention, with over 2,600 downloads in the last month.
summary: This paper aims to quantify the semantic traces remaining after object removal in 3D Gaussian Splatting. The core argument is that even when an object is visually removed successfully, invisible "semantic residuals" may still persist in the scene data. To address this challenge, the research introduces an evaluation framework and releases the Remove360 dataset, which contains pre-/post-removal RGB images and object-level masks from real-world scenes.
weaknesses: 1. The paper lacks a comparison with previous evaluation metrics such as PSNR, SSIM, and LPIPS. It would strengthen the paper's argument to demonstrate cases where the difference lies between these traditional metrics and the proposed metrics.
2. Regarding the paper's assertion that "the model can detect residual information invisible to the human eye": It is possible that the inpainted shape is similar to the original, or that artifacts introduced during the removal and inpainting process coincidentally trigger erroneous recognition by the segmentation model. These issues may stem from errors in the segmentation model itself, rather than flaws in the removal algorithm.   
3. Concerning post-removal recognizability: If the removed area were replaced entirely with noise, it would also be unrecognizable. In such a scenario, the reliability of metrics like IoU_drop and sim_SAM would be questionable, as they might not distinguish a successful, context-aware inpainting from simple noise filling.

---

confidence: 5
contribution: 2
initial_rating: 4
presentation: 3
questions: 1. The definition of semantic residuals is not clear.
2. This work only compares among existing methods, no technique novelty.
3. The introduction of the proposed method are incomplete .
4. This work requires more ablation studies.
soundness: 2
strengths: 1. A new dataset is released to measure the object removal, containing indoor and outdoor scene, which boosts the community.
2. This paper is well-written and the measurement method is easy to understand.
summary: This paper investigates the persistence of semantic information after object removal in 3D scene reconstruction, with a particular focus on privacy-preserving 3D Gaussian Splatting. The authors introduce Remove360, a new benchmark dataset containing real-world indoor and outdoor scenes with paired pre- and post-removal RGB images and object masks. In addition, they propose a set of four quantitative metrics based on semantic segmentation drop, recognition accuracy, instance mask similarity, and depth change to evaluate how much semantic information remains after object removal. Using five state-of-the-art 3DGS-based methods, the authors show that all existing approaches leave detectable semantic residuals, even when removed objects are visually absent. The work highlights key limitations in current 3D editing pipelines and provides a valuable benchmark for developing future privacy-aware scene manipulation methods.
weaknesses: 1. In the introduction, the definition of semantic residuals is unclear and lacks a rigorous mathematical or task-based formulation, which is critical for understanding and designing the subsequent evaluation metrics.

2. Although semantic residuals in 3DGS have been visualized, a deeper analysis is required to understand their underlying causes.

3. In the related work section, the authors only discuss 3D reconstruction and semantics in privacy protection applications. Object removal, however, is a subcategory of 3D scene editing, such as NeRF-based or 3DGS-based editing, and the relationships between these relevant studies and the current work should be discussed in more depth.

4. The authors mention privacy challenges across different domains, but a systematic discussion of how existing methods protect privacy is missing.

5. In the metric definition, the authors propose using semantic object recognition to measure the mIoU drop for identifying semantic residuals. However, segmentation performance is highly dependent on the chosen model, and altering the segmentation model could lead to inconsistent score drops, thereby introducing bias into the proposed metric.

6. Some important statistics, such as the number of objects in the dataset, are not provided.

7. While the experiments are comprehensive, they are entirely built upon existing techniques, with no newly designed method to demonstrate the authors’ own contribution.

8. The ablation studies are insufficient. In the metric definition, the authors employ several hyperparameters, yet key ones, such as the depth threshold, are not analyzed, which raises concerns about the robustness of the proposed metric.

9. The paper states that “operating on the 3D model offers privacy at a reasonable computational cost and better reconstruction quality.” However, a quantitative comparison of computational costs between the proposed 3D method and image-based editing is not provided.

10. In the discussion section, the authors mention that semantic residuals are difficult to eliminate, but no potential solutions or future directions are suggested.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: -How sensitive are the evaluation metrics to errors in the ground-truth object masks or the predictions of the off-the-shelf semantic segmentation models? Can you provide quantitative results on this sensitivity?

-The paper mentions combining metrics but doesn't provide details on how this is done. Are the metrics equally weighted, or is there a more sophisticated combination strategy? What is the justification for the chosen combination method?

-Can you provide a more detailed analysis of specific failure cases observed in the experiments, including potential reasons for these failures?
soundness: 3
strengths: -The paper addresses a critical and previously underexplored aspect of 3D scene editing – the presence of semantic residuals after object removal, particularly relevant for privacy-preserving applications.
-The proposed evaluation framework combines multiple complementary metrics (semantic, segmentation, and depth-based) to provide a thorough assessment of object removal effectiveness.
-The Remove360 dataset fills a gap by providing a diverse set of real-world indoor and outdoor scenes with pre/post-removal captures and object masks, better reflecting real-world complexities than existing datasets.
summary: The paper introduces a new benchmark and evaluation framework for assessing the effectiveness of object removal techniques in 3D Gaussian Splatting, focusing on whether semantic residuals remain after the removal process. It presents Remove360, a dataset of real-world indoor and outdoor scenes with pre- and post-removal RGB images and object masks.  The paper evaluates several existing object removal methods using a combination of semantic, segmentation, and depth-based metrics to quantify the presence of unintended semantic traces after object removal. The findings highlight limitations in current 3D object removal techniques and emphasize the need for more robust solutions.
weaknesses: -The evaluation metrics rely on off-the-shelf semantic segmentation models, which may introduce errors and limit the robustness of the evaluation. The paper acknowledges this but could benefit from further discussion of potential biases and limitations.

-Limited Analysis of Failure Cases: While the paper mentions failure cases (residual artifacts, incomplete removal, over-smoothing), a more detailed analysis of specific failure modes would be valuable.