GeoRemover: Removing Objects and Their Causal Visual Artifacts
Authors: Zixin Zhu
Haoxiang Li
Xuelu Feng
He Wu
Chunming Qiao
Junsong Yuan
Venue: NeurIPS 2025 spotlight ()
Keywords: Object removal
diffusion
Geometry
Primary area: applications

Abstract: Towards intelligent image editing, object removal should eliminate both the target object and its causal visual artifacts, such as shadows and reflections. However, existing image appearance-based methods either follow strictly mask-aligned training and fail to remove these casual effects which are not explicitly masked, or adopt loosely mask-aligned strategies that lack controllability and may unintentionally over-erase other objects. We identify that these limitations stem from ignoring the causal relationship between an object’s geometry presence and its visual effects. To address this limitation, we propose a geometry-aware two-stage framework that decouples object removal into (1) geometry removal and (2) appearance rendering. In the first stage, we remove the object directly from the geometry (e.g., depth) using strictly mask-aligned supervision, enabling structure-aware editing with strong geometric constraints. In the second stage, we render a photorealistic RGB image conditioned on the updated geometry, where causal visual effects are considered implicitly as a result of the modified 3D geometry. To guide learning in the geometry removal stage, we introduce a preference-driven objective based on positive and negative sample pairs, encouraging the model to remove objects as well as their causal visual artifacts while avoiding new structural insertions. Extensive experiments demonstrate that our method achieves state-of-the-art performance in removing both objects and their associated artifacts on two popular benchmarks. The project page is available at https://buxiangzhiren.github.io/GeoRemover.

GeoRemover: Removing Objects and Their Causal Visual Artifacts
Zixin Zhu 1.2 Haoxiang Lift Xuelu Feng'
He Wu?
Chunming Qiao'
Junsong Yuan'
University at Buffalo
2 Pixociil Technology
(zixinzhu, xuelufen, qiao, jeyuan)@buffalo.cm
haoxiang. lispixociall...............iiiii com, heu1998250gmail.com
Abstract
Towards intelligent image editing, object removal should climinate both the target object and its causal visual artifacts, such as shadows and reflections. However, ex- isting image appearance-based methods either follow strictly mask-aligned training and fail to remove these casual effects which are not explicitly masked, or adopt loosely mask-alignes strategies that lack controllability and may unintentionally over-erase other objects. We identify that these limitations stem from ignoring the causal relationship between an object's geometry presence and its visual effects. To address this limitation, we propose geometry-aware two-stage framework that decouples object removal into (1) geometry removal and (2) appearance rendering. In the first stage, we remove the object directly from the geometry (e.g.. depth) us- ing strictly mask-aligned supervision, enabling structure-aware editing with strong geometric constraints. In the second stage, we render a photorealistic RGB image conditioned on the updated geometry, where causal visual effects are considered im- plicitly as a result of the modified 3D geometry. To guide learning in the geometry removal stage, introduce preference-drie driven objective based on positive and neg- ative sample pairs, encouraging the model to remove objects as well as their causal visual artifacts while avoiding structural insertions. Extensive experiments demonstrate that our method achieves state-of-the-as performance in removing both objects and their associated artifacts page is available at https: ing.bbiiingggren.n.... gi sithub... io/ GeoRRemover
popular benchmarks. The project
Introduction
Object removal is a challenging computer vision task with applications in image editing and scene rendering, aiming to erase undesired objects as if they never present. Following the inpainting framework [1 2 4. traditional strictly mask-aligned approaches [5] 6 7 assume that user specified mask fully covers the objects to be removed, thus only deal with the masked region while do not change the remained image. However, in real-world scenarios, objects often cast causal visual artifacts (e.g., shadows and reflections) onto surrounding regions, leading to illumination inconsistencies beyond the masked removed, his shadow remains as the causal artifact. simple solution to address such an associated artifact is to extend the object removal mask to cover these artifacts, but this places a significant burden on users, who must identify and annotate all subtle, detached, and ambiguous artifacts. As result, this approach is neither scalable nor user-friendly.
As illustrated in Fig. Fig. [p, although the child is successfully
Therefore, recent methods 9
assume a more practical and user-friendly setting where the input mask only covers the objects to remove, but the model implicitly infers and removes causal visual artifacts such as shadows and reflections in an intelligent way. For example, previous methods
10
Work completed while author was an intern at Pixocial Technology. Conresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Training: Mask Area
Edit Area
Mask Tells Edit Areas
Training: Mask Arec = Edit Area
Mask connot Tell Edit Areas
n
M
Edit Area (Masked Object Area)
Cousal Visual Artifacts
Edit Area (Maskap Object Anea)
Edit Area Confusiin
Edit Area (Unmasked Shadow Area)
(o) Strictly Mask aligned Training
(b) Loosely Mask aligned Training
Stage Geometry Removal
Training: Edit
Mask Telle
Stage 2: Appearance Rendening
Area (Mosked Object Area)
Without
(c) Ours
Figure 1: Comparison of object removal training paradigms. (a) Strictly mask-aligned training edits only masked regions but leaves causal visual artifacts (shadow) unaddressed. (b) Loosely mask-aligned training allows broader context-aware corrections but lacks clear guidance, leading to confusion and uncontrollable edits. (c) Our method decouples geometry and appearance for object removal: we first edit the scene geometric representation (in the form of depth map) under strictly mask-aligned supervision, then render realistic image where both objects and causal visual artifacts (shadow) are cleanly removed.
have attempted adopt loosely mask aligned training strategies, encouraging models infer and correct inconsistencies beyond explicitly masked regions specified by the user. However, without specific design to guide the editing, most of them heavily rely on paired training data, which hinders controllability. Compared to strictly mask-aligned training, where the mask explicitly defines which regions can be modified and which must preserved, loosely mask-aligned settings, both the masked and unmasked regions may require edits, but the model itself has clear boundary guidance, leading to confusion about where modifications should occur. As shown in Fig. [p, while the model successfully removes the child's shadow, it also mistakenly removes the nearby adult, resulting in unintended alterations to the scene.
The above challenges suggest that solely optimizing training strategies is insufficient to enable models to reason about causal visual effects. We make key observation: these effects, such as shadows and reflections, are fundamentally caused by the object's geometry under specific lighting conditions. In other words, the geometry presence the cause, and the causal visual effects are its consequence. Intuitively, if the object's presence is removed from the scene geometry, then its associated illumination effects should no longer exist.
This insight motivates us to rethink object removal as a causal reasoning process: we firstly modify the geometric representation (c.g., via modifying the depth maps) to remove the object presence from the scene geometry; then, we render new image appearance based on the updated scene geometry. where causal visual artifacts will be naturally removed. This progressive design offers two key advantages. First, in the geometry removal stage, we can adopt strictly mask-aligned training: since causal visual artifacts do not need be considered the geometry domain, and object boundaries are clearly defined, thus the model can focus on removing only the object in masked region, making the task well-posed with strong supervision. This eliminates the risk of undesired modifications to unmasked regions. Second, in the rendering stage, the absence of the object naturally leads to the

---

removal of its associated artifacts. This implicitly enforces causal consistency by removing both the object and its visual artifacts. To enable this behavior, we train the rendering model with paired data: each pair consists of an image with the object and its causal effects (e.g., shadows or reflections), and the corresponding image shows the scene with both the object and its effects removed. By first localizing the removed object based on geometric differences, the rendering model can then establish the causal relationship between the object and its associated effects by analyzing the visual differences between the paired images. As shown in Fig. Ic. our method successfully removes both the object and its shadow, while preserving nearby unmasked content. Our contributions can be summarized as:
We propose a new two-stage framework leverage geometric representation decouple object removal into geometry removal and appearance rendering. Based on our observation that the geometric representation is free from causal visual artifacts, our method erases masked objects from the scene geometry followed by the removal of their visual artifacts.
To improve object removal quality in the geometric representation, we introduce a preference- guided loss prevent the model from inserting unexpected structures.
Compared to existing methods that utilize loosely mask-aligned training strategies ap- proach this problem in the same setting, which mostly suffer from loss of controllability and the unintended alteration issue, we demonstrate through experiments that the proposed framework improves the removal quality on two benchmark datasets.
2 Related work
Object removal and inpainting. Object removal is traditionally formulated as an inpainting
17 [8]. Most approaches [19 20 7 trained in a strictly mask-aligned manner, enabling precise control over the masked region. For instance, ClipAway [5] leverages harmonized CLIP embeddings to guide removal, SmartEraser (21] and Inst-Inpaint [6] explore instruction-baser or maskless generation. But they often leaving behind causal visual artifacts such as shadows and reflections. To address this, loosely mask-aligned methods [9] TO] [T] expand the removal scope beyond the user-defined mask automatically cleaning surrounding artifacts. However, this comes at the cost of reduced controllability and increased risk of over-editing. Different from them, our method bridges the gap between strictly mask-aligned precision and loosely mask-aligne flexibility by decoupling geometry and appearance, allowing for structure-awand editing and implicit removal of causal visual artifacts.
problem, where the model fills user-specified mask with realistic content [12 13 [4
[5
16
Geometry-aware generation. Incorporating geometric priors such as depth maps has shown
promise in editing [22], scene completion [23]. and novel view synthesis 24 25 While many prior methods use geometry as auxiliary input during single-stage generation 26 27,, they typically entangle structure and appearance modeling. Unlike prior methods that use depth as auxiliary input, we decouple geometry and appearance into two stages: editing in depth space and rendering in RGB. This design enables controllable structure editing and implicit removal of causal visual artifacts.
Diffusion preference optimization and human alignment. Preference-guidee training has emerged as practical alternative to supervised learning for aligning generative models with human intent 28 29 30 31. For instance, DPO (8 extends preference optimization to diffusion models by learning from ranked pairs. Other works 22 30 31 explore human alignment through benehmark design, direct reward modeling, and multi-dimensional preference decomposition for text-to-image generation. Inspired by these ideas, we adopt a DPO-style strategy in our geometry removal stage: instead of human-annotated rankings, we define preferences based on geometric flow smoothness, encouraging plausible structure completion while suppressing spurious insertions.
3 Method
3.1
Problem formulation
Given an input image I- E Hx x3 and an object mask M E {0, H1#x indicating the region to be removed, our goal is to generate an output image I+ E xxxx in which the object has been cleanly erased, its contextual effects (c.g., shadows reflections) removed, and the background
3

---

Positive
Negative Path
Training Path
Images
Element Subtroction
Condition
Conditions
log (c)
Flow Smoothhn thing
Depth Estimotor
Depth Mops
Diffusion
LoRA
Noised Depth Maps
log
Depth Flows
Figure 2: The training framework of Stage I: Geometry Removal. Given an input image and object mask, we first estimate the geometric representation (in the form of a depth map) and construct masked geometry input. The masked depth map, together with the mask, is then fed into diffusion model to predict the edited geometry. To discourage structure insertion and encourage object removal, we construct two geometry completion paths: a positíve path where the object is successfully removed with smooth depth flow, and a negative path where the object remains with sharp depth transitions. The model is trained to prefer the positive path and suppress the negative one.
realistically restored. Most existing methods formulate this problem as a direct image-to-image transformation task, learning mapping
where g a function that maps the masked image and mask to completed result. However, as illus- trated in Fig. Ta and Fig. Tb. such formulations often entangle geometric reasoning with appearance synthesis, making difficult to control structural edits and leading to unintended modifications.
To address this, we approach this problem by decoupling it into two sub-tasks: (1) geometry removal, which modifies the geometric representation to eliminate the object while preserving the surrounding structure; and (2) appearance rendering, which synthesizes an RGB image consistent with the updated geometry from geometry removal. This decoupling allows from pixel-level synthesis, and enables causal visual artifacts to be implicitly corrected through geometry-aware rendering. Formally, decompose the object removal process
separate structure-level editing
= D(1)), z = se M).
C , Z0 x
(2)
Stage geometry removal
Stage appearance rendering
where 'D is a geometry estimator, Ty is the estimated geometric representation of the input im- age, 2xJ = sp (x , .M) is the updated geometry predicted by the diffusion model under strictly mask-aligned supervision, and G synthesizes the final RGB output conditioned on the geometric transformation from zy to Tg and the input image [_
3.2 Stage 1: geometry removal
Geometry completion with strictly mask-aligned training. In the first stage, our goal is to remove the target object by modifying the scene geometry, while preserving the surrounding structure. We use depth as the geometric representation in this work due to the efficiency and accuracy of recent depth estimation models. Geometry removal is performed in the depth domain, where causal visual artifacts such as shadows and reflections do not appear, making the task well-suited for strictly mask-aligned supervision. The overall training pipeline is illustrated in Fig. 2 Formally, given an input RGB image I, an estimated depth map Fp, and an object mask M E {0. T) H indicating the removal region, the objective is to learn model that predicts edited depth map do where the object is removed within the masked region, while preserving geometry elsewhere. We enforce the constraint
L (i(i.) zo(i,)), V(i,j) where M 0.
naive solution to this task to treat the depth map as a colorized image and fine-tune a pre-trained diffusion-basee image inpainting model for depth editing. To maximize the log-likelihood log p(2n c), diffusion-based models optimize a denoising score matching objective, which minimizes the discrepancy between the model- predictdd score sp t, c) and the true score V 2, log p(æ, zo, c).

---

Here, Sp is parameterized score function, and (M, (1 M) * zo) denotes the conditioning input (i.e., the object mask and the masked depth map). The score matching loss is
(To. c) Er,,, [u(tt ( f, c) Va, log Io, c)lc]
(4)
where is noisy sample generated from zo and w(() ) is a weighting function over timesteps.
Preference-guided geometry completion via DPO. However, when applying this baseline directly, we observe that the model often hallucinates new structures within the masked region, shown in the second row Fig. 3a Rather than recovering a coherent surface, tends to insert unrealistic geometry that does not align with the surrounding structure. We hypothesize that this behavior arises from the lack geometry-aware constraints: without explicit structural supervision, the model cannot distinguish between completing missing surfaces and generating new, implausible content. To avoid hallucinating new content within the object removal region. inspired by recent advances in Direct Preference Optimization (DPO) (8). propose to model geometry removal through reward-based framework. DPO aims to align model outputs with user preferences by optimizing over ranked sample pairs, rather than relying solely explicit ground truth labels.
In our setting, we adopt a similar philosophy: we define preferences over geometry, where the depth in the masked region that does not contain the object is what we prefer, and the depth that includes the object is what we do not prefer. Ideally, a preferred depth map should be locally smooth inside the mask, with minimal abrupt depth changes that would otherwise indicate the presence of an object. As shown in Fig. 2 when the masked region contains an object (e.g., a dog), the depth flow, defined as the spatial gradient of depth values, exhibits sharp discontinuities due to the object's geometry. In contrast, when the object is successfully removed from the mask, the depth flow approaches zero, indicating smooth and coherent surface. Therefore, consider low depth flow within the mask as a key signal for realistic and desirable geometry.
We define the reward as monotonic function of log p(ro c), based on flow difference between the predicted depth map dn and ground-truth zo, measured by the flow loss Lnow (fo, zo). The reward is
r(c, mo) - Lnoow (ăo, zo) f (logp(xo c)). with f' 0.
(5)
Then we introduce how we define the flow loss (2o, za). Specifically, let de denote the depth value at pixel (i,j). The flow at pixel (i, j) is defined as the first-order spatial gradient
Ezy(2) = {|d++ di |d.++ d.gl).
(6)
which captures local depth transitions horizontal and vertical directions. Then the flow loss is defined as the average of per-pixel absolute differences between the predicted flow and the ground- truth flow. is
Lnow (2o, Fo)
1
Σ (i.j)en
[F))( Fiy(za))l
where Ω denotes the set of valid pixels. With the reward r(c, xo), to model preference between completions, we assume access to ranked sample pairs (2 ). indicating that To is preferred over zo under the same conditioning c. As illustrated in Fig. 2 we refer to these as the positive and negative geometry paths, respectively. We adopt the Bradley-Terry (BT) model to express the preference probability
LBT -E,,, :5 [log (r(c,at) r(c,,,))
The final loss combines the standard diffusion loss with the preference-guiee guided objective, which is
L = LDsM + X LBT:
where À balances score-based denoising supervision and geometry-consistent preference learning.
3.3 Stage 2: appearance rendering
In the second stage, our goal is to generate a realistic RGB image that reflects the scene after object removal, as defined by the updated geometry from Sec. 3.22 We formulate this task as conditional image translation problem, where the appearance of the output image is controlled by the geometric
5

---

Renoved Image
Conditiord
Condition2
Image Translation (Ieeertion)
Dificiion
Image (Kemoval)
wi oPo
Conditionn
Condition2
(a) Effect of direct preference optimization (DPO) in Stage 1.
(b) Stage 2: Appearance rendering.
Figure 3: (a) We compare model outputs trained with and without our DPO objective (i.e., LBT). Without DPO, the model often inserts retains undesired content in the masked region, leading to unrealistic geometry. With DPO, the model learns to prefer geometry completions that successfully remove the object while preserving surrounding structures. (b) The training framework of Stage 2. Given geometry-awar conditions (e.g., depth maps), we train a diffusion model to perform image translation for both object removal and insertion.
transformation between two depth maps. As illustrated in Fig. 3b the model takes as input a masked RGB image and geometry-aware conditions: Condition and Condition2. Both conditions are represented as depth maps: Condition encodes the geometry of the input image (e.g.. with an object present), while Condition2 defines the target geometry (e.g., with the object removed). The difference between Condition and Condition2 specifies the structural transformation to be performed. For example, if Condition2 removes an object that exists in Conditionl, the model guided erase that object and inpaint realistic background. Conversely, if Condition2 introduces localized depth discontinuity compared to flat Conditionl, the model learns insert a visually plausible object.
Formally, we define G as a connetrr--onddiiooned image translation model based on a diffusion backbone. The model takes as input a RGB image and a pair of depth maps indicating the geometry before and after editing. Let /- and I+ denote the input and output images, and let zg and a represent the corresponding depth maps before and after editing. The model learns perform both object removal and insertion through the following bidirectional formulation:
I' = G( nσ 20), r
g(I x =7))
(10)
Further details of the loss formulation for this stage are provided Appendix A
Experiments
Implementation details. The depth estimator ' is implemented using Depth Anything 33 For both geometry removal model (sg) and appearance rendering model (9), we adopt 1- dev [8] as the pre-trained diffusion backbone, and apply LoRA (33) fine-tuning with a rank of 64. All images are processed at a resolution of 1024x1024 For both stages, we use batch size of 24, learning rate of 1 x 10-4 4, and guidance scale of 1.0. The text prompt "a beautiful scene" is used during both training and inference. Stage is trained for 17,000 steps on NVIDIA H100 GPUs, taking approximately 24 hours, while Stage 2 requires around 60 hours for the same number of steps.
Datasets & Metrics. We use the training set from the RORD 34 dataset as our primary training data. RORD is large-scale real-world object removal dataset consisting of 516,705 images captured under 3,447 unique indoor scenes. Each scene contains paired images with and without the target object, along with manually annotated object masks. The dataset is designed to support training and evaluation for object removal and scene completion tasks in realistic environments. For evaluation, we follow prior works such as SmartEraser [21] and OmniEraser (9) We use both RORD Val and RemovalBench as our primary benchmarks. Moreover, we follow prior works 35
set of metrics to evaluate image generation quality. We use Frechet Inception Distance and adopt CLIP Maximum Mean Discrepancy (CMMD) Aesthetic Score (AS) 38 Learned Perceptual
Image Patch Similarity (LPIPS) 39] and Peak Signal-to-Noise Ratio (PSNR) [40].

---

Table 1: Comparison with Saate-of-the-ans methods on RemovalBench and RORD-VaL.
RemovalBench CMMD
RORD-Val
Method
FID
LPIPS
PSNR t AS f
FID 4 CMMD 4 LPIPS
PSNR
AS f
ZITS++
108.38
0.374
0.158
19.62
4.56
107.44
0.448
0.274
21.17
4.12
MAT 19
123.78
0.366
0.164
17.88
4.51
136.53
0.455
0,281
19.18
4.38
LaMa 42
99.88
0.351
0.156
18.72
4.55
100.21
0.294
0.229
20.50
4.23
RePaint 20
102.65
0.741
0.378
19.86
4.38
114.64
2.34S
0.525
17.68
4.71
BLD 43
128.66
0.553
0.233
17.43
4.39
224.61
0.862
0.273
17.13
4.74
LDM 7
108.79
0.365
0.157
19.24
4.47
128.19
0.506
0,221
19.02
4.12
SD-Inpaint
119.60
0.419
0.274
17.02
4.48
143.69
0.494
0.308
16.83
4.61
SDXL-Inpaint 7
104.97
0.398
0.187
17.87
4.63
147.01
0.460
0.210
17.69
4.76
BrushNet 35
120.97
0.549
0.191
18.68
4.63
234.87
0.745
0.293
16.51
4.41
FLUX.1-Fill 8
115.79
0.487
0.193
17.12
4.59
141.39
0.450
0,217
18,50
4.55
PowerPaint
114.55
0.392
0.240
18.25
4.56
102.33
0.408
0,241
18.29
4.38
CLiPAway 5
108.40
0.272
0.254
18.78
4.48
81.28
0,545
0.278
16.36
4.19
Seentive-Erasee
45
55,49
0.232
0.146
20.60
4.50
96.77
0.233
0,221
20.24
4.77
OmniEraser 9
39.52
0.208
0.133
21.11
4.66
43.71
0.153
0.166
22.13
4.99
Ours
29.88
0.089
0.124
25.52
4.54
31.15
0.182
0.103
23.70
4.69
Table 2: Ablation study on RORD-Va Val to evaluate the effectiveness of our design components. "Insert." denotes the percentage of cases where a new object is wrongly inserted into the removal region.
Table 3: Geometry removal accuracy (MAE masked re-
gion) on RORD- Val.
Method
FID CMMD LLIIP PSNR 1 AS |lnsert.
Method
MAE
One-Stage
56.24
0.577
0.315
17.52
4.27
2.81%
Input depth
0.0827
Two-Stage w/o DPO 34.24
0.230
0.131
22.81
4.51
5.09%
Two-Stage w/o DPO
0.0490
Two-Stage w/ DPO 31.15
0.182
0.103
23.70
4.69
1.48%
Two-Stage w/ DPO
0.0387
4.1 Comparison with SOTA methods
We compare our method against state-of-the-ar approaches the RemovalBench and RORD- Val datasets, as shown Tab. []] These baselines fall into two categories: strictly aaigned methods and loosely mask-aligned methods. Strictly mask-aligned methods (20) 19 5 45) are limited in their ability to handle contextual effects, since they are confined to the object region defined by the In contrast, loosely maik-aligned methods can adaptively clean surrounding regions affected by the object. Among loosely mask-aligned approaches, OmniEraser [9] is the only open-sourge method that supports causal visual artifacts removal. Although models like OmniPaint [II] and ObjectDrop [10] also aim to remove such effects, their code and models are not publicly available, and were therefore unable include them in evaluation. Across both benchmarks, our method consistently achieves the best scores in FID, CMMD, LPIPS, and PSNR, demonstrating superior visual quality and structure preservation in the removed regions.
4.2 Ablation study and discussion
Is the two-stage design necessary? Compared to prior one-stage approaches, our method intro- duces two key innovations: (I) a two-stage design that explicitly decouples geometry and appearance, and (2) the incorporation of geometric cues such as depth to guide object removal. To fairly isolate the contribution of the two-stage architecture, we construct one-stage version of our method that also takes the masked RGB image and masked depth map as input to the diffusion model. This ensures that both models operate on the same input modalities, and any performance gap can be attributed to the architectural difference. As shown in Fig. 4 despite access to depth information, the one-stage model often produces ambiguous or distorted edits due to the lack of explicit geometric guidance. Quantitative results in Tab. further confirm that the one-stage variant consistently underperforms the two-stage models across multiple metrics. This supports claim that the decoupled design, rather than merely the inclusion of depth, that enables model to reason more effectively. While the
7

---

Table 4: Removal performance of Table 5: Ablation study the RORD-Ve Val dataset comparing
causal artifacts on CausRem.
unidirectional and bidirectional rendering strategies in Stage 2.
Method
loU%
Method
FID CMMD & LPIPS PSNR AS
OmniEraser
9
68.29
Unidirectional rendering
38,43
0.215
0.136
23.58
4.19
Ours
73.76
Bidirectional rendering
31,15
0.182
0.103
23.70
4.69
Worked Tmage Macker Deyth wop
Edited Trage
Eriter Deyth Mop
Dmace
Mathes bepth
Edited Depth
Cencrteauted Tepur doey Walts Diwendas
Structarilyy Contosed Ortput
Staye Geabetty Reeadd
Staye F: pesioooe Benslnnin
(a) Results from one-stage model.
(b) Results from two-stage model
Figure 4: Comparison between our one-stage and two-stage object removal strategies. Two-stage design improves edit quality by separating geometry reasoning from appearance generation.
Wheng Deptt Mep
Moxted bastt Mep
Files Degri Hap
Lecd Nox Depth
EvvertrEstteet
Steget deanetry Beeev!
Woogy bepth
Teage
Trage
Filled Depth Map
Prediched bogth Meg
Trgat Image
Gatpet Image
Stape:: Appearance Sandering
Ramava Pulled
Stap22: Appearance Randering
Renova Sucoussfuly
(a) Failure cases under motion blur conditions,
(b) Improved results after applying Fill-in strategy.
Figure 5: Depth errors caused by motion blur result in removal failure. Applying a simple Fill-in strategy within the mask restores geometric contrast and yields correct removal.
two-stage design increases runtime, the gains in controllability and illumination consistency justify the cost.
How does the DPO strategy help model achieve better removal? To evaluate the effect of Direct Preference Optimization (DPO), we compare our two-stage model with and without DPO supervision (LBT). As shown in Tab. 2 DPO significantly reduces the "Insert." rate from 5.08% to 1.48%, indicating its effectiveness in suppressing semantic hallucinations. This result suggests that preference-driven learning helps the model better align with human expectations for clean and plausible object removal.
How accurate is geometry removal Stage 1? To evaluate geometry removal quality, we compute the mean absolute error (MAE) between the predicted and ground-truth depth maps within the masked region, which corresponds to the object intended for removal. For reference, we also report the MAE between the input depth map and ground-truth depth map in the same region, providing baseline for understanding the original geometric discrepancy. As shown in Tab, 3 our two-stage model significantly reduces the depth error within the masked region, and incorporating DPO further improves removal precision.
How effectively does our model remove causal visual artifacts? To evaluate our method's 's ability to remove causal visual artifacts, we construct the CausRem benchmark, consisting of 200 real- world images (100 with shadows, 100 with reflections). Each image is manually annotated with object masks and corresponding artifact masks (shadows or reflections). Representative samples are provided in Appendix E To estimate where the model implicitly removes such artifacts, we compute the pixel-wise absolute difference between the input and output within the annotated artifact regions. A threshold is then applied to identify significantly altered pixels, indicating predicted residue areas.

---

Masked Image
Attentive-Eraser
CLIPAway
OmniEraser
FLUX.1-Fill
Ours
Figure 6: Qualitative comparison with state-of-the-art methods on CausRem.

---

To set a robust threshold, analyze the boundary regions of the annotated causal visual artifact masks, where pixel values transition between the artifact and the background. We find the average difference in these regions to be approximately 20, which we adopt as fixed global threshold. We evaluate the predicted residue regions using loU against the ground truth. As shown Tab. our method achieves 73.76 73.76% IoU, outperforming the previous state-of-the-a OmniEraser
9 at 68. 29%.
Tab. Jub.5p demonstrates that bidirec- tional rendering enhances Stage 2 performance by promoting more precise alignment between the refined geometry and the synthesized image. By enforcing consistency in both removal and insertion rendering directions, the model is encouraged maintain structural coherence throughout the image, leading to improved visual quality and reduced artifacts in the final output.
Does bidirectional rendering improve performance in Stage 2?
4.3 Qualitative comparison on CausRem
Fig- Sisows a visual comparison of object and causal artifact removal (e.g., shadows, reflections) on the CausRem dataset. Compared with state-of-the-art methods, approach yields cleaner results. Prior methods like Attentive-Eraser and CLIPAway often leave shadows or blur the background. while OmniEraser may distort nearby textures. In contrast, our method removes both objects and their effects cleanly by leveraging geometry-gune guided rendering, preserving background structure.
Failure case
Despite the overall robustness of our two-stage design, failures can still arise when the geometric signal inside the mask weak or unreliable. Typical situations include fast motion, translucency, specular/reflectivee surfaces, occlusions, or low texture, where the "geometry-removed" depth becomes nearly indistinguishable from the input (Fig. [a)). Because Stage 2 identifies removal targets by differencing these two depth maps, the lack of geometrie contrast prevents it from triggering removal.
We address this with simple Local Max Depth Fill-in: for masked pixels lacking reliable estimates, we propagate the maximum depth from small local neighborhood (e.g., 10x 10 pixels). This lightweight completion restores boundary contrast and enables Stage remove the target while preserving a coherent background (Fig. (5)) Moreover, we provide additional challenging cases in Appendix D including transparent and reflective surfaces, self-emitting scenes, and failures from incomplete masks, together with practical mitigations.
6 Conclusion
We present a geometry-awand two-stage framework for object removal that effectively handles both the primary object and its associated causal visual artifacts. By decoupling the task into geometry removal and appearance rendering, our method achieves precise structural editing and seamless image restoration. Extensive experiments across multiple benchmarks validate that our approach outperforms existing methods in both quantitative metries and visual quality, especially in challenging cases involving shadows and reflections.
Broader impacts. Our work advances the controllability and accuracy of object removal systems, which can benefit applications in autonomous driving, AR/VR content editing, and photo restoration. However, the enhanced controllability of visual content manipulation also raises potential risks such as deepfakes. Mitigating these risks requires responsible deployment, provenance tracking, and clear guidelines for model usage.
Acknowledgments
This work is supported in part by the National Science Foundation (NSF) and the Institute of Education Sciences (IES) through Award #2229873 (AI Institute for Transforming Education for Children with Speech and Language Processing Challenges) and NSF CNS 2413876 and CNS-2120369.
10

---

References
[1] Ciprian Comeanu, Raghudeep Gadde, and Aleix M Martinez. Latentpaint: Image inpainting in latent space with diffusion models. In Proceedings of the IEEE/CVF winter conference applications of computer vision, pages 4334 4343, 2024.
[2] Haipeng Liu, Yang Wang, Biao Qian, Meng Wang, and Yong Rui. Structure matters: Tackling the semantic discrepancy in diffusion models for image impainting. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 5038-8047 2024.
[3] Yingchen Yu, Fangneng Zhan, Shijian Lu, Jianxiong Pan, Feiying Ma, Xuansong Xie, and Chunyan Miao. Waveñill: A wavelet-based generation network for image inpainting. In Proceedings of the IEEE/CVF international conference on computer vision, pages 14114- 14123, 2021.
[4] Shaoan Xie, Zhifei Zhang, Zhe Lin, Tobias Hinz, and Kun Zhang. Smartbrush: Text and shape guided object inpainting with diffusion model. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 22428-22437, 2023.
[5] Yigit Ekin, Ahmet Burak Yildirim, Erdem Eren Çağlar, Aykut Erdem, Erkut Erdem, and Aysegul Dundar. Clipaway: Harmonizing focused embeddings for removing objects via diffusion models. Advances in Neural Information Processing Systems, 37:17572-17601 2024.
[6] Ahmet Burak Yildirim, Vedat Baday, Erkut Erdem, Aykut Erdem, and Aysegul Dundar. Inst- inpaint: Instructing to remove objects with diffusion models. arXiv preprint arXiv:2304.03246, 2023.
[7] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björ Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684 10695, 2022.
[8] Black Forest Labs. Flux.
2024.
[9] Runpu Wei, Zijin Yin, Shuo Zhang, Lanxiang Zhou, Xueyi Wang, Chao Ban, Tianwei Cao, Hao Sun, Zhongjiang He, Kongming Liang. and Zhanyu Ma. Omnieraser: Remove objects and their effects in images with paired video-frame data. arXiv preprint arXiv:2501.0P9 200..5011..7..... 2025.
[10] Daniel Winter, Matan Cohen, Shlomi Fruchter, Yael Pritch, Alex Rav-Acha, and Yedid Hoshen. Objectdrop: Bootstrapping counterfactuals for photorealistic object removal and insertion. In European Conference on Computer Vision, pages 112-129. Springer, 2024.
[11] Yongsheng Yu, Ziyun Zeng, Haitian Zheng, and Jiebo Luo. Omnipaint: Mastering object- oriented editing via disentangled insertion-removal inpainting. arXiv preprint 200.:7250..08677, 2025.
[12] Marcelo Bertalmio, Guillermo Sapiro, Vincent Casclles, and Coloma Ballester. Image inpainting. Proceedings of the 27th annual conference on Computer graphics and interactive techniques, pages 417-424, 2000.
[13] Alexandru Telea. An image inpainting technique based on the fast marching method. Journal of graphics tools, 9(1):23-34, 2004.
[14] Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin La, and Thomas S Huang. Free-form image inpainting with gated convolution. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4471-4480, 2019.
[15] Zixin Zhu, Xuelu Feng, Dongdong Chen, Jianmin Bao, Le Wang. Yinpeng Chen, Lu Yuan, and Gang Hua. Designing better asymmetric vqgan for stablediffusion. arXiv preprint wrXiv:2306.04632. 2023.
[16] Xiefan Guo, Hongyu Yang, and Di Huang. Image inpainting via conditional texture and structure dual generation. Proceedings of the IEEF/CVF international conference on computer vision, pages 14134-14143, 2021.

---

[17] Jitesh Jain, Yuqian Zhou, Ning Yu, and Humphrey Shi. Keys to better image inpainting: Structure and texture go hand in hand. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 208-217, 2023.
[18] Pourya Shamsolmoali, Masoumeh Zareapoor, and Eric Granger. Transinpaint: Transformer- based image inpainting with context adaptation. In Proceedings of the IEEE/CVF international conference on computer vision, pages 849-858,20 2023.
[19] Wenbo Li. Zhe Lin, Kun Zhou, Lu Qi, Yi Wang, and Jiaya Jia. Mat: Mask-aware transformer for large hole image inpainting, In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10758- 10768, 2022.
[20] Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11461-11771
2022.
[21] Longtao Jiang. Zhendong Wang, Jianmin Bao. Wengang Zhou, Dongdong Chen, Lei Shi, Dong Chen, and Houqiang Li. Smarteraser: Remove anything from images using masked-region guidance. arXiv preprint arXiv:2501 08279, 2025.
[22] Tianyu Sun, Dingchang Hu, Yixiang Dai, and Guijin Wang. Diffusion- depth inpainting for transparent and reflective objects. IEEE Transactions on Circuits and Systems for Video Technology, 2024.
[23] Anh-Quan Cao and Raoul De Charette. Monoscene: Monocular 3d semantic scene completion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3991-4001, 2022.
[24] René Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. Vision transformers for dense prediction. In Proceedings of the IEEE/CVF international conference computer vision, pages 12179-
12188, 2021.
[25] Hanzhe Hu, Zhizhuo Zhou, Varun Jampani, and Shubham Tulsiani. Myd-fusion: Single-view 3d via depth- multi-view generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9698-97977 2024.
[26] Xi Chen, Yutong Feng, Mengting Chen, Yiyang Wang, Shilong Zhang, Yu Liu, Yujun Shen, and Hengshuang Zhao. Zero-shot image editing with reference imitation. Advances in Neural Information Processing Systems, 37:84010-84033 2024.
[27] Shelly Sheynin, Adam Polyak, Uriel Singer, Yuval Kirstain, Amit Zohar, Oron Ashual, Devi Parikh, and Yaniv Taigman. Emu edit: Precise image editing via recognition and generation tasks. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 18799 2024.
[28] Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusson model alignment using direct preference optimization. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 2228-8238, 2024.
[29] Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to-iman image synthesis. arXiv preprint
[30] Xiaoshi Wu, Keqiang Sun, Feng Zhu, Rui Zhao, and Hongsheng Li. Better aligning text-to- image models with human preference. arXiv preprint arXiv:2303. 14420, 1(3), 2023.
[31] Sixian Zhang, Bohan Wang, Junqiang Wu, Yan Li, Tingting Gao, Di Zhang, and Zhongyuan Wang. Learning multi-dimensional human preference for text-to-image generation. In Pro- ceedings of the IEFF/CVF Conference on Computer Vision and Pattrrn Recognition, pages 8018-8027, 2024.
12

---

[32] Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao. Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything v2. Advances Neural Information Processing Systems, 37:21875- 21911, 2024.
[33] Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 12))3, 2022.
[34] Min-Cheol Sagong, Yoon-Jae Yeo, Seung- Won Jung, and Sung-Jea Ko. Rord: A real-world object removal dataset. In BMVC, page 542, 2022.
[35] Xuan Ju, Xian Liu, Xintao Wang, Yuxuan Bian, Ying Shan, and Qiang Xu. Brushnet: A plug-and-play image inpainting model with decomposed dual-branch diffusion. In European Conference on Computer Vision, pages 150-168. Springer. 2024.
[36] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by two time-scale update rule converge a local nash equilibrium. Advances in neural information processing systems, 30, 2017.
[37] Sadeep Jayasumana, Srikumar Ramalingam, Andreas Veit, Daniel Glasner, Ayan Chakrabarti, and Sanjiv Kumar. Rethinking fid: Towards better evaluation metric for image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9307-9315, 2024.
[38] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion- 5b: An open large-scale dataset for training next generation image-text models. Advances in neural information processing systems, 35:25278 25294,229444 2022.
[39] Richard Zhang, Phillip Isola, Alexei Efros, Eli Shechtman, and Oliver Wang. The unrea- sonable effectiveness of deep features as perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586-595, 2018.
[40] Alain Hore and Djemel Ziou. Image quality metries: Psnr vs. ssim. In 2010 20th international conference on pattern recognition, pages
IEEE, 2010.
[41] Chenjie Cao, Qiaole Dong, and Yanwei Fu. Zits++: Image inpainting by improving the incremental transformer on structural priors. IEEE transactions on pattern analysis and machine intelligence, 45(10)::2667-12663 2023.
[42] Roman Suvorov, Elizaveta Logacheva, Anton Mashikhin, Anastasia Remizova, Arsenii Ashukha, Aleksei Silvestrov, Naejin Kong, Harshith Goka, Kiwoong Park, and Vietor Lempitsky. Resolution-robust large mask inpainting with fourier convolutions. In Proceedings of the IEEE/CVF winter conference applications of computer vision, pages 2149 2159, 2022.
[43] Omri Avrahami, Ohad Fried, and Dani Lischinski. Blended latent diffusion. ACM transactions on graphics (TOG), 42(4):1-11, 2023.
[44] Junhao Zhuang, Yanhong Zeng, Wenran Liu, Chun Yuan, and Kai Chen. A task is worth one word: Learning with task prompts for high-quality versatile image inpainting. In European Conference Computer Vision, pages 195-211. Springgr, 2024.
[45] Wenhao Sun, Xue-Mei Dong, Benlei Cui, and Jingqun Tang. Attentive eraser: Unleashing dif- fusion model's object removal potential via self-attention redirection guidance. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 20734-2077 2025.
[46] Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero Simoncelli. Image quality assessment: from visibility structural similarity. IEEE transactions on image processing, 13(4):600- 612, 2004.
[47] Keyan Ding, Kede Ma, Shiqi Wang, and Eero P Simoncelli. Image quality assessment: Unifying structure and texture similarity. IEEE transactions on pattern analysis and machine intelligence, 2465)22567- 2020.
13

---

[48] Stephanie Fu, Netanel Tamir, Shobhita Sundaram, Lucy Chai, Richard Zhang, Tali Dekel, and Phillip Isola. Dreamsim: Learning new dimensions of human visual similarity using synthetic data. arXiv preprint Xiiv:2306009344, 2023.
[49] Pontus Andersson, Jim Nilsson, Peter Shirley, and Tomas Akenine-Möller. Visualizing errors in rendered high dynamic range images. In Eurographics-Shee Papers, pages 25-28. Eurographics- European Association for Computer Graphics, 2021.
[50] Jianyi Wang, Kelvin CK Chan, and Chen Change Loy. Exploring clip for assessing the look and feel of images. In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 2555-2563, 2023.
14

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No]. or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors their evaluation. While "[Yes] is generally preferable to "[No]" it is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset used"). In general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer is often more nuanced, please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification:
The abstract and introduction clearly state the scope and main contributions of our work, including the proposed two-stage geometry- object removal framework and the use of DPO-inspired training. These claims supported by both theoretical motivation and experimental validation (see Section I and 3 for the motivation and contributions, and Section 4 for the empirical results).
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
15

---

Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: While a dedicated "Limitations" section is not included, we acknowledge the main drawback- high computational cost from the two-stage pipeline- whhnh is discussed in our ablation study. Training time and memory usage are higher than one-stage baselines, which may limit deployment in low-resource settings. However, our ablations also demon- strate that this additional cost leads substantial benefits, including improved structural fidelity, better handling of shadows and reflections.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic upproximations only holding locally). ). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full of assumptions and a complete (and correct) proof?
Answer: [NA]
The paper does not include formal theoretical results such as theorems or proofs. While we introduce loss formulations and a DPO-inspired training objective (see Section 3), these are part of the method design and do not constitute formal theoretical contributions.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
16

---

4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We describe all experimental settings in detail, including dataset preparation, model architecture, LoRA tine-tuning configurations, training hyperparameters (c.g.. batch size, learning rate, number of iterations), and evaluation metrics such as FID, LPIPS, and PSNR. We also report ablations and visual results support reproducibility (see Sections 4.1 1.44).
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g.. a large language model), then there should either be way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We plan to publicly release the full code, pretrained models, and data process- ing scripts upon acceptance. Although we do not include the code or data in the submission preserve anonymity, the paper provides sufficient experimental details (Sections 4. 1.14.3 to enable reproduction. We will include full implementation and reproducibilityy instructions in the camera-ready version.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
17

---

Please see the NeurIPS code and data submission guidelines (https://nips.cc for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ ctiips.cc/uublcc// gwwies/ggue///eeieeoeooiiicissee Codeeubmis issiiissoon mnloiiyy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We specify all experimental details in Sections 3 and 4, including dataset splits, model architecture, training epochs, learning rate, batch size, optimizer (Adam), LoRA rank, and loss functions.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: We report single-run quantitative results and do not include error bars or confidence intervals. While this does not affect the main trends, statistical variation is explicitly analyzed in this version.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
18

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes] We report compute details in Section 4.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: http///eeriip..ecc httpp//cbbc....ccccciiieeeeeeeee
Answer: [Yes]
Justification: We have reviewed the NeurlPS Code Ethics and confirm that our research complies with all stated guidelines. The data used publicly available synthetic, and the task does not involve human subjects, privacy concerns, or potentially harmful applications.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Our work advances the controllability and accuracy of object removal systems, which can benefit applications in autonomous driving, AR/VR content editing, and photo restoration.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
19

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our model is task-specific and evaluated on academic benchmarks, We believe it poses no significant risk of misuse that would require safeguards.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All external assets used in this work are properly credited.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the batasets.papersstt paperswi thcode. cww/ datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
20

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We introduce a new benchmark dataset annotated with object masks and their corresponding causal visual artifacts (e.g., shadows and reflections). The dataset will be released with full documentation, including annotation protocols, license terms, and usage guidelines, upon acceptance.
Guidelines:
The answer NA means that the paper does release assets.
Researchers should communicate the details the dataset/code/mmodll as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: This work does not involve crowdsourcing or research with human subjects. All experimental results are computed automatically without human annotations or subjective
evaluation.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation,
or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The work does not involve research with human subjects and does not require IRB equivalent ethical approval.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
21

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Large language models (LLMs) were not used in the development of the core methods or experiments in this paper. Any assistance from LLMs was limited to language refinement and did not affect the scientific content.
Justification:
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non non-standard components.
Please refer to our LLM policy for what should or should be described.
22

---

A Additional detals about stage 2: appearance rendering
As described in Sec. the appearance rendering model & learns to translate imput image containing an object into a realistic RGB output where both the object and its associated causal visual artifacts (e.g., shadows and reflections) are removed. This translation is guided by geometric transformations predicted in Stage 1. Rather than using separate conditioning vectors, we train g as direct image-to-image diffusion model by concatenating the input image and geometry maps into a single tensor. To enable this, we directly use the colorized depth maps zij and To produced in Stage which are already represented 3-channel RGB-like images. This allows us concatenate them directly with the RGB image / along the width dimension, forming a single composite input the diffusion model.
The input to the appearance rendering model G is constructed by concatenating a masked RGB image with its geometric representations before and after editing. Specifically, define two composite inputs for bidirectional training.
For object removal, the input is:
7removal Concat [ ,T 22):
premoval E R x3W)
(11)
and the corresponding target is:
imsert Concat( T6 20).
inseett
R [x(33)
(12)
For the insertion direction, the roles of input and target are reversed, i.c.. (finsent premavall forms the training pair. This bidirectional setup allows the same model to learn both removal and insertion through a unified diffusion process. The geometry maps 2 and zi are placed on the left side of the RGB image, enabling G to directly observe spatial geometric changes. Since both depth maps are colorized 3-channel tensors, they can be processed jointly with the RGB image without additional codality-speciffe encoders. The model trained using a standard denoising score matching loss
Lrender = Ee,e
w()) |
,t) - V peaod log (lggplpeemooe peemoval
1 I
(13)
+ Er,
w(t) | ,t) V Howe log p (Iineen
removal
2
where I, a noisy version of 7 at diffusion timestep 1, and v(( is a predefined weighting function. By jointly training on both directions, G Iearns to perform appearance synthesis conditioned on structured geometry edits, ensuring that generated results align with both scene content and layout.
B
Additional qualitative comparison
To further validate the effectiveness of our geometry-aware framework, we present additional qual- itative results the CausRem dataset, which contains real-world scenes involving shadows and reflections caused by removed objects.
As shown in Fig. 7 our method successfully removes both the object and its associated shadow, while other methods either retain shadow residues or introduce undesired distortions in unmasked regions. This highlights method's ability preserve unmasked content while achieving consistent object removal.
In Fig. 8 we provide additional comparisons in scenes with reflective surfaces. While baseline methods often fail to fully remove reflections or generate artifacts, our model leverages geometry- guided rendering to produce coherent appearances without explicit reflection modeling.
These results support our key insight: by removing the object structure in the geometric domain and rendering appearances based on updated geometry, our framework can implicitly climinate causal visual artifacts and maintain visual consistency in challenging real-world scenarios.
C Additional perceptual metrics
In Tab. 66, we additionally report SSIM 46), DISTS 47 DreamSim
FLIP 49 and CLIIP.
IQA |50 on RemovalBench and RORD-Va Val.
23

---

Masked Image
Attentive-Eraser
CLIPAway
OmniEraser
FLUX.1-Fill
Ours
Figure 7: Qualitative comparison on CausRem highlighting shadow removal performance.
24

---

Masked Image
Attentive-Eraser
CLiyAway
OmniEraser
FLUX.1-Fil
Ours
Figure Qualitative comparison on CausRem highlighting reflection removal performance.
D
Additional failure and challenging cases
Stage 2: transparent/reflective and self-emitting scenes. The third row in Fig. 9a shows semi- transparent objects on a reflective table. After removing one cup, the direct reflection is removed as well, while residual color bleeding remains in nearby reflections influenced by the removed cup. As shown
second row Fig. Da for self-emitting objects (e.g., colored bulbs), Stage 2 may hallucinate a diffuse glow rather than cleanly removing the illumination. This behavior is undesirable for removal, yet indicates that the renderer has learned meaningful correlations between light and its source, which could be useful if properly constrained.
25

---

Table Perceptual metrics on RemovalBench and RORD-Val. higher is better, + lower better.
RemovalBench
RORD-Val
Method
SSIMT DISTS,
DreamSim, FLIP) CLIP-IQAT
SSIMF DISTS)
DreamSim. FLIP) CLIP-IQAT
CLIPAway
5
0.6298
0.1656
0.1572
0.1175
0.4973
0.6074
0.1580
0.1304
0.1645
0.7986
Dicentiv-EEase
45
0.7084
161
0.0536
0.0854
0.4790
0.7186
0.1243
0.0878
0.1174
0.7270
OmniFraser
9
0.6367
0.1277
0.0539
0.1084
0.4339
0.6071
0.1325
0.0675
0.1524
0.6646
Ours
0.7367
0.0770
0.0304
0.0863
0.4146
0.8248
0.0798
0.0459
0.1026
0.7807
Teput Trage
Depes Mep
Predicted Depth Mep
Sernoved Resui
Depth Map
Predicted Depth Map
Inconalatts
Complitee
(b) Stage I failure under incomplete mask; complete versus partial masks yield success versus hallucinated completion.
Soler Reoding
(a) Challenging scenes for Stage 2: transparent semi- transparent objects and self-emitting (light-source) cases.
Figure 9: Challenging cases and failure analysis. (a) Residual color bleeding and hallucinated glow can appear in reflective or lighting scenes. (b) Incomplete masks confuse Stage I; simple dilation or stronger segmentation mitigates this.
Input Image
Depth Map
Predicted Depth Map
Removed Result
Figure 10: Watermark removal with peeudo-deptl cue. Applying Local Max Depth Fill-in within the watermark mask creates sufficient geometric contrast for Stage to detect and remove the watermark across both the lake and dock.
Stage 1: incomplete masks. Figure contrasts complete and partial masks for a semi-transparent bottle. With complete mask, Stage removes the geometry expected. With partial mask, the model attempts to complete the object, producing a hallucinated extra bottle. practice this can be avoided by simple mask dilation by using stronger segmentation models (e.g., SAM2) provide complete masks.
Watermark removal. Beyond geometry-related artifacts, our framework can handle scene-wide watermarks with light modification (Fig. [0]. In this example, the watermark spans both the lake surface and wooden planks but lacks a reliable depth estimate. We apply the same Local Max Depth Fill-in inside the watermark mask assigning each masked pixel the maximum depth from a pseudo-depth cue. Because Stage selects removable regions depth maps, this injected cue provides sufficient
a small local neighborhood-
by differencing the input and geometry- contrast for Stage to identify and remove the watermark, showing that minimal conditioning tweaks let our method generalize to non-geometric inpainting cases.
26

---

E
CausRem
We construct the CausRem dataset by collecting 200 high-quality images from the free stock platform Pexels including 100 images containing reflections and 100 with shadows. For each image, we manually annotate the primary object along with its causal visual effects reflections or shadows.
In the shadow subset, where multiple objects often co-occur, we randomly select two to three objects per image for annotation. Each object and its corresponding shadow mask stored using distinct IDs to preserve one-to-one causal relationship. In the reflection subset, due to the presence of fewer objects, annotate only single object-mask pair per image.
Fig. and Fig.
illustrate representative annotation examples from the dataset.
Image
Object
Object 2
Masked Image
Image
Object
Masked Image
Figure 11: Representative annotations in CausRem. Left: shadow examples; Right: reflection examples.
Image
Object
Object
Masked Image
Image
Object
Masked Image
Figure 12: Representative annotations in CausRem. Left: shadow examples; Right: reflection examples.
https:/wwwwppxels.com
27


Reviews:
[{"review_id": "vDAstzVDzs", "reviewer": "Reviewer_Dwf1", "summary": "The paper introduces GeoRemover, a two-stage framework for object removal that explicitly addresses the challenge of eliminating both target objects and their causal visual artifacts e.g., shadows, reflections. The first stage removes objects from geometric representations, depth maps, using strictly mask-aligned training, ensuring structural accuracy. The second stage renders a photorealistic RGB image conditioned on the updated geometry, depth map, implicitly removing artifacts. A preference-driven objective, inspired by DPO, is introduced to prevent hallucinations during geometry removal. Experiments on benchmark datasets shows state-of-the-art performance in both object and artifact removal.\n\nThe paper presents a significant advancement in object removal by addressing causal artifacts through a principled, two-stage approach. While computational costs and paired data requirements are limitations, these are common trade-offs in state-of-the-art methods and do not detract from the paper’s contributions.", "questions": "1. How does the two-stage framework handle dynamic or translucent objects where depth maps are ambiguous?\n2. What are the trade-offs between the strict mask alignment in Stage 1 and the flexibility of loose alignment in prior work?", "limitations": "1. High Resource Usage: The two-stage design increases computational overhead, compared with one-stage methods.\n2. Depth Estimator Dependency: Performance is tied to the quality of the depth map, which may introduce errors in low-texture or occluded regions.\n3. Artifact Scope: The method focuses on shadows and reflections; how about other artifacts, such as motion blur", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strengths:\n1. Novel Two-Stage Framework: Decoupling geometry removal and appearance rendering enables precise structural editing and implicit artifact removal, addressing a critical challenge occurred in previous work.\n2. Preference-Driven Training: The DPO-inspired loss effectively suppresses hallucinations and ensures geometric consistency, improving controllability compared to loosely mask-aligned methods.\n3. SOTA performance: GeoRemover outperforms existing methods on multiple metrics (FID, PSNR, IoU) across two benchmarks, including a new CausRem dataset for causal artifacts.\n\nWeaknesses:\n1. Computational Cost: The two-stage pipeline increases runtime and memory usage compared to one-stage methods.\n2. Reliance on Depth Estimation: The method depends on external depth estimators, whose accuracy could degrade in complex scenes.\n3. Paired Data Requirement: The rendering stage requires paired data (images with and without objects/artifacts), which may not always be available in real-world scenarios.", "was_revised": true, "final_justification": "I appreciate the authors' detailed thoughtful response. All my concerns got addressed fairly. I keep the final score \"Accept\".", "rebuttal": "### Computational cost & High resource usage.\n\nWe appreciate the reviewer’s comment on the computational cost. In terms of runtime, our current implementation takes approximately 1 s for depth estimation, 7 s for geometry removal (Stage 1), and 20 s for appearance rendering (Stage 2) per image (512 * 512) on a single H100 GPU. We would like to emphasize that our implementation has not yet been optimized for latency. In practical deployment scenarios, various acceleration strategies—such as model distillation and architectural pruning—can be applied to significantly reduce inference time. Therefore, although the current runtime introduces some overhead, it does not pose a significant obstacle to practical applications.\nRegarding memory usage, the overall peak memory consumption is 34GB, which poses no practical limitation for deployment on current GPU hardware. \n\n### Reliance on depth estimation & Depth estimator dependency.\n\nWe appreciate the reviewer’s insightful comment regarding the reliance on depth estimation. Our method indeed assumes accurate geometry as a prerequisite, and performance may degrade if the depth estimation fails in complex scenes. Nevertheless, recent progress in this field has significantly improved robustness in such scenarios. For example, Choi et al (\"Self-supervised Monocular Depth Estimation Robust to Reflective Surface Leveraged by Triplet Mining.\" ICLR. 2025.) explicitly address reflective and complex surfaces. With such rapid advances, obtaining reliable geometry is becoming increasingly feasible, which we believe will further strengthen the applicability of our method.\n\n### Paired data requirement.\n\nWe appreciate the reviewer’s valuable comment on the paired data requirement. While collecting perfectly paired data can be challenging in some real-world scenarios, recent works have shown that such data can be efficiently extracted from videos (e.g., RORD and OmniEraser), and most state-of-the-art object removal methods also rely on paired supervision. Moreover, with the rapid progress in text-to-video generation, creating paired data is becoming increasingly practical. For example, a simple prompt such as “A red apple sits on a table; a hand enters the frame and takes the apple away” can generate frames with and without the object, naturally forming paired training samples.\n\n### How does the two-stage framework handle dynamic or translucent objects where depth maps are ambiguous?\n**_Note: Dear Reviewer, we sincerely thank you for your time and effort in reviewing our work. Due to the official policy for this year’s rebuttal process, we are, unfortunately, unable to share the carefully prepared one-page document of additional visualizations. These include failure cases, intermediate results, and various edge case examples, which we found both interesting and thrilling to analyze, as they were inspired by the reviewers’ constructive comments. We also reached out to the AC to ask whether we could share this document in the same way as a code link, and the AC kindly consulted the program chairs on our behalf; however, we were informed that this is not allowed. Despite these constraints, we are making every effort to describe these visualizations in detail so that you can still obtain the corresponding information. These described figures are referred to using lettered indices (Case A–H). We will update our main submission with content from this document._**\n\nWe appreciate the reviewer’s insightful question. Our Stage 2 model determines which regions to remove by comparing the difference between the original and the geometry-removed depth maps. For dynamic or translucent objects, depth estimation can be incomplete, resulting in little or no difference between the two depth maps and thus limiting Stage 2’s effectiveness. \n\nIn Case A, three individuals are running, and motion blur caused by high-speed movement leads to erroneous depth estimation by Depth Anything. Specifically, the depth of one runner’s foot is completely misestimated as part of the background. Consequently, during Stage 1 geometry removal, this foot region lacks valid depth information, and the Stage 1 output becomes almost identical to the input depth map. \n\nSince Stage 2 relies on comparing the input and output depth maps from Stage 1 to identify the regions requiring removal, no removal occurs when both maps are nearly identical. In this case, the absence of reliable geometric guidance prevents Stage 2 from performing effective object removal.\n\nFortunately, this issue can be easily addressed by filling the missing depth values within the masked region using the maximum depth value from its local neighborhood (e.g., a 10×10 pixel window). Surprisingly, simply propagating the maximum depth value from nearby pixels—“borrowing depth from neighbors”—almost magically restores the missing foot geometry, enabling Stage 2 to confidently erase the leg. In addition, missing depth values can be detected by checking whether the depth within the masked region remains almost unchanged before and after geometry removal.\n\nAfter applying this Local Max Depth Fill-in strategy, the previously missing foot depth is filled with the maximum depth values from its local neighborhood, producing a pseudo-complete depth map and significantly enhancing the depth contrast between the masked region and its surroundings. Stage 2 can thus accurately identify the foot region as an object to be removed, resulting in the correct elimination of the runner’s leg and a visually consistent reconstruction of the ground surface. This demonstrates that even a simple geometric completion strategy can substantially improve the robustness of our two-stage framework under challenging dynamic conditions.\n\nWe also provide transparent and semi-transparent examples in Case B and Case C. The Case B shows a fully transparent glass sphere placed on grass. The distorted, magnified grass patch is particularly interesting, as it appears as though the grass “jumps forward” in depth, confusing the estimator. However, as long as the depth of the grass exists, our model can still remove the magnified grass behind the glass sphere. The Case C shows a large transparent water jug and five semi-transparent cups on a reflective table. What is fascinating here is that despite the complex light interactions on the reflective table, the model precisely identifies and removes the cups with its reflection. These results demonstrate that our model is reasonably robust to both transparent and semi-transparent objects. We will include these findings and discussions in the final version of the paper.\n\n### What are the trade-offs between the strict mask alignment in Stage 1 and the flexibility of loose alignment in prior work?\n\nThere is an inherent trade-off between strict and loose mask alignment. During training, strict mask alignment provides clear geometric guidance, explicitly indicating which regions should be modified and which should remain unchanged. However, its limitation becomes evident during inference: if the user provides an imperfect mask—e.g., failing to fully cover a cup—the uncovered parts will not be removed. In contrast, loose mask training improves robustness to low-quality masks by allowing the model to modify regions outside the given mask, reducing its reliance on precise mask inputs. The downside, as illustrated in Figure 1 of the paper, is that it often alters unintended regions, causing uncontrolled geometry modifications.\n\nWe therefore do not adopt loose mask training, as our strict alignment framework combined with mask-level augmentation can already reduce the dependence on high-quality masks during inference while avoiding unintended geometry modifications. Instead, following SmartEraser, we introduce mask-level data augmentation under the strict alignment framework, including mask dilation, boundary perturbation, and slight morphological expansion. This augmentation exposes the model to imperfect masks during training, enabling it to tolerate minor mask inaccuracies while retaining the structural precision guaranteed by strict mask alignment.  In practice, we observe that as long as the mask covers approximately 95% of the target object, our method can still achieve reliable geometry removal, demonstrating robustness to small inaccuracies in mask coverage.\n\n### Artifact scope.\n\nWe appreciate the reviewer’s question. Motion blur is indeed a causal artifact induced by object motion and thus, in principle, fits within the scope of our geometry-driven framework. However, our current training lacks paired data that explicitly provides motion blur supervision, which limits its performance in such scenarios. We observed in Case A, after the runner’s leg is removed, the model does not attempt to erase the motion blur associated with the leg. Instead, it interprets the occluded region as if it originally contained motion blur and “faithfully completes” that blurred streak, treating it as part of the static background rather than as an artifact to be removed. This limitation is common to all existing object removal methods, as paired supervision for motion blur removal is rarely available. As more paired datasets or video-based supervision become available, our two-stage design should naturally generalize to handle motion blur due to its reliance on geometric priors rather than artifact-specific heuristics. We are very interested in exploring this further as our next step."}, {"review_id": "tbqRpUV0wt", "reviewer": "Reviewer_w1kx", "summary": "The paper presents a two-step diffusion pipeline for object removal that also erases the object’s shadows using depth as a geometry signal. In the 1st stage, a depth-aware diffusion model takes the masked image and its depth map, then learns to remove the object in the depth domain with DPO loss without artefacts. Next, a second diffusion model receives three inputs which are the original RGB image, the original depth map, and the object removed depth map. With these inputs, the second model produces the final image in which both the object and any shadow or reflections are removed.", "questions": "- How robust is both stages of the pipeline with respect to the possible incorrect depth maps? For example, in a failure case of depth anything in stage 1, how does the geometry removal of the object behaviour is effected for the model. Also, if the geometry removal of stage 1 is not perfect, how robust is the second stage pipeline to such changes? I think adding some slight noise to the original RGB images or increasing resolution which is explained as a failure case for the model [2] (In limitations section), and then executing the pipeline and analyzing how the scores have been changed can provide a better explanation in terms of the inner workings of the model.\n\n- REMOVE [1] which is a metric that directly measures the object removal quality can also be used for analysis to better state the success of this paper compared to other baselines. \n\n- I would like to see example failure cases for both stages of the pipeline. Currently the paper only includes success cases.\n\n- What is the λ value used for the DPO coefficient in the stage 1 loss?    \n\n- What is the inference clock time overhead of this method compared to other baselines? This is stated as a limitation so can be better provided with numbers.\n\n---\n\n**I would like to see more failure case examples and robustness of the model as well as DPO's effectiveness compared to a more simple alternative. Given satisfactory answers, I would increase my score.**\n\n---\n[1] Chandrasekar, Aditya, et al. \"Remove: A reference-free metric for object erasure.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024. [code](https://github.com/chandrasekaraditya/ReMOVE)\n[2] Yang, Lihe, et al. \"Depth anything: Unleashing the power of large-scale unlabeled data.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.", "limitations": "Limitations need more explanation. The authors state the high computational cost of 2 stage pipeline as a limitation (lines 509-514) which is correct but I think the failure cases as a limitation also needs to be discussed for both stages.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 4, "originality": 3, "strengths_and_weaknesses": "**Strengths** \n- The paper is well written and easy to follow.\n- The paper tries to solve a valid problem being present in the current literature. Significance is high.\n- The paper made extensive comparisons with current literature including both GAN based and Diffusion based models as comparison.\n- Using Depth as a supervision for removing shadows in supervision as well as two-stage decoupling with the DPO-guided depth loss is a novel approach to tackle the problem.\n\n**Weaknesses** \n- **Limitations need more explanation:**  The authors state the high computational cost of 2 stage pipeline as a limitation (lines 509-514) which is correct but I think the failure cases as a limitation also needs to be discussed for both stages.\n- **Use of DPO despite a qualitative example in figure 3a needs a better explanation in terms of motivation:**\nFigure 3a does illustrate that training **with** DPO yields cleaner geometry than training **without** it, yet this single visual comparison does not fully justify adopting a RL-style objective in place of simpler depth-smoothness regularizer such as total variation loss. Conceptual explanation of why DPO is more suitable for this task needs more expanding. \n- **Computational overhead:** Due two 2 stage inference, this model is more computationally expensive compared to other baselines.", "was_revised": true, "final_justification": "I think this paper solves an open problem in a novel way. In rebuttal, I asked for failure case examples and robustness of the model as well as DPO's effectiveness compared to a more simple alternative. Given satisfactory answers (comparing dpo with tv loss and explaining some possible failure cases and robustness of the model), I had increase my score to an accept.", "rebuttal": "### Limitations need more explanation & Failure cases for each stage\n**_Note: Dear Reviewer, we sincerely thank you for your time and effort in reviewing our work. Due to the official policy for this year’s rebuttal process, we are, unfortunately, unable to share the carefully prepared one-page document of additional visualizations. These include failure cases, intermediate results, and various edge case examples, which we found both interesting and thrilling to analyze, as they were inspired by the reviewers’ constructive comments. We also reached out to the AC to ask whether we could share this document in the same way as a code link, and the AC kindly consulted the program chairs on our behalf; however, we were informed that this is not allowed. Despite these constraints, we are making every effort to describe these visualizations in detail so that you can still obtain the corresponding information. These described figures are referred to using lettered indices (Case A–H). We will update our main submission with content from this document._**\n\nWe appreciate the reviewer’s suggestion to elaborate on the failure cases of both stages. For Stage 2, we provide two representative failure cases in Case C and Case D.\n\nThe Case C shows a semi-transparent case with a large water jug and five semi-transparent cups on a reflective table. The reflections of these cups overlap, causing noticeable color bleeding. When removing one of the cups, the model successfully removes both the cup itself and its direct reflection. However, the color bleeding in the reflections of the other cups, which was influenced by the removed cup, remains visible. This reveals a limitation of our current approach in handling reflective surfaces with complex, overlapping color interactions.\n\nThe Case D presents a challenging self-emitting object removal case. Two light bulbs lie on the floor, one casting red light and the other green. Stage1 handles the geometry perfectly—both bulbs and their shapes are accurately removed in the depth map. However, Stage2 behaves unexpectedly: instead of simply erasing the red illumination, it “imagines” a diffuse, amorphous red glow appearing in midair, as if a new red light source had been added to the scene. Although the removal result is unsatisfactory, this behavior reveals an intriguing aspect of Stage 2’s reasoning. It has implicitly learned a causal relationship between illumination and its source—“believing” that if there is red light in the scene, there must be a corresponding red light source to explain it. This over-reasoning causes hallucinated artifacts in this case, yet it also suggests that our rendering model is not merely filling textures arbitrarily; instead, it captures some meaningful light–source correlations, which, if guided properly, could be advantageous in more physically consistent rendering tasks.\n\nTo the best of our knowledge, no existing inpainting or object removal method can effectively handle such cases either, mainly due to the lack of paired data to provide explicit guidance. To address these limitations, we plan to leverage text-to-video generation models to synthesize paired data for such challenging edge cases, which we expect will improve the model’s robustness in future work.\n\nAs for failure cases where Stage 1 struggles to remove the target geometry, they mainly occur when the user-provided mask is incomplete. In Case F, three semi-transparent glass bottles sit snugly inside a cardboard box. We attempt to remove the largest bottle on the left under two conditions: one with a complete mask fully covering the bottle, and another with an incomplete mask covering only about 70% of it. The difference is striking—when given the complete mask, Stage 1 neatly erases the bottle as expected. But with the partial mask, the model gets “confused” and hallucinates an extra bottle, as if trying to “fill in what it thinks should still be there.” This behavior, while undesirable for removal, actually reveals that the model has learned a non-trivial understanding of geometric structures: rather than blindly removing masked regions, it attempts to infer plausible object shapes based on surrounding geometry. Such behavior can be easily avoided in practice by applying simple mask dilation operations or using advanced segmentation models like SAM2 to provide complete masks.\n\n### DPO needs better motivation\n\n| Method | FID ↓ | CMMD ↓ | LPIPS ↓ | PSNR ↑ | AS ↑ | Insert. ↓ |\n|--------|--------|----------|----------|---------|------|------------|\n| TV     | 33.42  | 0.205    | 0.118    | 23.21   | 4.61 | 3.87%      |\n| DPO    | **31.15**  | **0.182**    | **0.103**    | **23.70**   | **4.69** | **1.48%**      |\n\nWe appreciate the reviewer’s insightful question regarding the motivation for adopting a DPO-inspired objective. Total variation (TV) loss encourages local smoothness, which is effective for reducing depth noise but does not explicitly distinguish between correct geometry completion and implausible insertions. In contrast, DPO optimizes pairwise preferences between correct and hallucinated samples, directly aligning the model’s output with the desired global geometry. This is particularly important in ambiguous regions, where local smoothness may over-smooth fine structures or even propagate incorrect geometry, while DPO explicitly suppresses hallucinated insertions by learning from ranked preferences. Quantitatively, replacing DPO with a TV loss results in degraded performance, as shown in the above Table, confirming its effectiveness in maintaining geometric consistency.\n\n### Computational Overhead & Inference clock time\n\n| **Method**        | **Time (s) ↓** \n|-------------------|----------------|\n| FLUX.1-Fill       | 7              |\n| CLIPAway          | 12             | \n| Attentive-Eraser  | 7              | \n| OmniEraser        | **6**          | \n| **Ours**          | 28             | \n\nWe appreciate the reviewer’s comment on the computational cost. To further clarify this point, we provide a comparison of the inference clock time in Table above, which explicitly shows the runtime overhead of our two-stage pipeline relative to one-stage methods. In terms of runtime for editing a 512 × 512 image, our current implementation takes approximately 1 s for depth estimation, 7 s for geometry removal (Stage 1), and 20 s for appearance rendering (Stage 2) per image on a single H100 GPU. In practical deployment scenarios, various acceleration strategies—such as model distillation and architectural pruning—can be applied to significantly reduce inference time. Therefore, although the current runtime introduces some overhead, it does not pose a significant obstacle to practical applications. Moreover, although this cost is indeed higher than that of one-stage methods, Figure4 and Table2 in the paper provide experimental evidence supporting the necessity of the two-stage design, as it significantly improves both object removal quality and artifact suppression.\n\n### How robust is both stages of the pipeline with respect to the possible incorrect depth maps?\n**_Note: due to space limitations, we may refer to responses addressed to other reviewers; we hope for your kind understanding._**\n\nWe appreciate that the reviewer precisely identified the issue and suggested improvement direction. Our Stage2 relies on comparing the input and geometry-removed depth maps from Stage1 to locate the regions to be removed; thus, severe depth estimation errors in Stage1 can indeed limit the overall performance. Moreover, we adopt an inference-time strategy similar to adding depth noise to mitigate this issue. A detailed example (Case A) is provided in our response to Reviewer Dwf1 Question **_How does the two-stage framework handle dynamic or translucent objects where depth maps are ambiguous?_**.  \n\n### Increasing resolution \n\n| Resolution | FID ↓ | CMMD ↓ | LPIPS ↓ | PSNR ↑ | AS ↑ | Insert. ↓ |\n|------------|--------|----------|----------|---------|------|------------|\n| 512×512    | 32.92  | 0.191    | 0.116    | 22.85   | 4.12 | 2.37%      |\n| 1024×1024  | **31.15**  | **0.182**    | **0.103**    | **23.70**   | **4.69** | **1.48%**      |\nWe thank the reviewer for this constructive suggestion. We have indeed trained and evaluated our model at a lower resolution of 512×512 in earlier experiments, and the results were clearly inferior to the current 1024×1024 version. It confirms that higher-resolution training substantially improves the model’s robustness, as the reviewer suggested.\n\n### REMOVE metric\n\n| Method           | RemovalBench | RORD-Val |\n|------------------|---------------|------------|\n| **Ground Truth**     | **0.931**         | **0.8541**     |\n| CLIPAway         | 0.8489        | 0.8059     |\n| Attentive-Eraser | 0.9201        | 0.9164     |\n| OmniEraser       |       0.9236        | 0.9163     |\n| **Ours**         | **0.9302**    | **0.8578** |\nWe appreciate the reviewer’s constructive suggestion. We conducted additional experiments using the REMOVE metric, and the results are reported in the above Table. Although the absolute REMOVE scores seem not perfectly calibrated (on RORD-Val, the Ground Truth only achieves 0.8541, which suggests that the metric may not be strictly upper-bounded by real images), our method achieves the score (0.8578) that is closest to the Ground Truth compared to all other baselines. This indicates that our two-stage framework produces object removal results that are perceptually more consistent with the real images.\n\n### DPO lambda value\n\nWe set λ = 0.1 in our final implementation. We also experimented with higher values (e.g., λ = 0.5 and λ = 1), but both led the model to converge to an “easy” solution, where the masked regions were overly smoothed and simply filled with uniform depth values, rather than preserving geometric details. The lower value of 0.1 provides a better balance between enforcing geometric consistency and maintaining fine structural details."}, {"review_id": "JZhThEntdB", "reviewer": "Reviewer_wwGK", "summary": "This paper addresses the problem of object removal in 2D images. To better remove objects along with their shadows and mirrored reflections, the authors propose using depth as an intermediate representation in image inpainting. Specifically, they first inpaint the depth map and then reconstruct the image conditioned on the inpainted depth. This allows artifacts caused by reflections and occlusions to be removed at the depth level. \nSpecifically, the authors introduce a two-stage framework: The first stage performs depth inpainting. The second stage generates the final image conditioned on the inpainted depth. Both stages are trained separately based on FLUX and LoRA.", "questions": "I have two main concerns regarding this paper:\n1. Handling of reflective objects: As shown in Figure 5, the method performs well on several mirror-like reflection cases. However, the paper does not include intermediate visualizations, which makes it somewhat difficult to understand which stage contributes most to the removal of reflection artifacts - whether it is the initial depth estimation, the depth inpainting stage, or the final image generation. Providing intermediate results from each stage could help me better understand the roles and effectiveness of each component in handling these challenging scenarios. \n2. Lack of failure case analysis: While Tables 1 and 4 show that the proposed method outperforms all state-of-the-art baselines in terms of quantitative metrics, the paper would benefit from a more in-depth discussion of failure cases. In particular, it would be informative to include examples where the full model does not perform well, rather than only showing ablation results like those in Figure 4. \nUnderstanding where and why the model fails would provide valuable insight into its limitations and potential areas for future improvement.", "limitations": "The method relies on the Depth Anything model for depth estimation, which may limit performance in certain challenging scenarios, especially where depth prediction is inherently ambiguous.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths: \n1. The idea of decoupling object removal into a two-stage process is novel and interesting. By leveraging two strong base models—Depth Anything and FLUX, the proposed approach achieves superior image generation quality, outperforming all state-of-the-art methods on standard benchmarks. \n2. The paper is logically well-structured, and the training procedure for both stages is clearly explained, making it easy to follow.\n3. The authors introduce a real-world test dataset, CausRem, which includes 200 challenging cases such as reflections and shadows. \n\nWeaknesses: \nThe method relies heavily on the depth estimation quality of the Depth Anything model.", "was_revised": true, "final_justification": "Thanks for the author's detailed response. Although the step-by-step visual results are not presented, I believe the author's analysis of the failure cases largely addresses my concerns. I will raise my score to borderline accept, and I hope the authors will include visualizations and further analysis in future versions.", "rebuttal": "### The method relies heavily on the depth estimation quality of the Depth Anything model.\nWe thank the reviewer for pointing out this important aspect. In our implementation, we use an off-the-shelf depth estimator solely to obtain a geometric representation.  Strictly speaking, our method does not rely on any specific depth estimator; rather, it assumes that an accurate geometric representation is available. Our core hypothesis and contribution lie in showing that, given a reasonably accurate geometry, object removal and its causal artifacts (e.g., shadows and reflections) can be effectively eliminated through our two-stage pipeline. Depth Anything is simply a strong depth estimator that helps us obtain more accurate geometry in practice, but our approach is not inherently tied to it. Moreover, the availability of many high-quality off-the-shelf depth estimators today makes it easy to obtain such geometry in practice.\n\nTo further verify this, we replaced Depth Anything with a different depth estimator VGG-T (Wang, Jianyuan, et al. \"Vggt: Visual geometry grounded transformer.\" Proceedings of the Computer Vision and Pattern Recognition Conference. 2025.).  As shown in the below Table, although the performance with VGG-T slightly drops compared to Depth Anything, our pipeline still achieves consistent artifact removal, confirming that our method can generalize to other depth estimators.\n\n| Depth Estimator | FID ↓ | CMMD ↓ | LPIPS ↓ | PSNR ↑ | AS ↑ | Insert. ↓ |\n|------------------|--------|---------|-----------|----------|--------|-------------|\n| VGG-T            | 31.27  | 0.185   | 0.107     | 23.68    | 4.35   | 1.71%       |\n| Depth Anything   | **31.15**  | **0.182**   | **0.103**     | **23.70**    | **4.69**   | **1.48%**       |\n\n\n### Providing intermediate results from each stage for handling of reflective objects. \n\nWe thank the reviewer for this valuable comment and apologize for not making this clearer in the paper. The intermediate reflections can actually be observed in Figure3(a) (rightmost boat example) in the paper. For instance, the boat’s reflection is estimated as having the same depth as the lake surface rather than a new boat-like geometry. Therefore, reflection artifacts are not present in the depth domain. After Stage1 removes the boat geometry, Stage2 leverages this updated geometry to simultaneously remove the boat and its corresponding reflection artifacts. In summary, the initial depth estimation and depth inpainting stage provide guidance on which objects need to be removed, while the final image generation stage removes the objects and their corresponding reflection artifacts.\n\n**_Note: Dear Reviewer, we sincerely thank you for your time and effort in reviewing our work. Due to the official policy for this year’s rebuttal process, we are, unfortunately, unable to share the carefully prepared one-page document of additional visualizations. These include failure cases, intermediate results, and various edge case examples, which we found both interesting and thrilling to analyze, as they were inspired by the reviewers’ constructive comments. We also reached out to the AC to ask whether we could share this document in the same way as a code link, and the AC kindly consulted the program chairs on our behalf; however, we were informed that this is not allowed. Despite these constraints, we are making every effort to describe these visualizations in detail so that you can still obtain the corresponding information. These described figures are referred to using lettered indices (Case A–H). We will update our main submission with content from this document._**\n\nWe further demonstrate the robustness of our pipeline to depth estimation errors in Case G. The reflection of a duck on the lake surface is incorrectly estimated by the depth estimator as having a duck-like shape rather than the lake’s flat depth. Nevertheless, the final removal is still correct because the actual object depth (the duck itself) is accurate, enabling Stage2 to identify it as removable and also remove its reflection. This robustness arises because Stage 2 is trained on depth maps produced by Depth Anything, which naturally contain some depth estimation errors. According to our statistics, in approximately 85% of cases where reflection depths are estimated incorrectly, the final removal remains correct.\n\n### Lack of failure case analysis. \n\nWe appreciate the reviewer’s suggestion to elaborate on the failure cases. For Stage 2, we provide two representative failure cases in Case C and Case D.\n\nThe Case C shows a semi-transparent case with a large water jug and five semi-transparent cups on a reflective table. The reflections of these cups overlap, causing noticeable color bleeding. When removing one of the cups, the model successfully removes both the cup itself and its direct reflection. However, the color bleeding in the reflections of the other cups, which was influenced by the removed cup, remains visible. This reveals a limitation of our current approach in handling reflective surfaces with complex, overlapping color interactions.\n\nThe Case D presents a challenging self-emitting object removal case. Two light bulbs lie on the floor, one casting red light and the other green. Stage1 handles the geometry perfectly—both bulbs and their shapes are accurately removed in the depth map. However, Stage2 behaves unexpectedly: instead of simply erasing the red illumination, it “imagines” a diffuse, amorphous red glow appearing in midair, as if a new red light source had been added to the scene. Although the removal result is unsatisfactory, this behavior reveals an intriguing aspect of Stage 2’s reasoning. It has implicitly learned a causal relationship between illumination and its source—“believing” that if there is red light in the scene, there must be a corresponding red light source to explain it. This over-reasoning causes hallucinated artifacts in this case, yet it also suggests that our rendering model is not merely filling textures arbitrarily; instead, it captures some meaningful light–source correlations, which, if guided properly, could be advantageous in more physically consistent rendering tasks.\n\nTo the best of our knowledge, no existing inpainting or object removal method can effectively handle such cases either, mainly due to the lack of paired data to provide explicit guidance. To address these limitations, we plan to leverage text-to-video generation models to synthesize paired data for such challenging edge cases, which we expect will improve the model’s robustness in future work.\n\nAs for failure cases where Stage 1 struggles to remove the target geometry, they mainly occur when the user-provided mask is incomplete. In Case F, three semi-transparent glass bottles sit snugly inside a cardboard box. We attempt to remove the largest bottle on the left under two conditions: one with a complete mask fully covering the bottle, and another with an incomplete mask covering only about 70% of it. The difference is striking—when given the complete mask, Stage 1 neatly erases the bottle as expected. But with the partial mask, the model gets “confused” and hallucinates an extra bottle, as if trying to “fill in what it thinks should still be there.” This behavior, while undesirable for removal, actually reveals that the model has learned a non-trivial understanding of geometric structures: rather than blindly removing masked regions, it attempts to infer plausible object shapes based on surrounding geometry. Such behavior can be easily avoided in practice by applying simple mask dilation operations or using advanced segmentation models like SAM2 to provide complete masks.\n\n### Limitations: The method relies on the Depth Anything model for depth estimation, which may limit performance in certain challenging scenarios, especially where depth prediction is inherently ambiguous.\nWe thank the reviewer for pointing out this important aspect. As discussed above, our model is not restricted to any single depth estimator. Furthermore, recent progress in depth estimation has significantly improved robustness in challenging scenarios. For example, Choi et al. (\"Self-supervised Monocular Depth Estimation Robust to Reflective Surface Leveraged by Triplet Mining,\" ICLR 2025) explicitly address reflective and complex surfaces. With such rapid advances, obtaining reliable geometry is becoming increasingly feasible, which we believe will further enhance the applicability and robustness of our proposed method.\n\n**_Note: due to space limitations, we may refer to responses addressed to other reviewers; we hope for your kind understanding._**\n\nAdditionally, when depth estimation is imperfect, our pipeline includes mechanisms to compensate for such inaccuracies. A detailed example (Case A) is provided in our response to Reviewer Dwf1 Question **_How does the two-stage framework handle dynamic or translucent objects where depth maps are ambiguous?_**."}, {"review_id": "RKMDoVoYj1", "reviewer": "Reviewer_KFtu", "summary": "This paper introduces a diffusion-based model for the removal of objects in images, in a way that also removes the artifacts that these objects create on the rest of the image, including reflections and shadows. The method relies upon a 2-step process: first, depth map object removal and then a cyclical diffusion model that performs both inpainting and removal simultaneously. The method is evaluated across several metrics, ablation studies, and comparisons with previous work, showing a significant advantage compared to baselines.", "questions": "- How does this model handle transparency and translucency?\n- How does the model depend on the diffusion backbone used for the image generation?\n- Is LORA a limiting factor in the model results?\n- How expressive is the reflection removal? Does it handle color bleeding well?\n- How does the model handle the removal of light-emitting objects (Eg  artificial light sources, bioluminescence, etc)?", "limitations": "A more comprehensive analysis of the model limitations would have benefited this paper.", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strenghts\n- The paper tackles a very salient problem in computer vision, which is that of removing shadows and reflections on image inpainting and object removal algorithms.  \n- The proposed method is built upon sound assumptions on how geometry affects images, and how to consider them for adequate removal of objects. \n- The method is sound, is well designed, and combines a smart use of existing components like LORA with particular advancements in the way geometry is completed with diffusion models using a reward-based framework. The second step of the method is very smart, and I think could inspire additional works on image-to-image translation and cycle-consistent generative models. \n- The method is evaluated very well against previous models, with relevant metrics\n- This paper is exceptionally well written, it is very easy to follow and the motivation is clear. \n- Code and data will be shared upon publication, which should greatly improve reproducibility. \n- Supplementary material contains a few additional examples on the ground truth labels. \n- The results of the method are visually excellent. \n- The paper introduces a benchmark dataset with 200 images for the object removal task, which, while limited in size, can be benefitial for the literature. \n\nWeaknesses\n- The paper tackles a somewhat niche problem, as it introduces a method to enhance the results of image inpainting algorithms in the cases where the objects to be inpainted are introducing additional artifacts in the image that are not necessarily present within the inpainting mask. These include shadows or reflections, which are important artifacts. However, I believe this feature also limits the scope of the method to some extent. By introducing this geometric prior, and the fact that the method heavily relies on depth maps, the method is limited to a set of objects and inpainting scenarios. Therefore, it may not be the optimal method for inpainting other types of images, including texture inpainting, watermarking removal, etc. \n- The method relies upon pre-trained diffusion models, and therefore their quality may be dependent on their expressiveness. \n- DreamSIM, SSIM, FLIP, DISTS and CLIP-IQA could have been additional useful metrics for the model evaluation, as they would provide additional insights on the model capabilities. \n- On tables 1-3, for the appropriate metrics (eg LPIPS or PSNR), I think it would be useful to include confidence intervals. These require no additional training or any added computational cost. \n- While I appreciate the use of the Aesthetics Score (AS) in the validation, I am not sure that it is measuring anything particularly related with object removal, as it does not seem to correlate with any other metric. \n- On tables 1-3, I think it would be useful for clarity to show the second best results, as well as the worst results in red.\n- I believe the L_flow should be ablated in more detail, as its impact is not particularly clear.  \n- Additional edge cases could have been interesting, including transparency (object removal through a piece of fabric or a window), reflections from light sources. etc.\n- The method is computationally very expensive.", "was_revised": true, "final_justification": "I appreciate the rebuttal provided by the authors, which has successfully addressed most of my concerns. I highly suggest the authors to incorporate the perceptual metrics into the paper, which really add value to the analyses. I thus increase my rating to a 6 and therefore fully recommend acceptance.  \n\nThe incorporation of perceptual metrics into the analysis, and the discussion about LORA, adds robustness to the paper.", "rebuttal": "### The paper tackles a somewhat niche problem.\n**_Note: Dear Reviewer, we sincerely thank you for your time and effort in reviewing our work. Due to the official policy for this year’s rebuttal process, we are, unfortunately, unable to share the carefully prepared one-page document of additional visualizations. These include failure cases, intermediate results, and various edge case examples, which we found both interesting and thrilling to analyze, as they were inspired by the reviewers’ constructive comments. We also reached out to the AC to ask whether we could share this document in the same way as a code link, and the AC kindly consulted the program chairs on our behalf; however, we were informed that this is not allowed. Despite these constraints, we are making every effort to describe these visualizations in detail so that you can still obtain the corresponding information. These described figures are referred to using lettered indices (Case A–H). We will update our main submission with content from this document._**\n\nOur method indeed is not specifically designed to optimize texture inpainting or watermark removal. However, our model can also handle general inpainting tasks. In fact, it performs competitively on ordinary inpainting scenarios, while showing clear advantages in cases involving causal artifacts. To further demonstrate its applicability, we present a watermark removal example in Case H: a photo of a wooden dock where the watermark is uniformly distributed across both the lake surface and the dock planks. Our model successfully removed the watermark. \n### The method relies upon pre-trained diffusion models & How does the model depend on the diffusion backbone used for the image generation?\n| Backbone | FID ↓ | CMMD ↓ | LPIPS ↓ | PSNR ↑ | AS ↑ | Insert. ↓ |\n|----------|--------|----------|-----------|-----------|--------|-------------|\n| SDXL     |    32.40    |     0.196     |    0.112       |    22.75       |   4.51     |      **1.35%**       |\n| FLUX.1-Fill (Ours) | **31.15** | **0.182** | **0.103** | **23.70** | **4.69** | 1.48% |\n\nWe appreciate the reviewer’s insightful question. Like many diffusion-based approaches, our method inevitably depends on the expressiveness of the underlying pre-trained diffusion backbone, as the quality of Stage 2 image generation is influenced by its ability to render realistic textures and details. However, this is a common setting in current state-of-the-art methods such as CLIPAway, OmniEraser, OmniPaint, and ObjectDrop, all of which also rely on the same family of pre-trained diffusion models. Moreover, to provide a fair reference, we already explicitly reported the baseline performance of the pre-trained diffusion model (FLUX.1-Fill) itself, allowing readers to clearly understand how much improvement our two-stage framework brings beyond the base model’s capability.\n\nTo quantify this dependency, we replaced the default Flux Fill backbone with SDXL Inpainting. As shown in the above table, Flux Fill only achieves slightly better performance on geometry-sensitive metrics, indicating that the overall performance of our method does not heavily rely on the specific diffusion backbone.\n\n###  DreamSIM, SSIM, FLIP, DISTS and CLIP-IQA.\n| Method | SSIM ↑ | DreamSim ↓ | FLIP ↓ | DISTS ↓ | CLIP-IQA ↑ | SSIM ↑ | DreamSim ↓ | FLIP ↓ | DISTS ↓ | CLIP-IQA ↑ |\n|--------|--------|-------------|---------|-----------|--------------|--------|-------------|---------|-----------|--------------|\n|        | *RemovalBench* |        |         |           |              | *RORD-Val* |        |         |           |              |\n| CLIPAway | 0.6298 | 0.1572 | 0.1175 | 0.1656 | **0.4973** | 0.6074 | 0.1304 | 0.1645 | 0.1580 | **0.7986** |\n| Attentive-Eraser | 0.7084 | 0.0536 | **0.0854** | 0.1168 | 0.4790 | 0.7186 | 0.0878 | 0.1174 | 0.1243 | 0.7270 |\n| OmniEraser | 0.6367 | 0.0539 | 0.1084 | 0.1277 | 0.4339 | 0.6071 | 0.0675 | 0.1524 | 0.1325 | 0.6646 |\n| **Ours** | **0.7367** | **0.0304** | 0.0863 | **0.0770** | 0.4146 | **0.8248** | **0.0459** | **0.1026** | **0.0798** | 0.7807 |\n\nWe appreciate the reviewer’s constructive suggestion. Following this advice, we added the suggested metrics for a more comprehensive evaluation. As shown in the above table, our method achieves the best performance on most of them across both RemovalBench and RORD-Val. Regarding CLIP-IQA, we note that, like the Aesthetics Score (AS), it primarily reflects overall image quality rather than object removal accuracy. While it may not strongly correlate with removal effectiveness, we include it for completeness.\n\n###  On tables 1–3, for the appropriate metrics (e.g., LPIPS or PSNR), I think it would be useful to include confidence intervals. \n| Method | LPIPS ↓ (RemovalBench) | PSNR ↑ (RemovalBench) | LPIPS ↓ (RORD-Val) | PSNR ↑ (RORD-Val) |\n|--------|------------------------|------------------------|---------------------|-------------------|\n| CLIPAway | 0.254 ± 0.0337 | 18.78 ± 1.0025 | 0.278 ± 0.0134 | 16.36 ± 0.4525 |\n| Attentive-Eraser | 0.146 ± 0.0153 | 20.60 ± 1.0495 | 0.221 ± 0.0125 | 20.24 ± 0.5060 |\n| OmniEraser | 0.133 ± 0.0225 | 21.11 ± 0.9188 | 0.166 ± 0.0087 | 22.13 ± 0.4093 |\n| **Ours** | **0.124 ± 0.0209** | **25.52 ± 1.1277** | **0.103 ± 0.0067** | **23.70 ± 0.4612** |\nWe have computed and reported the confidence intervals in the above Table. \n\n###  Aesthetics Score.\n\nWe will replace AS with more relevant metrics, such as those you suggested (DreamSIM, SSIM, FLIP, DISTS).\n\n###  Show the second-best results, as well as the worst results in red.\n\nWe appreciate the reviewer’s helpful suggestion. We will incorporate second-best and worst results in the revised version to improve clarity. However, due to the formatting restrictions of the OpenReview rebuttal system, we are unable to use colored fonts here.\n\n###  More ablation study about L_flow.\n\n**_Note: due to space limitations, we may refer to responses addressed to other reviewers; we hope for your kind understanding._**\n\nWe appreciate the reviewer’s constructive suggestion. More detailed ablation study is provided in the response to Reviewer w1kx, Questions **_DPO needs better motivation_** and  **_DPO lambda value_**.\n \n\n### The method is computationally very expensive.\n\nWe appreciate the reviewer’s comment on the computational cost. The detailed computation cost is provided in the response to Reviewer w1kx, Questions **_Computational Overhead & Inference clock time_**\n\n### Edge cases (transparency and translucency)\n\nWe appreciate the reviewer’s insightful question. The edge cases (Case B and Case C) for transparency and translucency can be found in our response to Reviewer Dwf1, Question **_How does the two-stage framework handle dynamic or translucent objects where depth maps are ambiguous?_**.\n\n###  Is LoRA a limiting factor in the model results?\n\nWe appreciate the reviewer’s question. LoRA is not a limiting factor in our framework. In fact, LoRA can already effectively adapt the backbone to our two-stage pipeline, which indicates that full fine-tuning would likely yield even better performance if sufficient computational resources were available. However, when the available training data is significantly smaller than the data used for pre-training—as is common in most downstream tasks—LoRA is often a better choice than full fine-tuning, as it reduces the risk of overfitting while achieving competitive or even superior generalization. Therefore, LoRA should not be considered a limitation in this context.\n\n### How expressive is the reflection removal? Does it handle color bleeding well?\n\nWe appreciate the reviewer’s question. Our method can handle most reflections without overlapping components but currently struggles with overlapping color bleeding. Specifically, for non-overlapping reflections, the model achieves strong performance, as also illustrated in our paper’s Figure 5. \n\nHowever, for complex reflections with overlapping components, such as color bleeding on highly reflective surfaces, our current model struggles. In Case C, a scene contains a large transparent water jug and five semi-transparent cups placed on a reflective table. The reflections of these cups overlap, causing noticeable color bleeding. When attempting to remove one of the semi-transparent cups, both the cup itself and its direct reflection was successfully removed. But the color bleeding in the reflections of the other cups, which was influenced by the removed cup, remains visible. \n\n\n###  Edge cases (light-emitting objects) \n\nOur current model cannot effectively handle light-emitting objects. Case D presents a challenging self-emitting object removal case. Two light bulbs lie on the floor, one casting red light and the other green. Stage1 handles the geometry perfectly—both bulbs and their shapes are accurately removed in the depth map. However, Stage2 behaves unexpectedly: instead of simply erasing the red illumination, it “imagines” a diffuse, amorphous red glow appearing in midair, as if a new red light source had been added to the scene. Although the removal result is unsatisfactory, this behavior reveals an intriguing aspect of Stage 2’s reasoning. It has implicitly learned a causal relationship between illumination and its source—“believing” that if there is red light in the scene, there must be a corresponding red light source to explain it. This over-reasoning causes hallucinated artifacts in this case, yet it also suggests that our rendering model is not merely filling textures arbitrarily; instead, it captures some meaningful light–source correlations, which, if guided properly, could be advantageous in more physically consistent rendering tasks.\n\n### Potential solution for all edge cases\nThis limitation mainly arises from the lack of paired data for all edge cases, which we plan to address by synthesizing targeted training data using text-to-video generation models in future work, using prompts like _A hand removes a red light bulb from a table._"}]

Author rebuttal:

Meta-review: The paper proposes a geometry-aware method for object removal from images. First, the approach proposes to remove the object from the depth using a preference-driven objective. As a second step, the method renders the image conditioned on the updated depth . This work addresses a long-standing issue of high-quality object removal in images, as it is notoriously hard to identify and remove causal effects like shadows or reflections from the image. 

The authors acknowledge that the limitation of the method is that it is hard to handle transparent or dynamic objects, Light-emitting objects or other cases where depth is ambiguous. Throughout the rebuttal, the authors added a new removal metric, perceptual metrics, inference time estimates, ablations on preference-based training, mask-level augmentation.

The reviewers agree on the strong contributions of the paper.
Decision: Accept (spotlight)