Title: GeoRemover: Removing Objects and Their Causal Visual Artifacts
Venue: NeurIPS 2025
Accepted: yes
Keywords: Object removal

diffusion

Geometry
Abstract: Towards intelligent image editing, object removal should eliminate both the target object and its causal visual artifacts, such as shadows and reflections. However, existing image appearance-based methods either follow strictly mask-aligned training and fail to remove these casual effects which are not explicitly masked, or adopt loosely mask-aligned strategies that lack controllability and may unintentionally over-erase other objects. We identify that these limitations stem from ignoring the causal relationship between an object’s geometry presence and its visual effects. To address this limitation, we propose a geometry-aware two-stage framework that decouples object removal into (1) geometry removal and (2) appearance rendering. In the first stage, we remove the object directly from the geometry (e.g., depth) using strictly mask-aligned supervision, enabling structure-aware editing with strong geometric constraints. In the second stage, we render a photorealistic RGB image conditioned on the updated geometry, where causal visual effects are considered implicitly as a result of the modified 3D geometry. To guide learning in the geometry removal stage, we introduce a preference-driven objective based on positive and negative sample pairs, encouraging the model to remove objects as well as their causal visual artifacts while avoiding new structural insertions. Extensive experiments demonstrate that our method achieves state-of-the-art performance in removing both objects and their associated artifacts on two popular benchmarks. The project page is available at https://buxiangzhiren.github.io/GeoRemover.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a diffusion-based model for the removal of objects in images, in a way that also removes the artifacts that these objects create on the rest of the image, including reflections and shadows. The method relies upon a 2-step process: first, depth map object removal and then a cyclical diffusion model that performs both inpainting and removal simultaneously. The method is evaluated across several metrics, ablation studies, and comparisons with previous work, showing a significant advantage compared to baselines.\n\nSTRENGTHS AND WEAKNESSES:\nStrenghts\n- The paper tackles a very salient problem in computer vision, which is that of removing shadows and reflections on image inpainting and object removal algorithms.  \n- The proposed method is built upon sound assumptions on how geometry affects images, and how to consider them for adequate removal of objects. \n- The method is sound, is well designed, and combines a smart use of existing components like LORA with particular advancements in the way geometry is completed with diffusion models using a reward-based framework. The second step of the method is very smart, and I think could inspire additional works on image-to-image translation and cycle-consistent generative models. \n- The method is evaluated very well against previous models, with relevant metrics\n- This paper is exceptionally well written, it is very easy to follow and the motivation is clear. \n- Code and data will be shared upon publication, which should greatly improve reproducibility. \n- Supplementary material contains a few additional examples on the ground truth labels. \n- The results of the method are visually excellent. \n- The paper introduces a benchmark dataset with 200 images for the object removal task, which, while limited in size, can be benefitial for the literature. \n\nWeaknesses\n- The paper tackles a somewhat niche problem, as it introduces a method to enhance the results of image inpainting algorithms in the cases where the objects to be inpainted are introducing additional artifacts in the image that are not necessarily present within the inpainting mask. These include shadows or reflections, which are important artifacts. However, I believe this feature also limits the scope of the method to some extent. By introducing this geometric prior, and the fact that the method heavily relies on depth maps, the method is limited to a set of objects and inpainting scenarios. Therefore, it may not be the optimal method for inpainting other types of images, including texture inpainting, watermarking removal, etc. \n- The method relies upon pre-trained diffusion models, and therefore their quality may be dependent on their expressiveness. \n- DreamSIM, SSIM, FLIP, DISTS and CLIP-IQA could have been additional useful metrics for the model evaluation, as they would provide additional insights on the model capabilities. \n- On tables 1-3, for the appropriate metrics (eg LPIPS or PSNR), I think it would be useful to include confidence intervals. These require no additional training or any added computational cost. \n- While I appreciate the use of the Aesthetics Score (AS) in the validation, I am not sure that it is measuring anything particularly related with object removal, as it does not seem to correlate with any other metric. \n- On tables 1-3, I think it would be useful for clarity to show the second best results, as well as the worst results in red.\n- I believe the L_flow should be ablated in more detail, as its impact is not particularly clear.  \n- Additional edge cases could have been interesting, including transparency (object removal through a piece of fabric or a window), reflections from light sources. etc.\n- The method is computationally very expensive.\n\nQUESTIONS:\n- How does this model handle transparency and translucency?\n- How does the model depend on the diffusion backbone used for the image generation?\n- Is LORA a limiting factor in the model results?\n- How expressive is the reflection removal? Does it handle color bleeding well?\n- How does the model handle the removal of light-emitting objects (Eg  artificial light sources, bioluminescence, etc)?\n\nLIMITATIONS:\nA more comprehensive analysis of the model limitations would have benefited this paper.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-16"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the problem of object removal in 2D images. To better remove objects along with their shadows and mirrored reflections, the authors propose using depth as an intermediate representation in image inpainting. Specifically, they first inpaint the depth map and then reconstruct the image conditioned on the inpainted depth. This allows artifacts caused by reflections and occlusions to be removed at the depth level. \nSpecifically, the authors introduce a two-stage framework: The first stage performs depth inpainting. The second stage generates the final image conditioned on the inpainted depth. Both stages are trained separately based on FLUX and LoRA.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n1. The idea of decoupling object removal into a two-stage process is novel and interesting. By leveraging two strong base models—Depth Anything and FLUX, the proposed approach achieves superior image generation quality, outperforming all state-of-the-art methods on standard benchmarks. \n2. The paper is logically well-structured, and the training procedure for both stages is clearly explained, making it easy to follow.\n3. The authors introduce a real-world test dataset, CausRem, which includes 200 challenging cases such as reflections and shadows. \n\nWeaknesses: \nThe method relies heavily on the depth estimation quality of the Depth Anything model.\n\nQUESTIONS:\nI have two main concerns regarding this paper:\n1. Handling of reflective objects: As shown in Figure 5, the method performs well on several mirror-like reflection cases. However, the paper does not include intermediate visualizations, which makes it somewhat difficult to understand which stage contributes most to the removal of reflection artifacts - whether it is the initial depth estimation, the depth inpainting stage, or the final image generation. Providing intermediate results from each stage could help me better understand the roles and effectiveness of each component in handling these challenging scenarios. \n2. Lack of failure case analysis: While Tables 1 and 4 show that the proposed method outperforms all state-of-the-art baselines in terms of quantitative metrics, the paper would benefit from a more in-depth discussion of failure cases. In particular, it would be informative to include examples where the full model does not perform well, rather than only showing ablation results like those in Figure 4. \nUnderstanding where and why the model fails would provide valuable insight into its limitations and potential areas for future improvement.\n\nLIMITATIONS:\nThe method relies on the Depth Anything model for depth estimation, which may limit performance in certain challenging scenarios, especially where depth prediction is inherently ambiguous.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a two-step diffusion pipeline for object removal that also erases the object’s shadows using depth as a geometry signal. In the 1st stage, a depth-aware diffusion model takes the masked image and its depth map, then learns to remove the object in the depth domain with DPO loss without artefacts. Next, a second diffusion model receives three inputs which are the original RGB image, the original depth map, and the object removed depth map. With these inputs, the second model produces the final image in which both the object and any shadow or reflections are removed.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths** \n- The paper is well written and easy to follow.\n- The paper tries to solve a valid problem being present in the current literature. Significance is high.\n- The paper made extensive comparisons with current literature including both GAN based and Diffusion based models as comparison.\n- Using Depth as a supervision for removing shadows in supervision as well as two-stage decoupling with the DPO-guided depth loss is a novel approach to tackle the problem.\n\n**Weaknesses** \n- **Limitations need more explanation:**  The authors state the high computational cost of 2 stage pipeline as a limitation (lines 509-514) which is correct but I think the failure cases as a limitation also needs to be discussed for both stages.\n- **Use of DPO despite a qualitative example in figure 3a needs a better explanation in terms of motivation:**\nFigure 3a does illustrate that training **with** DPO yields cleaner geometry than training **without** it, yet this single visual comparison does not fully justify adopting a RL-style objective in place of simpler depth-smoothness regularizer such as total variation loss. Conceptual explanation of why DPO is more suitable for this task needs more expanding. \n- **Computational overhead:** Due two 2 stage inference, this model is more computationally expensive compared to other baselines.\n\nQUESTIONS:\n- How robust is both stages of the pipeline with respect to the possible incorrect depth maps? For example, in a failure case of depth anything in stage 1, how does the geometry removal of the object behaviour is effected for the model. Also, if the geometry removal of stage 1 is not perfect, how robust is the second stage pipeline to such changes? I think adding some slight noise to the original RGB images or increasing resolution which is explained as a failure case for the model [2] (In limitations section), and then executing the pipeline and analyzing how the scores have been changed can provide a better explanation in terms of the inner workings of the model.\n\n- REMOVE [1] which is a metric that directly measures the object removal quality can also be used for analysis to better state the success of this paper compared to other baselines. \n\n- I would like to see example failure cases for both stages of the pipeline. Currently the paper only includes success cases.\n\n- What is the λ value used for the DPO coefficient in the stage 1 loss?    \n\n- What is the inference clock time overhead of this method compared to other baselines? This is stated as a limitation so can be better provided with numbers.\n\n---\n\n**I would like to see more failure case examples and robustness of the model as well as DPO's effectiveness compared to a more simple alternative. Given satisfactory answers, I would increase my score.**\n\n---\n[1] Chandrasekar, Aditya, et al. \"Remove: A reference-free metric for object erasure.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024. [code](https://github.com/chandrasekaraditya/ReMOVE)\n[2] Yang, Lihe, et al. \"Depth anything: Unleashing the power of large-scale unlabeled data.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.\n\nLIMITATIONS:\nLimitations need more explanation. The authors state the high computational cost of 2 stage pipeline as a limitation (lines 509-514) which is correct but I think the failure cases as a limitation also needs to be discussed for both stages.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces GeoRemover, a two-stage framework for object removal that explicitly addresses the challenge of eliminating both target objects and their causal visual artifacts e.g., shadows, reflections. The first stage removes objects from geometric representations, depth maps, using strictly mask-aligned training, ensuring structural accuracy. The second stage renders a photorealistic RGB image conditioned on the updated geometry, depth map, implicitly removing artifacts. A preference-driven objective, inspired by DPO, is introduced to prevent hallucinations during geometry removal. Experiments on benchmark datasets shows state-of-the-art performance in both object and artifact removal.\n\nThe paper presents a significant advancement in object removal by addressing causal artifacts through a principled, two-stage approach. While computational costs and paired data requirements are limitations, these are common trade-offs in state-of-the-art methods and do not detract from the paper’s contributions.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. Novel Two-Stage Framework: Decoupling geometry removal and appearance rendering enables precise structural editing and implicit artifact removal, addressing a critical challenge occurred in previous work.\n2. Preference-Driven Training: The DPO-inspired loss effectively suppresses hallucinations and ensures geometric consistency, improving controllability compared to loosely mask-aligned methods.\n3. SOTA performance: GeoRemover outperforms existing methods on multiple metrics (FID, PSNR, IoU) across two benchmarks, including a new CausRem dataset for causal artifacts.\n\nWeaknesses:\n1. Computational Cost: The two-stage pipeline increases runtime and memory usage compared to one-stage methods.\n2. Reliance on Depth Estimation: The method depends on external depth estimators, whose accuracy could degrade in complex scenes.\n3. Paired Data Requirement: The rendering stage requires paired data (images with and without objects/artifacts), which may not always be available in real-world scenarios.\n\nQUESTIONS:\n1. How does the two-stage framework handle dynamic or translucent objects where depth maps are ambiguous?\n2. What are the trade-offs between the strict mask alignment in Stage 1 and the flexibility of loose alignment in prior work?\n\nLIMITATIONS:\n1. High Resource Usage: The two-stage design increases computational overhead, compared with one-stage methods.\n2. Depth Estimator Dependency: Performance is tied to the quality of the depth map, which may introduce errors in low-texture or occluded regions.\n3. Artifact Scope: The method focuses on shadows and reflections; how about other artifacts, such as motion blur", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-13"}