GeoComplete: Geometry-Aware Diffusion for Reference-Driven Image Completion
Authors: Beibei Lin
Tingting Chen
Robby T. Tan
Venue: NeurIPS 2025 poster ()
Keywords: Reference-Driven Image Completion
Primary area: applications

Abstract: Reference-driven image completion, which restores missing regions in a target view using additional images, is particularly challenging when the target view differs significantly from the references. Existing generative methods rely solely on diffusion priors and, without geometric cues such as camera pose or depth, often produce misaligned or implausible content. We propose GeoComplete, a novel framework that incorporates explicit 3D structural guidance to enforce geometric consistency in the completed regions, setting it apart from prior image-only approaches. GeoComplete introduces two key ideas: conditioning the diffusion process on projected point clouds to infuse geometric information, and applying target-aware masking to guide the model toward relevant reference cues. The framework features a dual-branch diffusion architecture. One branch synthesizes the missing regions from the masked target, while the other extracts geometric features from the projected point cloud. Joint self-attention across branches ensures coherent and accurate completion. To address regions visible in references but absent in the target, we project the target view into each reference to detect occluded areas, which are then masked during training. This target-aware masking directs the model to focus on useful cues, enhancing performance in difficult scenarios. By integrating a geometry-aware dual-branch diffusion architecture with a target-aware masking strategy, GeoComplete offers a unified and robust solution for geometry-conditioned image completion. Experiments show that GeoComplete achieves a 17.1% PSNR improvement over state-of-the-art methods, significantly boosting geometric accuracy while maintaining high visual quality.

GeoComplete: Geometry-Aware Diffusion for Reference-Driven Image Completion
Beibei Lin Tingting Chen Robby Tan
National University of Singapore
(beibei. lin, tingting. cu. mus..uu..... edu, robby. tan@nus edu. edu.sg
Abstract
Reference-driven image completion, which restores missing regions in target view using additional images, is particularly challenging when the target view diffees significantly from the references. Existing generative methods rely solely on diffusion priors and, without geometric cues such as camera pose or depth, often produce misaligned or implausible content. We propose GeoComplete, a novel framework that incorporates explicit 3D structural guidance to enforce geomet- ric consistency in the completed regions, setting it apart from prior image-only approaches. GeoComplete introduces two key ideas: conditioning the diffusion process on projected point clouds to infuse geometric information, and applying target-aware masking to guide the model toward relevant reference cues. The framework features a dual-branch diffusion architecture. One branch synthesizes the missing regions from the masked target, while the other extracts geometric features from the projected point cloud. Joint self-attention across branches en- sures coherent and accurate completion. To address regions visible in references but absent the target, we project the target view into each reference to detect occluded areas, which are then masked during training. This target-aware masking directs the model to focus on useful cues, enhancing performance in difficult sce- dual-branch diffusion architecture with
narios. By integrating geometry target-aware masking strategy, GeoComplete offers unified and robust solution for recometry-condittonee image completion. Experiments show that GeoComplete achieves 17.1 % PSNR improvement over state-of-the-arr methods, significantly boosting geometric accuracy while maintaining high visual quality.
Introduction
Reference-driven image completion restores missing regions in target image using additional views of the same scene. However, variations in viewpoint, occlusions, dynamic content, and camera settings make difficult identify and transfer useful information, posing significant challenges for accurate completion.
To address these challenges, traditional geometry-based methods 33
rely sequential pipeline of pose estimation, depth reconstruction, 3D warping, patch fusion, and image harmonization. However, highlighted in [40]. this approach is fragile, carly-stage errors often cascade and lead to failure in complex scenes with occlusions, dynamic content, or ambiguous geometry. To handle complex scenes, generative methods like RealFill [40] fine-tune diffusion models masked reference images to directly synthesize missing regions. While effective, RealFill struggles when the target view differs significantly from the references. Without geometric cues like camera poses depth, it often produces hallucinated structures or misaligned completions.
48 49
In this paper, propose GeoComplete, a geometry-aware image completion framework that synthe- sizes missing regions with strong geometric consistency. GeoComplete is based on two key ideas: (1)
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Ref. Images
Target Images
GT
Puint-by by-bxaplee
Ours
Figure 1: Given few reference images of the same scene and target image with missing regions, our method completes the target's missing regions while preserving geometric consistency more effectively than the state-of-the-ar Paint-by-Fxample the known, unaltered regions of the target image.
Semi-transparent white masks indicate
injecting explicit geometric cues into diffusion model by conditioning on the projected point cloud, and (2) guiding the model to focus on informative reference regions via target-aware masking. We define "informative regions"
that visible in reference views but missing from the target.
To obtain point clouds, our framework integrates two components: Visual Geometry Grounded Transformer (VGGT) [I] and Language Segment Anything (LangSAM) 25
Unlike traditional geometry-basee methods (e.g.. [35 48 49)) that rely on sequential estimation steps, VGGT predicts key 3D attributes in single forward pass. Trained on large-scale data, VGGT delivers accurate and efficient geometry estimation, even in complex scenes, though its performance may degrade with dynamic objects. To address this, integrate LangSAM, which segments dynamic regions using text prompts. By filtering out moving content before point cloud generation, LangSAM enhances the robustness of geometry estimation. Prompts be provided manually generated automatically by large language model (LLM) 1
22 32
The resulting point cloud is projected to the target view and fed into dual-branch diffusion framework, comprising a target branch and a cloud branch. The target branch encodes the masked image generate missing content. The cloud branch processes the projected point cloud provide geometric guidance. Joint self-attention fuses the two branches, enabling geometry- synthesis of missing regions.
To address the challenge of completing regions not visible the target view, we introduce target- aware masking to guide the model toward useful and non-redundant reference cues. Using 3D attributes from VGGT, we project the target view into each reference identify informative regions. Rather than masking reference images randomly in RealFill [40), selectively mask these informative regions to encourage the model learn from content that complements the target view.
Figure g shows that GeoComplete significantly outperforms state-of-the-art methods, producing missing regions with strong geometric consistency. The main contributions of this work are:
Dual-branch Diffusion: We propose a geometry-aware dual-branch diffusion model that synthesizes missing regions with strong geometric consistency. It comprises target branch, which conditions the diffusion model on the masked image to generate missing content, and cloud branch, which conditions it the projected point cloud to provide geometric cues.
Target-aware Masking Strategy: Unlike RealFill, which applies random masking to reference image, method selectively masks informative regions to guide the diffusion model toward meaningful cues, leading to more accurate and coherent completions.
Extensive Experiments: GeoComplete significantly outperforms existing methods in both structural accuracy and visual fidelity. Specifically, our method surpasses ofate-of-the-art approaches by 17.1% % in PSNR on benchmark datasets.

---

Point Cloud Generation
Target-sware Masking
Dual-branch Diffasion
Conditional
Cloud Branck
Joint Self-Atteutio
Projected Point Clouds
Cloud Masking
Torget Point
Avy
Ref. Images
Generated Point Cloud
Projected Point Clond
EdE Enceder
Poler
Weights
(wery
Frozea
Training Inguts
Conditional Reference Masking
Loss
Target Images
Torget Bronch
LaRA
Figure 2: Overview of GeoComplete framework: We first construct point cloud from the reference and target images. During training, target-aware masking selectively occludes both reference images and their projected point clouds to highlight informative regions. These inputs processed by a dual-branch diffusion model: the target branch encodes the masked image, while the cloud branch encodes the projected point cloud. Joint self-attention fuses the two, allowing geometric cues to guide synthesis. At inference, the masked target image and its projected point cloud are fed into the finetuned model to complete the missing regions.
2 Related Work
Image Completion Traditional methods 15
21
38 18
employ task-specific networks to fill
42 20] leverage pre-trained diffusion models to achieve strong image generation capabilities. Inspired by this, several fine-tune diffusion models with prompt guidance for image completion. In our setting, text prompts fail to capture the rich cues available in reference images, leading to suboptimal combine depth and pose estimation, image warping, error-prone and often compound failures, especially in dynamic scenes. Morcover, their limited generative ability hinders plausible content synthesis. Recent diffusion-based methods [4 40 draw Stable Diffusion priors. Paint-by-Exampl 44
missing regions. Existing generative approaches 45
10
29 7
2
methods [37 29 2
results. Refeerencedrrvenn methods [49
4 3
and harmonization, but these components
uses the target image and a CLIP embedding (28) of a single reference for semantic guidance, while RealFill [40] adapts the diffusion model per scene via LoRA to reconstruct masked references with multiple inputs. However, both approaches neglect geometric cues such as depth and pose, which are erucial for spatial consistency across views. Our method addresses this gap by explicitly injecting geometry into the diffusion model, enabling geometry-aware generation with improved spatial alignment. Concurrently, other works 33 36 couple NeRF (26) 3DGS [17 with
diffusion for scene inpainting. For example, the Geometric-aware 3D Scene Inpainter (33 conditions
diffusion on multi-view images and geometry reconstruct 3D structure. These methods, however, assume shared geometry across views, limiting applicability in dynamic or varying conditions.
Geometric Information Estimation Existing geometry-basee
completion methods 49
48
35
depend on separate estimation modules such as camera pose [46] [6], monocular depth 30
19
14],
3] 4 to enable view warping. In contrast, the Visual Geometry Grounded Transformer (VGGT) 41 unifies these tasks by jointly predicting camera parameters, depth maps, point maps, and 3D tracks directly from input views. While VGGT achieves strong results in static scenes, it struggles with dynamic objects. To address this, we incorporate LangSAM 25 22 32 to filter dynamic content before applying VGGT, enabling more reliable 3D>
and feature matching with robust fitting [34] 12
attribute prediction in such settings.
3 Proposed Method
Figure shows the overall pipeline of GeoComplete, which comprises three key components: point cloud generation, dual-branch diffusion, and target-aware masking. The point cloud generation module estimates camera parameters and depth maps from the reference and target images, constructs a 3D point cloud, and projects onto both views to provide geometric guidance. The dual-branch diffusion model then synthesizes the missing regions while integrating this geometric information. Finally, the target-aware masking strategy directs the model to focus on reference regions that are not visible from the target view, encouraging the use of complementary cues.

---

Projected
Prompt "cal"
Reference
VGGT
Point
Projectod Ref, Images
Imugges
Masks
Cloud
Human
Target Image
Projected
Torget Image
Projected
Figure 3: Overview of our point cloud generation pipeline. Given reference and target images, we first obtain a text prompt describing dynamic objects, either provided by users or generated by an LLM D. Based on the prompt, LangSAM [25] 22 [32 is employed to segment and remove dynamic
regions. VGGT [4] then applied estimate camera parameters and depth maps, which are used to construct and project the 3D point cloud.
3.1 Point Cloud Generation
Figure shows point cloud generation pipeline. Given set of reference images and target image, we first obtain a text prompt that deseribes dynamic objects the scene. The prompt is preferably provided by the user; if unavailable, it is automatically generated by large language model (LLM) LangSAM 25 22 32 uses this prompt to segment and filter out dynamic regions in both the reference and target images. By removing dynamic objects, geometry estimation focuses on the static scene, enabling reliable correspondences across views.
We use VGGT 41 to jointly estimate camera parameters and depth maps from the filtered reference and target images, avoiding the eπror accumulation common in multi-stage geometry pipelines. The resulting 3D attributes form point clouds, which projected onto both views provide explicit geometric guidance. To prevent over-reliance on potentially inaccurate point clouds, we apply conditional cloud masking strategy (Section 3.2 that introduces random masking during training.
Given a reference images (x!et | = 1,2,..... .Nref) and target image xtar we introduce scene- spenifcc text prompt p dyy to describe dynamic objects. If not provided by the user, p dyn is automatically generated by a large language model (LLM). Using this prompt, we apply LangSAM segment dynamic regions in both reference and target images. The resulting segmentation masks are {māef) for the references and m for the target. We then mask out these dynamic regions to produce filtered images (x!ef) } and tar, preserving only the static content of the scene.
To estimate the camera parameters and depth maps for all filtered reference images and the target image, we formulate the prediction process using VGGT as:
(fctef) clar (dref)
d fvggt ([x)0)) (.ggtt
(1)
where (cfet) and {d are the predicted camera parameters and depth maps for the reference images, and tar and d are those for the target image. 0vggt the pre-trained parameters of VGGT.
To obtain the projected point cloud for each reference image x,ef, we first exclude its own information during point cloud construction. The resulting point cloud is then projected onto the reference view, formulated as:
p ref
=
(-- ((d))t ccjof 1 fd (d .c,e))
c,ar)),
where π (-) denotes the back-projection from depth maps to 3D space, and T-)) denotes the forward projection onto the 2D image plane. p, is the projected point cloud for the reference image x Similarly, the point cloud constructed from all reference images is projected onto the target view:
tar (ir (fdgof ccd
ref Vj}) ctar),
where ter is the projected point cloud for the target image x 1ar

---

3.2 Target-aware Masking
During training, we apply target-aware masking to selectively mask both the reference images and their projected point clouds. Using 3D geometric attributes from point cloud generation, we project the target image into each reference view to identify regions that are absent in the target (i.e., informative regions). As shown in Figure 2 information, while the remaining areas are treated as redundant cues.
these informative regions provide complementary
We then apply two conditional masking strategies: conditional reference masking and conditional cloud masking. The reference masking randomly masks informative regions while preserving redundant ones, encouraging the model to learn from complementary content. The cloud masking, on the other hand, randomly applies white padding to the projected point maps while keeping informative regions intact, guiding the model to leverage geometric cues in these informative areas.
Given the predicted depth maps (fd,ef) and d!arr) and camera parameters (fc,*)) and c!ar), project the filtered target image xtar onto each reference view x defined as:
tar-ref Pi
(dtar, c!arr c..
where p. paar-ref is the projection of the target view into the i-th reference view. We convert this projection into a binary mask reef where visible regions are set to 0 and zero-valued regions are set to 1.
The conditional reference masking is defined as:
x
x @ ((1-r + r:el c m rand)
(5)
where m m.and {0, l is random binary mask applied only the informative regions, and H and W are the height and width the reference image. The operator @ denotes clement-wise multiplication, and &g is the resulting masked reference image. This operation preserves redundant content while randomly masking informative, non-redundant regions.
In contrast, conditional cloud masking retains non-redundant geometric regions while applying random masking to redundant ones. defined as:
point r + (1 (1-rr,e) O m m
rand
(6)
ref p m point Van x m Point),
where Van a predefined fill value assigned to masked-out points, and pf is the masked projected point cloud for the i-th reference view. This operation guides the model to rely geometric cues in regions where visual reference content is lacking.
Following the training strategy of 440, we also sample the target image x tar as input during training. We apply random masking to obtain x tar while the projected point cloud ptar is directly used as ptar.
3.3 Dual-branch Diffusion
Figure shows the pipeline of our dual-branch diffusion model, which consists of target branch and cloud branch. The target branch conditions the diffusion model on masked image generate missing regions. The cloud branch conditions on a projected point cloud to provide geometric cues. The masked image and projected point cloud are first encoded into the latent space using VAE encoder. The resulting latent features, along with noisy latent, are then passed a UNet for denoising.
To enable information exchange between branches, we concatenate hidden features from the target and cloud branches before computing self-attention. This design allows the model to adaptively integrate visual and geometric cues. However, since most regions in the target image are masked, the resulting latent features often lack meaningful information. Although these masked tokens can attend to the cloud branch, they struggle to extract useful guidance. To overcome this, we modify the attention mask to explicitly link each masked token in the target branch to its corresponding token in the cloud branch. This ensures that the target branch receives direct geometric cues, even when visual information is absent.
5

---

Table Quantitative comparisons on the RealBench benchmark. We evaluate both prompt- and reference-based inpainting methods across lew-level (PSNR, SSIM, LPIPS), mid-level (DreamSim), and high-level (DINO, CLIP) metrics. Higher PSNR, SSIM, DINO, CLIP, and User Study scores
(ranging from to 5), and lower LPIPS and DreamSim scores, indicate better performance. We highlight the best and second- best results for each metric.
RealBench
QualBench
Method
Low-level
Mid-level
High-level
User
PSNR
SSIM T
LPIPS
DreamSim J
DINO f
CLIP f
Study f
SD Inpaint
10.63
0,282
0,605
0.213
0.831
0.874
2.13
Prompt-bused
Generative Fill
10.92
0,311
0.598
0.212
0.851
0.898
2.61
Paint-by by- Example
10.13
0.244
0.642
0.237
0.797
0.859
1.85
TransFill
13.28
0,404
0,542
0.192
0.860
0.866
Reference-based
RealFill
14.75 18.78
0.424
0.077
0.962
Ours
17.32
0.578
0.197
0.036
0.986
0.987
4.61
Given the hidden features from the target and cloud branches, denoted as ht and hpt. we concatenate them along the token dimension:
heat = Concat (h hpt),
where heat. E R2Lxd, with L representing the number of tokens per branch and d the feature dimension. The combined features heat are then used for self-attention.
To control information flow, we introduce an attention mask mattn 22 during self-attention. The mask constructed to: (I) allow tokens within the same branch attend to each other, (2) permit each barget-branch token to attend to its corresponding cloud-branch token, and (3) block all
other cross- cross-branch interactions. An illustration of the attention mask is shown in Figure The joint self-attention is formulated as:
hattn fselr-at (beat Mattn)
(8)
where fselt-amm-)) denotes the masked self-attention operation, and hattn is the resulting attended feature.
During training, the diffusion loss is defined as:
1 Σ Er, [Iw; (e-eo(x,(t).. P;, B
C
where C is the diffusion loss, B is the batch size, and E-(-) denotes the predicted noise. The conditional inputs satisfy &; E (xpref) } U (xt"r} and P. E [pre(] } U (ptar) Here, xt) is the ground-truth image at timestep I, and Wj is weighting map indicating valid regions (e.g., visible areas in the target view). The loss is computed only over these valid regions.
During inference, use p and tar conditional inputs to guide the dual-branch diffusion, generating missing regions while preserving geometric structures.
4 Experiments
In our experiments, we follow the evaluation protocol of (40 and test on two challenging reference- based image completion datasets: RealBench and QualBench.
RealBench 40 contains 33 scenes (23 outpainting and 10 inpainting). Each scene provides 1-5 reference images, target image with missing regions, a binary mask, and ground-truth completion. Scenes include large variations between target and references, such as viewpoint, blur, lighting, style, and pose. Evaluation uses six metrics: PSNR, SSIM, LPIPS [47), DreamSim [3). DINO [8). and CLIP 28). PSNR, SSIM, and LPIPS capture low-level quality, while DreamSim, DINO, and CLIP assess perceptual fidelity mid- and high-levels.

---

Ref.
GT
Ours
Paint-by-Example
TransFill
RealF
44 and our method. The red bounding box marks the known, unaltered region of the target image (i.c., the area inside the box), except for the first-row images, where the known region lies outside the box. Our method synthesizes missing regions while ensuring better geometric consistency.
Figure 4: Qualitative comparisons from Transfill 49). RealFill [40],
includes 25 scenes collected in the same way but without ground-truth completions. We therefore conduct user study where participants rate each result (1-5) based on: (1) realism of the restored content, (2) consistency with references, and (3) structural and color coherence with the unmasked target. Higher scores reflect more natural, geometrically consistent, and visually coherent completions.
QualBench 40
4.1 Implementation Details
All experiments are conducted on server equipped with four NVIDIA GPUs, each with 24 GB of memory. Our implementation involves three key components: point cloud generation, target-aware masking, and dual-branch diffusion, each of which is described in detail below.
Point Cloud Generation Our point cloud generation pipeline incorporates two key components: LangSAM 25 22 32 and VGGT [4I] In LangSAM, we employ SAM L-arge [32] for segmen- tation. The text prompts are either manually provided by users automatically generated by large language model, CharGPT-44 (27). Since VGGT only supports inputs of size 518 x 518, we resize the reference and target images while preserving their aspect ratios. After resizing, center crop is applied to obtain the final 518 x 518 resolution.
Target-aware Masking Our target-aware masking consists of conditional reference masking and a conditional point masking. Following the strategy in [8 39), the conditional reference masking first generates multiple random rectangles and constructs the initial mask by either taking their union or the complement of their union. Subsequently. following Equation 5 it selectively unmasks less informative regions in the reference images. Similarly, the conditional point masking first generates the initial mask and then selectively unmasks non-redundant geometric regions, as defined in Equation o The fill value Uan is set to (white) to replace masked-out regions in the projected point cloud.
Dual-branch Diffusion Our diffusion model is built upon Stable Diffusion 2 Inpainting 377. We
fine-tune with LoRA, updating only rank-decomposed layers the U-Net while keeping original weights frozen. The LoRA rank is set to 8 to balance adaptation capacity and training efficiency. For each scene, we fine-tune the model for 2,000 iterations with batch size of During training, all reference and target images resized resolution of 512 x 512.

---

FER Ba
Paiat-by-Example
TransFill
RealFill
Paint-by-Examin 44 and our method. The red bounding box marks the known, unaltered region of the target image (i.e., the area inside the box). Our method synthesizes missing regions while ensuring better geometric consistency.
Figure 5: Qualitative comparisons from Transfill 49 RealFill [40).
4.2 Evaluation
Quantitative Table D compares GeoComplete with state-of-the-art methods on RealBench and QualBench. Baselines include prompt- based approaches (SD Inpaint [37] Generative Fill (2]) and reference-basee methods models rely on text input, while the reference- based ones use images for completion.
[44). TransFill [49] RealFill [40]). The prompt-based
Compared to prompt-based methods, GeoComplete achieves large gains across all low-level metrics, improving PSNR by over geometry-awan baseline, our model benefits from VGGT and LangSAM to generate more reliable point clouds, yielding notable improvements. While RealFill already performs strongly with masked reference conditioning, GeoComplete further improves SSIM (0.424 - 0.555) and reduces LPIPS by 1949, producing sharper and more perceptually faithful reconstructions. These results underscore the importance of explicit 3D geometrie priors and validate the effectiveness of our dual- bual-branch diffusion design.
and reducing L.IPS from 0.605 to 0.237. Against TransFill,
Qualitative Figures 4 and show qualitative comparisons on RealBench. Generative frameworks such as RealFill leverage reference images to produce plausible completions, but without explicit geometry they often fail to maintain spatial consistency, leading to misaligned or implausible content. In contrast, GeoComplete enforces geometric consistency by integrating priors from LangSAM and VGGT within a dual-branch architecture that jointly encodes visual and 3D cues. As illustrated in Figure 4 GeoComplete reconstructs fine details and preserves scene-level alignment, even under large viewpoint changes between the target and references.
4.3 Ablation Studies
We conduct ablation experiments evaluate the contributions of target-aware masking and dual- branch diffusion. Results are reported in Table Without geometric guidance, our method reduces to RealFill, shown in the first row as the baseline.
Dual-branch Diffusion We compare GeoComplete with and without explicit geometric cues. As shown in Table removing geometry causes clear drops across all metrics (e.g., PSNR and SSIM decrease by 1.59 and 0.13... Figures 4 and 5 further illustrate that without geometry, RealFill often produces hallucinated misaligned content. By contrast, GeoComplete integrates geometric information into the generation process, yielding structurally consistent results. This highlights the importance of explicit geometry in guiding diffusion-based restoration.

---

Table 2: Ablation study on the effectiveness of dual-branch diffusion, joint self-attention, and target-aware masking. We report low-level (PSNR, SSIM, LPIPS), mid-level (DreamSim), and high-level (DINO, CLIP) metrics. Higher PSNR, SSIM, DINO, and CLIP scores and lower LPPPS and DreamSim scores indicate better performance.
Dual-branch
Joint Self-Attentio
Target-awure
Low-level
Mid-level
High-level
Diffusion
with Mask
Masking
PSNR
SSIM f
LPIPS
DreamSim 4
DINO
CLIP
X
X 14.78
0.424
0,431
0.077
0,948
0,962
x
X 16.37
0.555
0.237
0.049
0.981
0.982
V
X 16.85
0.564
0.219
0,045
0,983
0,984
V 17.32
0.578
0.197
0.036
0.986
0.987
Table 3: Robustness of GeoComplete to VGGT and LangSAM Errors, We simulate (1) noisy point clouds, (2) sparse point clouds, and (3) LangSAM segmentation errors. "0% / 25% 50% / 75% indicate ratios of points perturbed or removed. For the LangSAM case, "w/." and "w/o." denote using or removing the masks, while "+Rand." denotes randomly adding 10% extra masked regions per mask. CM Conditional Cloud Masking, JSA = Joint Self-Attention PSNR (dB) is reported.
Noisy Point Cloud
Sparse Point Cloud
LangSAM (13 scenes)
Method
0%
25%
50%
75%
0%
25%
50%
75%
w/.
w/o.
+Rand.
Realliill
14.78
14.78
14.78
14.78
14.78
14.78
14.78
14.78
14.44
14.44
14.44
Ours w/o. CM JSA
16.37
14.60
14.51
14.35
16.37
14.58
14.50
14.35
15.92
14.54
14.58
Ours
17.32
17.14
17.03
16.90
17.32
17.18
16.83
16.50
16.83
16.66
16.51
Joint Self-Attention We ablate the joint self-attention module, which fuses target and floud-brance features under a controlled attention mask. As shown in Table removing this module results in noticeable drops in PSNR, CLIP, and DINO, reflecting weaker low-level fidelity and high-level semantic alignment. These results demonstrate the role of joint self-attention ensuring alignment between the target and projected point cloud.
Target-aware Masking We also evaluate the effect of target-aware masking. Removing this strategy consistently reduces performance across all metrics (e-g., PSNR and SSIM drop by 0.47 and 0.014, while CLIP and DINO decrease by 0.003). This indicates that target-aware masking helps the model focus on non-redundant regions in the references, improving inference accuracy and fidelity.
Robustness of GeoComplete GeoComplete relies on upstream modules such VGGT and LangSAM, whose outputs may contain errors (see See. 3.1 To mitigate this, we introduce condi- tional cloud masking (CM), which prevents the model from over-relying on unreliable geometry. We also employ joint eeff-attention with masking (JSA), which enforces token-to-tooe links between the target and cloud branches, ensuring that noisy cloud tokens do not propagate globally through cross-attention, particularly when they dominate.
To evaluate robustness, we simulate (1) noisy point clouds, (2) sparse point clouds, and (3) LangSAM segmentation errors. Details can be found the Appendix. As shown in Table 3 GeoComplete shows only small drops under these perturbations and consistently outperforms both RealFill and the variant without CM and JSA. These results demonstrate the effectiveness of CM and ISA in maintaining strong performance even when upstream predictions are noisy or partially erroneous.
5 Conclusion
We introduced GeoComplete, a gcometry-guided diffusion framework for reference-driven image completion. Unlike existing generative methods that operate solely in the image domain, GcoCom- plete incorporates explicit 3D geometry by conditioning the diffusion model on projected point clouds. To guide the model toward meaningful reference cues, we propose a target-aware masking strategy that filters redundant content and emphasizes complementary regions. Our dual- branch ar- chitecture jointly processes geometric and visual tokens through self-attention, enabling the synthesis of structurally accurate and visually coherent results. Extensive experiments on real-world and syn- thetic benchmarks demonstrate that GeoComplete achieves clear improvements over state-of-ther methods in both geometric consistency and perceptual quality.

---

References
[1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Iige Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint ar 2rX3vv2303.08774, 2023.
[2] Adobe Inc. Adobe photoshop, 2023.
[3] Eric Brachmann, Alexander Krull, Schastian Nowozin, Jamie Shotton, Frank Michel, Stefan Gumhold, and Carsten Rother. Dsac-differentiable ransac for camera localization. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pages 6684-6692, 2017.
[4] Eric Brachmann and Carsten Rother. Neural-guie guided ransac: Learning where to sample model hypotheses. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4322 1231-433, 2019.
[5] Xiao Cao, Beibei Lin, Bo Wang, Zhiyong Huang, and Robby T Tan, Ssnerf: Sparse view semi-supervised neural radiance fields with augmentation. arXiv preprint arXiv:2408.09) 09144, 2024.
[6] Xiao Cao, Beibei Lin, Bo Wang, Zhiyong Huang, and Robby T Tan. 3d-ott: Texture transfer for 3d objects from a single reference image. arXiv preprint arXiv:2503. 18853, 2025.
[7] Xiao Cao, Yuyang Zhao, Robby Tan, and Zhiyong Huang. Bridging editing and paired dataset creation for 2d nighttime-to-daytinng translation. In /CVPR 2025 Workshopl SyntaGen: 2nd Workshop on Harnessing Generative Models for Synthetic Visual Datasets.
[8] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In 2021 IEFF/CVF International Conference on Computer Vision, ICCV 2,2 Montreal, QC, Canada, October 10-17, 2021, pages 9630- 9640. IEEE, 2021.
[9] Huiwen Chang, Han Zhang, Jarred Barber, AJ Maschinot, Jose Lezama, Lu Jiang, Ming- Hsuan Yang. Kevin Murphy, William T Freeman, Michael Rubinstein, et al. Muse: Text-to-image generation via masked generative transformers. ArXiv preprint, abs/2301 000044, 2023.
Ming-Hsuan
[10] Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William T. Freeman. Maskgit: Masked generative image transformer. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA. June 18-24, 2022, pages 11305-11315 IEEE, 2022.
[11] Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang. Shu Zhong, Weihao Yu, Xiaonan Nie, Ziang Song. Guang Shi, and Haoqi Fan. Emerging properties in unified multimodal pretraining. arXiv preprint arXiv:2505. 2025.
[12] Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. Superpoint: Self-supervised interest point detection and description. In Proceedings of the IEEE conference on computer vision and pattem recognition workshops, pages 224-236, 2018.
[13] Stephanie Fu, Netanel Tamir, Shobhita Sundaram, Lucy Chai, Richard Zhang, Tali Dekel, and Phillip Isola. Dreamsim: Learning new dimensions of human visual similarity using synthetic data, ArXiv preprint. abs/2306.09344 2023.
[14] Lam Huynh, Phong Nguyen-Ha, Jiri Matas, Esa Rahtu, and Janne Heikkilä. Guiding monocular depth estimation using -atttttnttenionn volume. In European Conference on Computer Vision, pages 581-597. Springer, 2020.
[15] Satoshi lizuka, Edgar Simo-Serra, and Hiroshi Ishikawa. Globally and locally consistent image completion. ACM Transactions Graphics (ToG), 36,
[16] Alex Kendall, Matthew Grimes, and Roberto Cipolla. Posenet: A convolutional network for real-time 6-dof camera relocalization. In Proceedings of the IEEE international conference computer vision, pages 2938 2946, 2015.
[17] Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trunsactions on Graphicx, 42(4):1-14, 2023.
[18] Soo Ye Kim, Kfir Aberman. Nori Kanazawa, Rahul Garg, Neal Wadhwa, Huiwen Chang, Nikhil Karnad, Munchurl Kim, and Orly Liba. foom-to-inpaint Image inpainting with high-frequency details. In IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2022, New Orleans, LA, USA. June 19-20, 2022, pages 476 486. IEEE, 2022.
10

---

[19] Jin Han Lee, Myung-Ko Kyu Han, Dong Wook Ko, and II Hong Suh. From big to small: Multi-scale local planar guidance for monocular depth estimation. arXiv preprini vrXi::107.113266, 2019.
[20] Beibei Lin, Zifeng Yuan, and Tingting Chen. pol--ttiooiiztionnn estimation: A new task and benchmark study. arXiv preprint arXiv:2505. 133500, 2025.
[21] Guilin Liu, Fitsum A Reda, Kevin Shih, Ting-Chun Wang, Andrew Tao, and Bryan Catanzaro. Image inpainting for irregular holes using partial convolutions. In Proceedings of the European conference on computer vision (ECCV), pages 100, 2018.
[22] Shilong Liu, Zhanyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang. Qing Jiang, Chumyuan Li, Jianwei Yang, Hang Su, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection, In European Conference Computer Vision, pages 38-55. Springer, 2024.
[23] Shiyu Liu, Yucheng Han, Peng Xing, Fukun Yin, Rui Wang, Wei Cheng, Jiaqi Liao, Yingming Wang. Honghao Fu. Chunrui Han, et al. Steplx-edit: A practical framework for general image editing. arXiv preprint arx::22004. 17761, 2025.
[24] Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Lue Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11461-11471, 2022.
[25] Luca Medeiros. Language segment anything (lang-sam), 2023. Accessed: 20255-0-25.
[26] Ben Mildenhall, Pratul Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes neural radiance fields for view synthesis. Communications of the ACM, 65(1):99-106 2021.
[27] OpenAL. Chatgpt: Optimizing language models for dialogue. hatpss://openai.coom// httpi/aa...oooooo//gooeoeoo chattppt 2023. Accessed: Novemher 17, 2023.
[28] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748 8763, PMLR,
2021.
[29] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. ArXiv preprint, abs/2204.06125, 2022.
[30] René Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. Vision transformers for dense prediction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12179- 12179-12188, 12188, 2021.
[31] René Ranftl and Vladlen Koltun. Deep fundamental matrix estimation. In Proceedings of the European conference on computer vision (ECCV), pages 284-299, 2018
[32] Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, al. Sam Segment anything in images and videos. arXiv preprint arXiv:2408.00714 2024.
[33] Ahmad Salimi, Tristan Geometry-aware diffusion models for multiview scene inpainting. arXiv preprint urXiv:2502.13333 2025.
Marcus A Brubaker, and Konstantinos G Derpanis.
[34] Paul-Edouard Sarlin, Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. Superglue: Learning feature matching with graph neural networks. In Proceedings of the IEFFICVF conference on computer vision and pattern recognition, pages 4938-4947, 2020.
[35] Qi Shan, Brian Curless, Yasutaka Furukawa, Carlos Hernandez, and Steven M Seitz. Photo uncrop. In Computer Vision ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part VI 13, pages 16 31. Springer, 2014.
[36] Jingyu Shi, Achleshwar Luthra, Jiazhi Li, Xiang Gao, Xiyun Song, Zongfang Lin, David Gu, and Heather Yu. Occludenerf: Geometry-aware 3d scene inpainting with collaborative score distillation in nerf. In P Proceedings of the Computer Vision and Pattern Recognition Conference, pages 284-294, 2025.
[37] Stability AI.
Rablledifffuson-22 2-iffusion--nnaaintinn inpaaiinin
https///nggghgffccccoo/saaa cot.://aa.cccccttaiiiaa
stable- stable-diffuste diffusion--n2 2-inpaintinn 2022.

---

[38] Roman Suvorov, Elizaveta Logacheva, Anton Mashikhin, Anastasia Remizova, Arsenii Ashukha, Aleksei Silvestrov. Naejin Kong, Harshith Goka, Kiwoong Park, and Victor Lempitsky. Resolution-robus! large mask inpainting with fourier convolutions. Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 2149-2155 2022.
[39] Luming Tang, Menglin Jia, Qianqian Wang. Cheng Perng Phoo, and Bharath Hariharan. Emergent correspondence from image diffuusion. In Seirth-seventh Conference on Neural Information Processing Systems, 2023.
[40] Luming Tang, Nataniel Ruiz, Qinghao Chu, Yuanzhen Li, Aleksander Holynski, David E Jacobs, Bharath Hariharan, Yael Pritch, Neal Wadhwa, Kfir Aberman, et al. Realfill: Reference-drie driven generation for authentic image completion. ACM Transactions Graphics (TOG), 43(4):1 12,1 2024.
[41] Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny, Vggt: Visual geometry grounded transformer. arXiv preprint arXiv:2503. /651,, 2025.
[42] Yihang Wu, Xiao Cao, Kaixin Li, Zitan Chen, Haonan Wang. Lei Meng, and Zhiyong Huang. Towards better text-to-image generation alignment via attention modulation. In International Conference Neural Information Processing, pages 332-347. Springer, 2024.
[43] Shitao Xiao, Yueze Wang, Junjie Zhou, Huaying Yuan, Xingrun Xing, Ruiran Yan, Chaofan Li, Shuting Wang, Tiejun Huang, and Zheng Liu. Omnigen: Unified image generation. Proceedings of Computer Vision and Pattern Recognition Conference, pages 13294-13304, 2025.
[44] Binxin Yang, Shuyang Gu, Bo Zhang, Ting Zhang, Xuejin Chen, Xiaoyan Sun, Dong Chen, and Fang Wen. Paint by example: Exemplar-based image editing with diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18381-18391, 2023.
[45] Raymond A. Yeh, Chen Chen, Teck- Yian Lim, Alexander G. Schwing, Mark Hasegawa-Johnson, and Minh N. Do. Semantic image inpainting with deep generative models. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017, pages 6882-6890. IEEE Computer Society, 2017.
[46] Jiahui Zhang, Dawei Sun, Zixin Luo, Anhang Yao, Lei Zhou, tianwei Shen, Yurong Chen, Long Quan, and Hongen Liao. Learning two-view correspondences and geometry using order-aware network. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5845 5854, 2019.
[47] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable elfectiveness deep features as a perceptual metric. Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586-595, 2018.
[48] Yunhan Zhao, Connelly Barnes, Yuqian Zhou, Shechtman, Sohrab Amirghodsi, and Charless Fowlkes. Geofill: Reference-basee image inpainting with better geometric understanding. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1776-1786, 2023.
[49] Yuqian Zhou, Connelly Barnes, Eli Shechtman, and Sohrab Amirghodsi. Transfill: Reference-guider image inpainting by merging multiple color and spatial transformations. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages
2021.
12

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justifiicton right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No]", ", it is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: The claims in the abstract and introduction are consistent with the paper's actual contributions and scope, as detailed in Lines 7-17.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
13

---

Justification:
Limitations are discussed in the supplementary material.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approxinations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proof'
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification:
This paper does not include theoretical result.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: All implementation details, including are provided in Section 4.1 to ensure reproducibility of the main experimental results.
and training settings,
Guidelines:
14

---

The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: The code will be publicly available upon acceptance of this paper, along with instructions for reproducing the main experimental results.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ //nis.cc/public//uust http:///cc/giiee//eeeuuubiiiiiii
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
15

---

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Please refer section 4.1.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: We do not include error bars in this paper; instead, we report PSNR, SSIM, and LPIPS as evaluation metrics, which are widely used in the field.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula,
call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
Question:
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification: Please refer section 4.1.
Guidelines:
The answer NA means that the paper does not include experiments.
16

---

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
We have fully adhered to the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The potential societal impacts are provided in the supplementary material.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: This paper does not pose such risks.
17

---

Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All external assets properly credited, and their licenses and terms of use are respected, as detailed in Section 4.1.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website), the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, pacerswithcode thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is available online, the authors are encouraged to reach out the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We will provide detailed documentation and our code upon acceptance of this paper to ensure clarity and reproducibility.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember anonymize your assets (if applicable). You either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
18

---

Answer:
[NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing. editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: The core method development is this research does not involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non stanndaddrrd components.
Please refer to our LLM policy (https://neut) / for what should or should not be described.
19

---

Appendix
A Implementation Details
A.1 Detailed Clarification of Workflow
Initially, we process all reference images and the target image using VGGT and LangSAM to obtain the point cloud.
During training, given reference image Lee and its corresponding projected point cloud Lelond. we use our conditional reference masking to mask the reference image, obtaining the augmented reference image laug-re/ E IR H and the mask image mask E IR HxW We also use our conditional cloud masking to augment the projected point cloud, resulting in auug-dloud E R H xW We then encode these images into the latent space using VAE. This results latent reference image ref (used as the ground truth), a latent masked reference Tlotent and a latent projected point /8 and w W/8. The mask is also downsampled to
cloud platent
ang cloud all in [xxxxx where mask E Rixhxw.
platent
To fine-tune the diffusion model, we add noise to ground truth latent /eff to obtain a noisy latent noisy The input to the target branch is the concatenation of puteeti flteet and flatent -aug-ref resulting in tensor of shape R9xxxw Similarly, the input the cloud branch is the concatenation of I platent, noosy Latent and platent cloud also in 19xhx The objective is to estimate the added noise, which has shape 4xxxxx
During inference, given the target image har: its corresponding projected point cloud Ieloud: and the mask image mask, we directly process them into the latent space. This results in latent target platent and a latent projected point cloud L The mask is also downsampled to Iaash We initialize platent using standard Gaussian noise. Then, we concatenate the corresponding latent tensors construct the inputs for the target and cloud branches. After an iterative denoising process and using the VAE decoder, we obtain the final output.
A.2 Details of Baseline Methods
For SD Inpaint (37 and Generative Fill [2]. we follow the instructions from RealFill to generate long descriptions for each scene with the help of ChatGPT. For RealFill [40], we follow their official setting by fixing the text prompt to sentence containing rare token, i.e., "a photo of [V]". For fair comparison, our method also adopts this setting.
A.3 Robustness Evaluation Details
To evaluate robustness, we simulate three conditions that introduce errors from VGGT and LangSAM:
1. Noisy Point Cloud:
Gaussian noise is added to a subset of points in the generated 3D point cloud to mimic degraded geometry.
2. Sparse Point Cloud: projection to simulate sparse geometry.
A ratio of points is randomly dropped from the 3D point cloud before
3. Segmentation Errors:
We manually selected 13 RealBench scenes with significant dynamic objects and tested two variants: (1) removing LangSAM masks entirely and (2) introducing errors by randomly adding 10% extra masked regions per mask.
A.4 Prompt Design
we introduce LangSAM 25 22 32) to filter out dynamic content before applying VGGT, thereby enabling robust 3D attribute prediction even in dynamic scenes. Prompts can be provided manually or generated automatically by large language model (LLM) (I.
Since dynamic objects can significantly affect the geometric predictions VGGT 41
When using an L.M,, we upload all reference images along with the target image. The following guided prompt is used:
20

---

Table Comparison with existing enterence-guider image generation methods. Results are reported on different scene subsets (13 for StepIX -eppX-Edit,, 28 for OmniGen, and 28 for Bagel), PSNR and SSIM are reported.
13 scenes (Steplx-Edit)
28 scenes (OmniGen)
28 scenes (Bagel)
Method
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
StepIX-Edi
9.95
0.3678
OmniGen
8.93
0.3525
Bagel 15.75
10.83
0,4705
RealFill
0.5130
14.92
0,5156
14.92
0.5043
Ours
18.12
0.5869
17.37
0.5857
17.48
0.5827
Identify and list only the objects that are inconsistent across the images, such as dynamic objects that change position, appearance, or are missing. Ignore consistent background objects even if the viewpoint changes slightly.
B Comparison with Reference-Guided Image Editing Methods
In this section, we evaluate existing reference-guided image editing methods, including OmniGen [43] StepIx-Edit [23], and Bagel [I]]. We initially follow the official instructions to run these baseline models. For scenes where the models fail perform adequately, employ ChatGPT to generate prompts and manually refine them as needed. However, these methods fail to handle all scenes. some cases, the restored results become completely white visually meaningless. In summary, only 28 scenes from OmniGen, 13 scenes from StepIX-Eae E-Eiit, and 28 scenes from Bagel produce valid outputs. PSNR and SSIM are computed on these successfully restored results, as summarized in Table Overall, the results suggest that under the reference- based image completion setting, existing reference-guided image generation methods still perform suboptimally.
C Computational Cost
Table the computational cost (using four 24G GPUs) of Paint-by-Fxample RealFill [40] and our method. Both RealFill and our method are based on per-scene optimization. For fair comparison, we adopt identical experimental settings, including batch size, number of optimization steps, and number of GPUs. Although our approach introduces slightly higher overhead than RealFill, it achieves significantly better reconstruction quality. Notably, our method reaches promising results within 500 steps (18 mins), outperforming RealFill even at 2000 steps (50 mins).
44
Compared one-shot models such Paint-by Seint-by-Exammnn Example, per-scene optimization methods gener- ally provide more accurate content restoration but are less suitable for time-sensitive or real-time applications. As shown in Figure 4 and Figure our method produces more faithful results, while Paint-by-Example often fails to preserve fine-grained content from the reference images.
To explore potential acceleration strategies, in our framework lies in the 2,000-step per-scene fine-tuning process. One potential solution is to pre-train the LoRA parameters of the diffusion model large-scale, task-specific dataset. This would serve as a strong initialization for subsequent per-scene adaptation, thereby significantly reducing the number of required optimization steps while preserving the quality of generated results. We consider this an important direction for future work.
first identify that the primary computational bottleneck
D Limitations
While GeoComplete effectively leverages geometric cues for reference-driven image completion, it inherits certain limitations from its components. For example, the quality of the projected point cloud depends on the accuracy of the geometry estimation module (e.g., VGGT [1].. To mitigate the impact of inaccurate point clouds, introduce conditional cloud masking strategy that prevents
21

---

Table 5: Computational cost and performance comparison. training time. inference time, and PSNR (dB).
We report pre- pre-processing overhead,
Method
Pre-processing (one time)
Training Time
Inference
PSNR
Paint-by-Example
None
One-shot
~30
10.13
RealFill [40
(500 steps)
None
12 mins
88
13.67
RealFill 40
(2000 steps)
None
48 mins
~8
14.78
Ours (500 steps)
VGGT + LangSAM <30s
18 mins
155
16.33
Ours (2000 steps)
VGGT LangSAM <30s
72 mins
~ 155
17.32
the model from relying on unreliable geometric input. This allows our framework to generate realistic results even when the point cloud is inaccurate. However, when the point cloud is imprecise, the framework may not be able to fully exploit geometric information, which can affect completion quality those regions.
E
Societal Impact
Reference-driven image completion benefit various applications, including occlusion removal, image editing, and scene understanding in both consumer and industrial domains. GeoComplete introduces explicit geometric information to guide the completion process, reducing hallucination risks and improving structural fidelity. However, as the framework relies on generative models to synthesize missing content, it may still produce plausible yet inaccurate completions, especially in regions with limited geometric or visual cues. Therefore, we recommend caution when applying such methods in safety-critical forensic contexts that require guaranteed factual accuracy.
22


Reviews:
[{"review_id": "LUr0GjEfOr", "reviewer": "Reviewer_4jUZ", "summary": "This paper introduces GeoDiff, a framework for reference-driven image completion, a task that aims to fill missing regions in a target image using information from other views of the same scene. The authors identify a critical weakness in existing state-of-the-art generative methods, such as RealFill, which, despite their powerful image priors, often produce geometrically inconsistent or misaligned results due to a lack of explicit 3D awareness. GeoDiff proposes to remedy this by directly integrating 3D geometric information into the generative process. Empirically, the authors demonstrate substantial improvements over existing methods on the RealBench and QualBench datasets (which are also introduced in this paper).", "questions": "* Given the existence of several peer-reviewed works on geometry-aware generative in-painting from 2022-2025, could you please clarify and refine your paper's core novelty claim beyond being the \"first\" to couple geometry and diffusion for this task?\n\n* The framework relies heavily on the output of VGGT. Could you discuss the robustness of GeoDiff to potential failures in the geometry estimation stage? Specifically, how does the model perform when provided with a noisy or inaccurate point cloud, and have you considered adding an ablation study to demonstrate this?\n\n* Could you please expand on the evaluation setting for the baseline methods (e.g., how were the prompts chosen for SD, RealFill, etc)?", "limitations": "yes", "rating": 2, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 1, "clarity": 2, "significance": 2, "originality": 1, "strengths_and_weaknesses": "Strengths\n * The paper tackles an important problem. Methods like RealFill have pushed the state of the art by personalizing diffusion models on a few reference images, but their reliance on a 2D image prior without geometric understanding often leads to structural failures, hallucinations, and misalignments when there are significant viewpoint changes between the reference and target images. GeoDiff's central goal of enforcing explicit 3D geometric consistency is therefore not just an incremental improvement but a necessary and logical next step for building more robust, reliable, and authentic image completion tools. Solving this problem has direct applications in computational photography, virtual reality, and digital content creation.\n\n* Well-motivated technical approach that leverages VGGT and Lang-SAM pre-trained models. \n\n* Compelling empirical evaluation -- the paper presents a comprehensive set of experiments that, on the surface, convincingly demonstrate the method's effectiveness.\n\nWeaknesses:\n * Incomplete literature review. The failure to cite and discuss these works is not a minor oversight. It creates a cascade of problems: it allows for an inflated and inaccurate novelty claim, it prevents a meaningful comparison against the true state-of-the-art in geometry-aware generative inpainting, and it ultimately weakens the perceived contribution of the paper. Some important omissions include: \n   *  GeoFill (WACV 2023), explicitly estimates a 3D mesh and relative camera pose from two views to guide reference-based inpainting. While it does not use a diffusion model for the final synthesis, it establishes a strong and direct precedent for leveraging explicit 3D scene representation for this task.\n   * Geometry-Aware Diffusion Models for Multiview Scene Inpainting. It introduces a \"geometry-aware conditional diffusion model\" that uses projected geometric and appearance cues from reference images to perform multi-view consistent inpainting.\n   * OccludeNeRF focuses on 3D NeRF inpainting by guiding a diffusion model with collaborative score distillation. It explicitly reasons about geometry and occlusions to achieve 3D consistent results, representing another instance of combining 3D scene representations with diffusion models for completion tasks.\n\n* QualBench & RealBench - are not standard benchmarks. The paper introduces these two datasets, instead of re-using a dataset from a prior work, e.g., RealFill has one available. Generally, it's not a good idea to evaluate a new method on a new (not a well established) dataset.", "was_revised": false, "final_justification": "", "rebuttal": "### **[W1: Literature Review]**\nThank you for highlighting these relevant works. Below, we clarify the differences between our method and prior approaches:\n- GeoFill: We would like to clarify that our manuscript already discusses GeoFill [36] in lines 24–30 and 75–79. In particular, as noted in the RealFill [30] paper, GeoFill heavily relies on accurate 3D scene representations, making it sensitive to reconstruction errors. Moreover, RealFill highlights several limitations of GeoFill, including its difficulty in handling complex scene geometry, appearance changes, and scene deformation. In contrast, our method treats the 3D point cloud as an auxiliary input rather than a rigid intermediate representation. While the geometry cues from the point cloud may be imperfect, our model selectively leverages reliable information through a conditional cloud masking mechanism and a joint self-attention module. This design enhances robustness and enables more accurate and flexible image completion across diverse scenarios.\n\n- Geometric-aware 3D Scene Inpainter: This method conditions a diffusion model on multi-view images and scene geometry to reconstruct accurate 3D structure, which in turn supports high-quality inpainting. However, it assumes that all multi-view inputs share the same underlying scene geometry, which limits its applicability under challenging conditions such as dynamic objects or varying lighting. In contrast, our approach uses 3D geometry only as auxiliary guidance, rather than a reconstruction target. By leveraging the generative capabilities of the diffusion model, our method focuses on completing missing image regions with greater flexibility and robustness, even when the geometric cues are imperfect.\n\n- OccludeNeRF: OccludeNeRF distills knowledge from a pre-trained diffusion model for 3D scene inpainting using Score Distillation Sampling. In contrast, our method directly conditions the diffusion model on projected point clouds and target cues to complete missing regions. While both approaches incorporate geometry within a diffusion framework, the underlying mechanisms and task objectives are fundamentally different.\n\nWe note that Geometry-aware 3D Scene Inpainter (2025.02)  and OccludeNeRF (2025.04) are very recent arXiv preprints released around the same time as our submission. We will include the above discussion to clarify the distinctions between these works and ours in the revised version.\n \n---\n\n### **[W2: QualBench & RealBench]** \nWe would like to clarify that both QualBench and RealBench are not newly created datasets, but directly derived from the RealFill dataset [30], covering all evaluation data used in RealFill. We follow the same evaluation settings, including all datasets and metrics, as established in RealFill to ensure fair and consistent comparison.\n\n---\n\n### **[Q1: Contribution]** \nBeyond integrating diffusion with geometry, **our key innovations lie in the dual-branch diffusion architecture and the target-aware masking strategy, both specifically designed for reference-based image completion.**\n\nOur dual-branch diffusion comprises a target branch and a cloud branch. The target branch conditions the diffusion model on the masked image to generate missing content, while the cloud branch conditions it on the projected point cloud to provide geometric cues. A joint self-attention module is introduced to fuse information from both branches.\n\nThis design enables the model to adaptively integrate visual and geometric signals. However, since most regions in the target image are masked, the resulting latent features often lack meaningful content. While masked tokens can attend to the cloud branch, they may still struggle to extract effective guidance.\n\nTo address this, we modify the attention mask to explicitly link each masked token in the target branch to its corresponding token in the cloud branch, ensuring that the target branch receives direct geometric cues even in the absence of visual information. \n\nOur target-aware masking guides the model to focus on useful and non-redundant reference cues. It consists of two components: conditional reference masking and conditional cloud masking. Unlike RealFill [30], which randomly masks reference images, conditional reference masking selectively masks informative regions to encourage the model to learn from content that complements the target view. Conversely, conditional cloud masking randomly applies white padding to the projected point maps, masking redundant regions while preserving the informative ones, thereby guiding the model to effectively utilize geometric cues. Additionally, masking the projected point cloud informs the model that not all information from the point cloud is trustworthy. This prevents the model from over-relying on potentially noisy, sparse, or inaccurate geometry, thereby enhancing its robustness.\n\nAll these task-specific designs enable our method to selectively leverage geometric cues, even when they are imperfect. This distinguishes our approach from prior work, which often relies on highly accurate 3D reconstructions or treats geometry as a rigid intermediate representation.\n\n---\n\n### **[Q2: Robustness of GeoDiff]** \nWe agree that during both training and inference, the output of VGGT may contain inaccuracies, as noted in Lines 115–116.\n\nTo address this issue, we introduce a conditional cloud masking (CM) strategy (Lines 150–159) that prevents the model from over-relying on potentially noisy or erroneous geometry. During training, although the projected point cloud may already contain inaccuracies from VGGT, we further mask portions of it. With supervision from ground-truth RGB images, the model learns that not all information from point clouds is trustworthy, enabling it to adaptively interpret noisy geometry cues. \n\nIn addition, our joint self-attention module with masking (JSA) can potentially mitigate this issue. It establishes explicit token-to-token links between the target and cloud branches. This design prevents noisy cloud tokens from globally influencing all target tokens through cross-attention, especially when noisy tokens are more numerous. \n\nConsequently, at inference time, even when VGGT outputs are inaccurate, our model can still adaptively integrate information from both projected point clouds and RGB signals to produce reliable results.\n\nTo support our claim, we present the following experiments for verification.\n1) **Noisy Point Cloud Simulation:** To simulate degraded or poor-quality geometry, we randomly add Gaussian noise to a subset of points in the generated 3D point cloud. The experimental results are shown in Table A. Although the performance of our method slightly degrades with increasing noise levels, it consistently outperforms the baseline method RealFill across all settings. \n\n*Table A. PSNR under Different Ratios of Noisy Points in the Generated 3D Point Cloud.*\n| Method               |  Original   | 25% Noisy Points   | 50% Noisy Points    |  75% Noisy  Points  |\n|----------------------|-------------|----------------|----------------|----------------|\n| RealFill             | 14.78       | 14.78          | 14.78          | 14.78          |\n| Ours w/o. CM & JSA | 16.37       | 14.60  (↓1.77)  | 14.51 (↓1.86)  | 14.35 (↓2.02)  |\n| Ours                 | **17.32**   | 17.14  (↓0.18)  |  17.03 (↓0.29)  | 16.90  (↓0.42)  |\n\n2) **Sparse Point Cloud Simulation:** We randomly drop a certain ratio of points from the point cloud before projection to simulate sparse projected point clouds. The experimental results are in Table B. It can be observed that when the point cloud becomes sparse, even with only 25% of points dropped, the method without CM and JSA experiences a significant performance drop. In contrast, our full method remains more robust under these conditions. \n\n*Table B. PSNR under Different Drop Ratios of Points in the Generated 3D Point Cloud.*\n| Method               | Original   | Drop 25%  Points    | Drop 50%   Points    | Drop 75%    Points   |\n|----------------------|-------------|----------------|----------------|----------------|\n| RealFill             | 14.78       | 14.78          | 14.78          | 14.78          |\n| Ours w/o. CM & JSA | 16.37       | 14.58 (↓1.79)  | 14.50 (↓1.87)  | 14.35 (↓2.02)  |\n| Ours                 | **17.32**   | 17.18 (↓0.14)  | 16.83 (↓0.35)  | 16.50 (↓0.82)  |\n\n3) **Ablation Study on LangSAM:** We manually select 13 scenes with significant dynamic objects from RealBench to evaluate the robustness of our method. We conduct two experiments: (1) removing the Langsam masks entirely, and (2) simulating segmentation errors by randomly adding 10% additional masked regions to each Langsam mask. The results are shown below. When Langsam fails, our method exhibits only a slight drop of approximately 0.3 dB in PSNR. In contrast, the variant without CM and JSA shows a substantially larger drop of approximately 1.4 dB, highlighting the robustness of our full method to segmentation errors introduced by Langsam. \n\n*Table C. PSNR under Various Langsam Mask Conditions for Scenes with Challenging Dynamic Objects.*\n| Methods                   | w/. langsam  | w/o. langsam| langsam + random mask|\n|---------------------------|-------------|------------|------------|\n| RealFill                  | 14.44       | 14.44      |14.44|\n| Ours w/o. CM & JSA        | 15.92       | 14.54 (↓1.38)     |14.58 (↓1.34)|\n| Ours                      | 16.83       | 16.66  (↓0.17)    |16.51 (↓0.32)|\n\n---\n\n### **[Q3: Details of Baseline Methods]**\nFor SD Inpaint [27] and Generative Fill [2], we follow the instructions from RealFill, which generate long descriptions for each scene with the help of ChatGPT. For RealFill [30], we follow their official setting by fixing the text prompt to a sentence containing a rare token, i.e., “a photo of [V]”. For a fair comparison, our method also adopts this setting."}, {"review_id": "21noqPl2Wf", "reviewer": "Reviewer_vSDB", "summary": "This paper presents GeoDiff, a framework for reference-driven image completion that explicitly leverages 3D geometry to guide a diffusion-based generation process. The work targets a key limitation of existing generative methods: when there is a significant viewpoint discrepancy between the reference and target views, current approaches often yield geometrically inconsistent or misaligned results due to the absence of explicit 3D priors. The proposed method demonstrates impressive visual quality and delivers substantial improvements over prior state-of-the-art baselines, both quantitatively and qualitatively.", "questions": "While the paper presents promising results, I still have a few unresolved concerns. I would be willing to re-evaluate my score upon receiving satisfactory clarification from the authors.\n1. How robust is the method to errors from upstream modules such as LangSAM and VGGT?\nThe proposed pipeline depends on accurate segmentation and depth/pose estimation from external models. Have the authors evaluated how errors in these stages (e.g., incorrect masks or noisy point clouds) affect the final image completion results? Are there any mechanisms to mitigate error propagation?\n2. Can the authors elaborate on the computational cost and scalability of the approach?\nGiven that the method requires 2,000 iterations of per-scene fine-tuning in addition to inference from LangSAM and VGGT, how practical is the framework for large-scale or real-time applications? Have the authors explored any strategies for reducing the computational burden?\n3. Models such as OmniGen, Step1x-Edit, and Bagel also support reference-guided image generation and could serve as strong baselines. Could the authors clarify the rationale behind the choice of baselines, and possibly include comparisons with these more recent methods?", "limitations": "Please refer Questions for details.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n1. Novelty and Significance. A key strength of the paper lies in its novel integration of explicit 3D geometric priors into a diffusion-based framework for reference-guided image completion. This design directly addresses a major limitation of prior generative approaches—namely, the inability to maintain geometric consistency under large viewpoint changes—and achieves demonstrable improvements in geometric fidelity.\n\n2. Good results. The paper conducts comprehensive empirical validation, including both standard quantitative metrics (PSNR, SSIM, LPIPS, etc.) and a user study for qualitative assessment. The ablation studies are well-designed and effectively highlight the individual contributions of each model component.\n\n**Weaknesses**\n1. Dependency on Upstream Modules. The framework's reliance on external modules such as LangSAM (for segmentation) and VGGT (for depth and pose estimation) introduces a significant vulnerability. Inaccuracies in these modules can propagate through the pipeline and compromise the quality of geometric guidance, ultimately degrading the final output. However, the paper does not sufficiently analyze or quantify these potential failure modes.\n\n2. High Computational Cost.The proposed approach requires per-scene fine-tuning of the diffusion model over 2,000 iterations, alongside the overhead of running LangSAM and VGGT. This makes the method computationally demanding and limits its applicability in time-sensitive or real-time settings. A discussion on scalability or potential acceleration strategies would be beneficial.\n\n3. Limited Handling of Dynamic and Complex Scenes. The reliance on text-based prompts in LangSAM introduces limitations in dynamic or cluttered environments, especially when accurate or complete textual descriptions are unavailable. This restricts the method’s generalizability to complex, real-world scenes with multiple or moving objects.\n\n4. Lack of Comparison with Stronger Baselines. The experimental section omits comparison with several recent multi-modal generation methods, such as OmniGen[1], Step1x-Edit[2], and Bagel[3]. Including such baselines would provide a more complete picture of the method's relative advantages and limitations, especially since these models also support reference-conditioned generation in diverse settings.\n\nReferences:\n- [1] OmniGen: Unified Image Generation\n- [2] Step1X-Edit: A Practical Framework for General Image Editing\n- [3] Unified Model for Multimodal Understanding and Generation", "was_revised": true, "final_justification": "Most of my concerns have been addressed, and I will be increasing my rating accordingly. Additionally, I strongly encourage the authors to update their final draft based on the rebuttal (Computational Cost & Potential Baselines).", "rebuttal": "### **[W1/Q1: Robustness for Upstream Modules]** \nWe acknowledge that during both training and inference, the upstream modules (LangSAM and VGGT) may consequently generate inaccurate projected point clouds, as noted in Lines 115–116. \n\nTo address this issue, we introduce a conditional cloud masking (CM) strategy (Lines 150–159) that prevents the model from over-relying on potentially noisy or erroneous geometry. During training, although the projected point cloud may already contain inaccuracies from VGGT, we further mask portions of it. With supervision from ground-truth RGB images, the model learns that not all information from point clouds is trustworthy, enabling it to adaptively interpret noisy geometry cues. \n\nIn addition, our joint self-attention module with masking (JSA) can potentially mitigate this issue. It establishes explicit token-to-token links between the target and cloud branches. This design prevents noisy cloud tokens from globally influencing all target tokens through cross-attention, especially when noisy tokens are more numerous. \n\nConsequently, at inference time, even when VGGT or LandSAM outputs are inaccurate, our model can still adaptively integrate information from both projected point clouds and RGB signals to produce reliable results.\n\nTo support our claim, we present the following experiments for verification.\n1) **Noisy Point Cloud Simulation:** To simulate degraded or poor-quality geometry, we randomly add Gaussian noise to a subset of points in the generated 3D point cloud. The experimental results are shown in Table A. Although the performance of our method slightly degrades with increasing noise levels, it consistently outperforms the baseline method RealFill across all settings. \n\n*Table A. PSNR under Different Ratios of Noisy Points in the Generated 3D Point Cloud.*\n| Method               |  Original   | 25% Noisy Points   | 50% Noisy Points    |  75% Noisy  Points  |\n|----------------------|-------------|----------------|----------------|----------------|\n| RealFill             | 14.78       | 14.78          | 14.78          | 14.78          |\n| Ours w/o. CM & JSA | 16.37       | 14.60  (↓1.77)  | 14.51 (↓1.86)  | 14.35 (↓2.02)  |\n| Ours                 | **17.32**   | 17.14  (↓0.18)  |  17.03 (↓0.29)  | 16.90  (↓0.42)  |\n\n2) **Sparse Point Cloud Simulation:** We randomly drop a certain ratio of points from the point cloud before projection to simulate sparse projected point clouds. The experimental results are in Table B. It can be observed that when the point cloud becomes sparse, even with only 25% of points dropped, the method without CM and JSA experiences a significant performance drop. In contrast, our full method remains more robust under these conditions. \n\n*Table B. PSNR under Different Drop Ratios of Points in the Generated 3D Point Cloud.*\n| Method               | Original   | Drop 25%  Points    | Drop 50%   Points    | Drop 75%    Points   |\n|----------------------|-------------|----------------|----------------|----------------|\n| RealFill             | 14.78       | 14.78          | 14.78          | 14.78          |\n| Ours w/o. CM & JSA | 16.37       | 14.58 (↓1.79)  | 14.50 (↓1.87)  | 14.35 (↓2.02)  |\n| Ours                 | **17.32**   | 17.18 (↓0.14)  | 16.83 (↓0.35)  | 16.50 (↓0.82)  |\n\n3) **Ablation Study on LangSAM:** We manually select 13 scenes with significant dynamic objects from RealBench to evaluate the robustness of our method. We conduct two experiments: (1) removing the Langsam masks entirely, and (2) simulating segmentation errors by randomly adding 10% additional masked regions to each Langsam mask. The results are shown in Table C below. When Langsam fails, our method exhibits only a slight drop of approximately 0.3 dB in PSNR. In contrast, the variant without CM and JSA shows a substantially larger drop of approximately 1.4 dB, highlighting the robustness of our full method to segmentation errors introduced by Langsam. \n\n*Table C. PSNR under Various Langsam Mask Conditions for Scenes with Challenging Dynamic Objects.*\n| Methods                   | w/. langsam  | w/o. langsam| langsam + random mask|\n|---------------------------|-------------|------------|------------|\n| RealFill                  | 14.44       | 14.44      |14.44|\n| Ours w/o. CM & JSA        | 15.92       | 14.54 (↓1.38)     |14.58 (↓1.34)|\n| Ours                      | 16.83       | 16.66  (↓0.17)    |16.51 (↓0.32)|\n\n---\n\n### **[W2/Q2: Computational Cost and Potential Acceleration Strategies]** \nThe Table D below summarizes the computational cost (using four 24G GPUs) of Paint-by-Example [32], RealFill [30], and our method. Both RealFill and our method are based on per-scene optimization. For a fair comparison, we adopt identical experimental settings, including batch size, number of optimization steps, and number of GPUs. Although our approach introduces slightly higher overhead than RealFill, it achieves significantly better reconstruction quality. Notably, our method reaches promising results within 500 steps (18 mins), outperforming RealFill even at 2000 steps (50 mins).\n\nCompared to one-shot models such as Paint-by-Example, per-scene optimization methods generally provide more accurate content restoration but are less suitable for time-sensitive or real-time applications. As shown in Figures 4 and 5, our method produces more faithful results, while Paint-by-Example often fails to preserve fine-grained content from the reference images.\n\nTo explore potential acceleration strategies, we first identify that the primary computational bottleneck in our framework lies in the 2,000-step per-scene fine-tuning process. One potential solution is to pre-train the LoRA parameters of the diffusion model on a large-scale, task-specific dataset. This would serve as a strong initialization for subsequent per-scene adaptation, thereby significantly reducing the number of required optimization steps while preserving the quality of generated results. We consider this an important direction for future work.\n\n*Table D. Computational Cost and Performance Comparison.*\n| Method    | Pre-processing (one time)        | Training Time | Inference | PSNR    |\n|-----------|------------------------|----------------|-----------|---------|\n| Paint-by-Example  | None           | One-shot       | ～30s        | 10.13   |\n| RealFill (500 steps)  | None                   | 12 mins        | ～8s        | 13.67 \t    |      \n| RealFill (2000 steps) | None                   | 48 mins        | ～8s        | 14.78   |\n| Ours  (500 steps)    | VGGT + LangSAM < 30s   | 18 mins        | ～15s        | 16.33 |\n| Ours  (2000 steps)    | VGGT + LangSAM < 30s   | 72 mins        | ～15s        | **17.32** |\n\n---\n\n###  **[W3: Dynamic and Complex Scenes]** \nWe acknowledge that LangSAM may struggle in dynamic or cluttered environments, especially when accurate or complete textual prompts are unavailable. These limitations can affect the quality of the generated masks and, consequently, the reconstructed point clouds. However, our method treats the 3D point cloud as an auxiliary input rather than a rigid intermediate representation. While the geometric cues from the point cloud may be imperfect, our model selectively leverages reliable information through a conditional cloud masking mechanism and a joint self-attention module. This enables the model to focus on trustworthy regions while ignoring potentially misleading geometry.\n\nMoreover, we would like to emphasize that the benchmark datasets RealBench and QualBench encompass a wide range of complex scenes, including significant dynamic content, multiple moving objects, and varying lighting conditions. The corresponding experimental results are presented in the Table C. These results demonstrate the strong robustness and generalization ability of our method, even in dynamic and complex scenarios.\n\n---\n\n### **[W4/Q3.1: Potential Baselines]** \nFollowing the suggestion, we evaluate existing reference-guided image generation methods, including OmniGen, Step1x-Edit, and Bagel. We initially follow the official instructions to run these baseline models. For scenes where the models fail to perform adequately, we employ ChatGPT to generate prompts and manually refine them as needed. However, these methods fail to handle all scenes. In some cases, the restored results become completely white or visually meaningless. In summary, only 28 scenes from OmniGen, 13 scenes from Step1X-Edit, and 28 scenes from Bagel produce valid outputs. PSNR and SSIM are computed only on these successfully restored results. The results suggest that under the reference-based image completion setting, existing reference-guided image generation methods still perform suboptimally. \n\n| 13 scenes  | PSNR  | SSIM   | |  28 scenes  | PSNR  | SSIM   |\n|-----------|-------|--------| --| ----------|-------|--------|\n| Step1X-Edit    | 9.95  | 0.3678 |  | OmniGen   | 8.93  | 0.3525 |\n| RealFill  | 15.75 | 0.5130 |  | RealFill  | 14.92 | 0.5156 |\n| Ours      | 18.12 | 0.5869 |  | Ours      | 17.37 | 0.5857 |\n\n| 28 scenes | PSNR  | SSIM   |\n|-----------|-------|--------|\n| Bagel     | 10.83 | 0.4705 |\n| RealFill  | 14.92 | 0.5043 |\n| Ours      | 17.48 | 0.5827 |\n\n---\n\n### **[Q3.2: Rationale of Baselines]** \nOur baseline methods are primarily selected from the RealFill paper [30], which serves as a strong and recent benchmark in this field. Specifically, we include prompt-based approaches such as SD Inpaint [27] and Generative Fill [2], as well as reference-based completion methods like Paint-by-Example [32], TransFill [37], and RealFill [30]. Notably, both TransFill and RealFill are trained on dedicated image completion datasets, making them particularly suitable for comparison. Overall, these baselines constitute a representative and diverse set of state-of-the-art techniques, enabling fair and meaningful evaluation."}, {"review_id": "DvjwPkQBjp", "reviewer": "Reviewer_VDvq", "summary": "The authors propose a new method for image completion (inpainting, outpainting, etc.) given (a) an image to be completed and (b) several other reference views of the scene, from different viewpoints. This longstanding problem in computer vision has been worked on even since before the days of DNNs, and work continues into the diffusion age. This paper suggests using a state-of-the-art point cloud method to do (sparse) novel viewpoint generation from the reference images, then trains an image completer that can consume these projected pointclouds. They make experimental comparisons to RealFill and Paint-By-Example.", "questions": "How do comparisons with Transfill work?\n\nHow does the model degrade, as compared to other methods' degradation, as there is less overlap, i.e., the pointcloud covers less and less of the image and the model is forced to do more and more whole-cloth extrapolation?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Quality: Qualitatively and quantitatively the results are convincing. It seems that TransFill is a more apt comparison than PBE, especially given the paper’s claims about the superiority of one-shot geometry (lines 27-30), so I would prefer for the paper to include some qualitative comparisons here, especially ones that show clearly why pointclouds are a better 3D representation than multiple homographies.\n\nClarity: the paper is fairly readable and appropriately concise. This reviewer was a little bit confused about how the conditioning was added to the diffusion model; it looks at first like the conditioning itself has noise added; it’s hard to see how this could work at inference time since the noise, at least in traditional diffusion levels, would destroy all the conditioning signal! At first I thought it might be some kind of CFM variant  – but now I think the latents of the conditioning are concatenated onto the noise (lines 167-168).\n\nOriginality and significance: this is a novel approach that seems to improve the SOTA on a long-standing and well-established vision problem.\n\nI note that the authors promise to release code and training scripts, which is great from a reproducibility standpoint.\n\n(edited to add:) the authors may also be interested in comparing with this work: https://arxiv.org/pdf/2502.13335 (I believe the work to be roughly simultaneous).", "was_revised": true, "final_justification": "I remain in support of accepting this paper; I'm happy that the authors will address some less-than-clear parts of the paper.", "rebuttal": "### **[W1: One-shot Geometry vs. Multiple Homographies]** \nThe limitations of multiple homographies have been previously identified and evidenced by RealFill [30], which demonstrates catastrophic failure when the scene’s structure cannot be accurately estimated. In contrast, extensive experiments in VGGT validate the superiority of one-shot geometry estimation. Due to this year’s rebuttal policy, we are unable to include additional comparison figures. However, some TransFill results are already available in the RealFill paper [30]. We also commit to including direct qualitative comparisons with TransFill in the revised version.\n\n---\n\n### **[W2: Clarification of Workflow]** \nInitially, we process all reference images and the target image using VGGT and Langsam to obtain the point cloud. \n    \nDuring training, given a reference image $I_{ref}$ and its corresponding projected point cloud $I_{cloud}$, we use our conditional reference masking to mask the reference image, obtaining the augmented reference image $I_{aug-ref} \\in \\mathbb{R}^{3 \\times H \\times W}$ and the mask image $I_{mask} \\in \\mathbb{R}^{1 \\times H \\times W}$. We also use our conditional cloud masking to augment the projected point cloud, resulting in $I_{aug-cloud} \\in \\mathbb{R}^{3 \\times H \\times W}$. We then encode these images into the latent space using a VAE. This results in a latent reference image $I_{ref}^{latent}$ (used as the ground truth), a latent masked reference $I_{aug-ref}^{latent}$, and a latent projected point cloud $I_{aug-cloud}^{latent}$, all in $\\mathbb{R}^{4 \\times h \\times w}$, where $h = H / 8$ and $w = W / 8$. The mask is also downsampled to $I_{mask}^{latent} \\in \\mathbb{R}^{1 \\times h \\times w}$.\n\n    \nTo fine-tune the diffusion model, we add noise to the ground truth latent $I_{ref}^{latent}$ to obtain a noisy latent $I_{noisy}^{latent}$. The input to the target branch is the concatenation of $I_{noisy}^{latent}$, $I_{mask}^{latent}$, and $I_{aug-ref}^{latent}$, resulting in a tensor of shape $\\mathbb{R}^{9 \\times h \\times w}$. Similarly, the input to the cloud branch is the concatenation of $I_{noisy}^{latent}$, $I_{mask}^{latent}$, and $I_{aug-cloud}^{latent}$, also in $\\mathbb{R}^{9 \\times h \\times w}$. The objective is to estimate the added noise, which has shape $\\mathbb{R}^{4 \\times h \\times w}$.\n\nDuring inference, given the target image $I_{tar}$, its corresponding projected point cloud $I_{cloud}$, and the mask image $I_{mask}$, we directly process them into the latent space. This results in a latent target $I_{tar}^{latent}$ and a latent projected point cloud $I_{cloud}^{latent}$. The mask is also downsampled to $I_{mask}^{latent}$. We initialize $I_{noisy}^{latent}$ using standard Gaussian noise. Then, we concatenate the corresponding latent tensors to construct the inputs for the target and cloud branches. After an iterative denoising process and using the VAE decoder, we obtain the final output.\n\n\n\n\n\n---\n\n### **[W3: Comparison with Recent Work]**\nFollowing the suggestion, we provide a discussion comparing our method with Geometric-aware 3D Scene Inpainter [R1]. This method conditions a diffusion model on multi-view images and scene geometry to reconstruct accurate 3D structure, which in turn supports high-quality inpainting. However, it assumes that all multi-view inputs share the same underlying scene geometry, which limits its applicability under challenging conditions such as dynamic objects or varying lighting. In contrast, our approach uses 3D geometry only as auxiliary guidance, rather than a reconstruction target. By leveraging the generative capabilities of the diffusion model, our method focuses on completing missing image regions with greater flexibility and robustness, even when the geometric cues are imperfect.\n\n[R1] Salimi, Ahmad, et al. \"Geometry-aware diffusion models for multiview scene inpainting.\" arXiv preprint arXiv:2502.13335 (2025).\n\n---\n\n### **[Q1: Clarification of TransFill [37]]** \nThe quantitative results of TransFill are directly taken from the RealFill [30] paper, and we follow the same evaluation setting to ensure a fair comparison. Since the official code of TransFill is not publicly available, we, like the RealFill team, contacted the authors to obtain the qualitative results.\n\n---\n\n### **[Q2: Robustness of GeoDiff]** \n\nFollowing the suggestions, we conduct ablation studies on the scenario with reduced overlap. As the point cloud covers less of the image, the number of points decreases, leading to reduced pixel information in the projected point cloud. To simulate this scenario, we perform the following experiments:\n\n1) **Pixel Drop in the Projected Point Cloud:** We randomly drop pixels within the target regions of the projected point cloud, directly reducing the available point cloud information. The experimental results are presented in Table A.\n\n2) **Region Drop in the Projected Point Cloud:** We randomly remove multiple rectangular blocks within the target regions of the projected point cloud until the total number of removed points reaches a fixed ratio of the points in these regions. This block-based removal simulates region-level information loss. The experimental results are presented in Table B.\n    \n3) **Sparse Point Cloud Simulation:** We randomly drop a certain ratio of points from the point cloud before projection to simulate sparse projected point clouds. The experimental results are in Table C. \n\nIt can be observed that our method remains more robust under these conditions, consistently outperforming the baseline method RealFill. This robustness arises from the fact that our method does not heavily rely on the projected point cloud for image completion. Instead, we introduce two key components: Conditional Cloud Masking (CM) and the Joint Self-Attention (JSA) module with masking, which are designed to prevent the model from over-relying on potentially noisy or inaccurate geometric information.\n\nDuring training, although the projected point cloud may already contain inaccuracies (e.g., due to VGGT), the conditional cloud masking module further masks portions of it. Supervised by ground-truth RGB images, the model learns that not all point cloud information is reliable, allowing it to adaptively interpret and utilize geometric cues under challenging conditions.\n\nIn addition, our joint self-attention module with masking (JSA) can potentially mitigate this issue. It establishes explicit token-to-token links between the target and cloud branches. This design prevents noisy cloud tokens from globally influencing all target tokens through cross-attention, especially when noisy tokens are more numerous. \n\nFurthermore, in Tables A, B, and C, we present the ablation studies for our CM and JSA modules. When the point cloud or projected pixels become sparse, even with only 25% of the points dropped, the variant without CM and JSA exhibits a significant performance drop. In contrast, our full method remains robust under these conditions.\n     \n*Table A. PSNR under Different Drop Ratios of Pixels in the Projected Point Cloud.*\n\n| Method               | Original   |  Drop 25%  Pixels    | Drop 50%   Pixels    | Drop 75%    Pixels   |\n|----------------------|--------------|-------------|-------------|-------------|\n| RealFill             | 14.78       | 14.78       | 14.78       | 14.78       |\n| Ours w/o. CM & JSA | 16.37       | 14.55 (↓1.82)      | 14.45 (↓1.92)      | 14.29 (↓2.08)       |\n| Ours           | **17.32**    | 16.85 (↓0.47)      | 16.51 (↓0.81)       | 15.76 (↓1.56)      |\n\n*Table B. PSNR under Different Drop Ratios of Regions in the Projected Point Cloud.*\n| Method               | Original   |  Drop 25%  Regions    | Drop 50%   Regions    | Drop 75%    Regions   |\n|----------------------|--------------|-------------|-------------|-------------|\n| RealFill             | 14.78       | 14.78       | 14.78       | 14.78       |\n| Ours w/o. CM & JSA | 16.37       | 14.67 (↓1.7)      | 14.65 (↓1.72)      | 14.56  (↓1.81)       |\n| Ours           | **17.32**    | 17.08 (↓0.24)      | 16.83 (↓0.49)       | 16.61  (↓0.71)      |\n\n*Table C. PSNR under Different Drop Ratios of Points in the Generated 3D Point Cloud.*\n| Method               | Original   | Drop 25%  Points    | Drop 50%   Points    | Drop 75%    Points   |\n|----------------------|-------------|----------------|----------------|----------------|\n| RealFill             | 14.78       | 14.78          | 14.78          | 14.78          |\n| Ours w/o. CM & JSA | 16.37       | 14.58 (↓1.79)  | 14.50 (↓1.87)  | 14.35 (↓2.02)  |\n| Ours                 | **17.32**   | 17.18 (↓0.14)  | 16.83 (↓0.35)  | 16.50 (↓0.82)  |"}, {"review_id": "bRmY2DFyYU", "reviewer": "Reviewer_qNjF", "summary": "This paper introduces a reference-guided image outpainting method by introducting feed-forward 3D foundation models to faciliate 2D tasks. It predicts geometric-aware outpainted regions through conditioning on the a 3D point cloud, and finetunes or use off-the-shelf diffusion models to fill in the missing region. It is an interesting and reasonable idea in itself that has demonstrated good results over the baseline. The evaluation on quality is sufficient.", "questions": "How would a language model at all be able to tell what is the dynamic region of the image? I don't think the paper clearly defines it either. Considering the example on the first page, the cat was not in the reference images, how is it considered by the language model then? \n\nHow is the runtime of this model compare to baseline and especially RealFill?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "In general the idea makes sense, as filling in a 2d capture of a 3d scene with the reference from the 3D content will naturally outperform pure 2d based models in terms of the persective of the surroundings. My only concern is the runtime overload this method, which I believe should be heavier than 2D based methods which only requires model finetuning. It would be nice if the author can acknowledge this in the limitation or provide justification otherwise.", "was_revised": true, "final_justification": "I keep my initial rating for the paper as borderline accept. The rebuttal provided by the authors addressed my concerns sufficiently.", "rebuttal": "### **[W1/Q2: Runtime Comparisons]**\nThe Table A below summarizes the computational cost (using four 24G GPUs) of Paint-by-Example [32], RealFill [30], and our method. Both RealFill and our method are based on per-scene optimization. For a fair comparison, we adopt identical experimental settings, including batch size, number of optimization steps, and number of GPUs. Although our approach introduces slightly higher overhead than RealFill, it achieves significantly better reconstruction quality. Notably, our method reaches promising results within 500 steps (18 minutes), outperforming RealFill even at 2000 steps (50 minutes).\n\nCompared to one-shot models such as Paint-by-Example, per-scene optimization methods generally provide more accurate content restoration but are less suitable for time-sensitive or real-time applications. As shown in Figures 4 and 5, our method produces more faithful results by leveraging scene-specific optimization, whereas Paint-by-Example often fails to transfer fine-grained content from the reference images due to the absence of such tuning.\n\n*Table A. Computational Cost and Performance Comparison.*\n| Method    | Pre-processing (one time)        | Training Time | Inference | PSNR    |\n|-----------|------------------------|----------------|-----------|---------|\n| Paint-by-Example  | None           | One-shot       | ～30s        | 10.13   |\n| RealFill (500 steps)  | None                   | 12 mins        | ～8s        | 13.67 \t    |      \n| RealFill (2000 steps) | None                   | 48 mins        | ～8s        | 14.78   |\n| Ours  (500 steps)    | VGGT + LangSAM < 30s   | 18 mins        | ～15s        | 16.33 |\n| Ours  (2000 steps)    | VGGT + LangSAM < 30s   | 72 mins        | ～15s        | **17.32** |\n\n---\n\n### **[Q1: Clarification on Dynamic Regions]** \nTo identify dynamic regions, we use an LLM (GPT-4o) with the target image and all reference images as input, prompting it to detect any dynamic content. The exact prompt used is provided in our supplementary material (Lines 8–11).\n\nWe acknowledge that LangSAM may fail, particularly in highly complex scenes. Such failures can degrade the quality of the generated masks and, consequently, the reconstructed point clouds. However, our method does not heavily rely on the projected point cloud for image completion. Instead, we introduce two key components: Conditional Cloud Masking (CM) and the Joint Self-Attention (JSA) module with masking, which are designed to prevent the model from over-relying on potentially noisy or inaccurate geometric information.\n\nDuring training, although the projected point cloud may already contain inaccuracies (e.g., due to VGGT), the conditional cloud masking module further masks portions of it. Supervised by ground-truth RGB images, the model learns that not all point cloud information is reliable, allowing it to adaptively interpret and utilize geometric cues under challenging conditions.\n\nIn addition, our joint self-attention module with masking (JSA) can potentially mitigate this issue. It establishes explicit token-to-token links between the target and cloud branches. This design prevents noisy cloud tokens from globally influencing all target tokens through cross-attention, especially when noisy tokens are more numerous. \n\nTo support our claim, we present the following experiments for verification. We manually select 13 scenes with significant dynamic objects from RealBench to evaluate the robustness of our method. We conduct two experiments: (1) removing the Langsam masks entirely, and (2) simulating segmentation errors by randomly adding 10% additional masked regions to each Langsam mask. The results are shown in Table B. When Langsam fails, our method exhibits only a slight drop of approximately 0.3 dB in PSNR. In contrast, the variant without CM and JSA shows a substantially larger drop of approximately 1.4 dB, highlighting the robustness of our full method to segmentation errors introduced by Langsam. \n\n*Table B. PSNR under Different Langsam Mask Conditions on Dynamic Scenes.*\n| Methods                   | w/. langsam  | w/o. langsam| langsam + random mask|\n|---------------------------|-------------|------------|------------|\n| RealFill                  | 14.44       | 14.44      |14.44|\n| Ours w/o. CM & JSA        | 15.92       | 14.54 (↓1.38)     |14.58 (↓1.34)|\n| Ours                      | 16.83       | 16.66  (↓0.17)    |16.51 (↓0.32)|"}]

Author rebuttal:

Meta-review: This paper introduces GeoDiff, a geometry-aware diffusion model designed for reference-driven image completion. The key contribution lies in explicitly incorporating geometric priors—via structure-aware representations and consistency constraints—into the diffusion process, thereby improving the alignment of generated content with the provided reference image. The problem is important for image synthesis and completion, where preserving geometry is crucial, and the proposed method is technically sound and well-motivated.

Reviewers highlighted several strengths. The approach is novel and impactful, addressing a long-standing limitation in diffusion-based image completion by embedding geometric awareness. Empirical results demonstrate consistent improvements over existing baselines, with both qualitative and quantitative evidence supporting the claims. The paper is also clearly written and well-organized, allowing readers to follow the pipeline and contributions effectively.

While some concerns were raised, they were generally minor and have been well addressed during the rebuttal. Reviewer 4jUZ did not participate in any rebuttal discussions, despite multiple reminders from both the AC and the authors. The initial score of 3 is primarily attributed to relatively minor issues in the literature review and benchmark. Such reasoning does not provide a sufficiently convincing basis for a fair overall recommendation. The other reviewers agreed that these points were not critical. Therefore, ACs believe the reviews from Reviewer 4jUZ can be disregarded, and such irresponsible behavior should be flagged.

Overall, this is a timely contribution that advances the state-of-the-art in geometry-aware generative modeling. The novelty, solid empirical performance, and potential impact in downstream tasks such as editing and content creation justify acceptance.
Decision: Accept (poster)