Aligned Novel View Image and Geometry Synthesis via Cross-modal Attention Instillation
Authors: Min-Seop Kwak
Junho Kim
Sangdoo Yun
Dongyoon Han
Taekyung Kim
Seungryong Kim
Jin-Hwa Kim
Venue: ICLR 2026 Poster (2026)
Keywords: Novel view synthesis
diffusion model
Primary area: generative models
TL;DR: Cross-modal Attention Instillation for Aligned Novel View Image and Geometry Synthesis

Abstract: We introduce a diffusion-based framework that generates aligned novel view images and geometries via a warping‐and‐inpainting methodology. Unlike prior methods that require dense posed images or pose-embedded generative models limited to in‐domain views, our method leverages off‐the‐shelf geometry predictors to predict partial geometries viewed from reference images, and formulates novel view synthesis as an inpainting task for both image and geometry. To ensure accurate alignment between the generated image and geometry, we propose cross-modal attention instillation where the attention maps from an image diffusion branch are injected into a parallel geometry diffusion branch during both training and inference. This multi-task approach achieves synergistic effects, facilitating both geometrically robust image synthesis and geometry prediction. We further introduce proximity‐based mesh conditioning to integrate depth and normal cues, interpolating between point cloud and filtering erroneously predicted geometry from influencing the generation process. Empirically, our method achieves high-fidelity extrapolative view synthesis, delivers competitive reconstruction under interpolation settings, and produces geometrically aligned point clouds as 3D completion.

Published as a conference paper at ICLR 2026
ALIGNED NOVEL VIEW IMAGE AND GEOMETRY SYN- THESIS VIA CROSS-MODAL ATTENTION INSTILLATION
Min-Scop Kwak!* Taekyung Kim?
Junho Kim?
Sangdoo Yun
Dongyoon Han? Jin-Hwa Kim
Seungryong Kim 1t
KAIST AI
NAVER Al Lab
SNU AIIS
ABSTRACT
We introduce a diffusion-based framework that generates aligned novel view im- ages and geometries via a aapping-and-inpaittin methodology. Unlike prior methods that require dense posed images or pose-embedded generative models limited to in-domain views, our method leverages off-the-shelf geometry predic- to predict partial geometries viewed from reference images, and formulates novel view synthesis as an inpainting task for both image and geometry. To en- sure accurate alignment between the generated image and geometry, we propose cross-Modal Attention Instillation (MoAI) where the attention maps from an im- age diffusion branch are injected into parallel geometry diffusion branch during both training and inference. This multi-task approach achieves synergistic effects, facilitating both geometrically robust image synthesis and geometry prediction. We further introduce proximity-based mesh conditioning to reduce erroneous projections and to integrate depth and normal cues to the correspondence condi- tions. Empirically, method achieves high-fidelitg extrapolative view synthesis, delivers competitive reconstruction under interpolation settings, and produces ge- ometrically aligned point clouds 3D completion. Project page is available at https: //c wwwllabbkaiss. www.tb.gt.h............ thub.. M
Reference Images
Projected
Novel View Image
RGB
[Unposed]
Completed 3D Geometry
Impainting
Syrthesized Goometry
Pediiction
Don-moda Astention
proper
Aligned
Geametry
Geametry Campletion
Tarpet Carnera Varapennt
Projected
Novel View
Reference Camera Wesapoint
Geomtry
Figure 1: Overview our diffusion- based framework. From one or more unposed reference images, predict partial colored point cloud and project it the target view. Our diffusion model then inpaints missing regions with the cross-Modal Attention Instillation (MoAI), ensuring alignment between image and geometry, resulting in complete 3D scene.
INTRODUCTION
Novel view synthesis (NVS), the task of reconstructing 3D scenes from sparse reference images, represents fundamental challenge that requires neural networks to understand and model the under- lying 3D structure a scene from limited 2D observations. Seminal works such as NeRF (Mildenhall
et al. (021) and 3DGS (Kerbl et al. 2023) implicitly model volumetric geometry and appearance by optimizing scene-specine representations to fit individual 3D scenes from reference images. To
generalize beyond single-scene optimization, feedforward methods (Wang et al.) 2024 Charatan
Work done during internship at NAVER Lab..
Co-corresponding authors.

---

Published as a conference paper at ICLR 2026
al.
2024
Chen et al.
2024 Ye et al.
2024)
have emerged for direct 3D prediction. In addition, the
advent of diftusion models (Rombach et al.
2022) has introduced generative NVS methods (Gao
et al.
2024 Sco et al.
2024 Ren et al.
2025 that achieve remarkable fidelity in NVS.
However, significant challenges remain. Feedforward methods (Wang et al. 2024 Charatan et al. Chen
2024| Chen et al. [024 Ye et al.] 2024) show high fidelity in interpolative settings by filling the regions visible in reference images, but they lack extrapolation capabilities for synthesizing occluded or unseen areas. Conversely, generative NVS methods (Gao al., 2024 Seo et al.) 2024 Ren et al.) 2025), trained with known camera poses, can extrapolate beyond the reference views. However, when conditioned on camera viewpoints underrepresented during training, these models often produce erroneous novel views (Voleti et al., 2024). Consequently, these methods require known reference camera poses, limiting them to the posed NVS setting.
Building on sarppin--and-ipaiiniinn methods (Chung et al. 2023 Seo et al. 2024), we propose an alternative framework for multi-view NVS. This framework leverages an off-the-shelf geometry prediction model (Ke et al.) 2024] Wang et al.] 2024] 2025) to estimate geometry and camera pose from reference images, project this geometry to target viewpoint, and guide the generative process for plausible NVS. More specifically, we predict geometry from multiple reference views, aggregate and project them to a novel viewpoint, and use this coarse geometric conditioning to guide spatial coss-attention in diffusion networks. By framing NVS approach synthesizes novel views at arbitrary viewpoints from unposed reference images, while also supporting reconstruction and generation at extrapolative viewpoints. Additionally, method to novel view geometry synthesis by training a geometry denoising U-Net that inpaints the target geometry from reference views. This offers a key advantage: unlike prior depth-prediction methods Chung et al. (202)); Seo et al. (2024) that suffer from a scale-shift discrepancy between predicted and known reference geometry (Ke et al.) (024). our method ensures geometric alignment by generating as a continuation of the reference geometry.
inpainting problem,
extend this
To encourage synergistic multi-task learning across image and depth modalities, ensuring their align- ment, we introduce cross-Modal Attention Instillation, shortened as MoAI, where spatial attention maps from the image denoising network, which implicitly capture cross-view correspondences, are instilled to replace those of the geometry network during both training and inference. More specifi- cally, our design enables to learn from relatively robust and consistent geometry completion task to regularize image generation through instilled attention map, while the geometry network leverages rich semantic features from images to improve synthesis quality. Additionally, our proximity- based mesh conditioning incorporates additional geometric cues (depth and normal) into correspondence conditions, interpolating sparse geometry and filtering of erroneous projections.
Our method demonstrates strong extrapolation capabilities for both novel view image and geometry synthesis, resulting in aligned colored point clouds that achieve 3D scene completion (Fig. []] achieves state-of-the-art performance in extrapolative settings, while maintaining competitive reconstruction and zero-shot generalization to unseen data.
RELATED WORK
Non-generative sho-shot NVS. Neural 3D representations such as NeRF (Mildenhall al. 2021 and 3DGS (Kerbl et al. 2023) require numerous calibrated views to optimize the neural radiance field effectively. Optimization-
few-shot methods (Jain et al., 202T Niemeyer et al. 2022, Kim et al.
2022, Kwak et al. 2023) alleviate this issue by tailoring single 3D scene from sparse views. For instance, DietNeRF (Jain al. (02T) enforces semantic consistency between rendered images from novel viewpoints and available reference images, while RegNeRF (Niemeyer et al. 2022) regularizes the geometry and appearance of patches from unobserved viewpoints. However, these approaches are unable to generalize beyond individual scenes and are computationally expensive.
Feedforward NVS approaches (Yu et al. 2021 Chen et al. 2024, Wang et al. 2024 Ye et al. 2024 Hong et al., 2023) address few-shot novel view synthesis without per-scene optimization. PixeINeRF (Yu et al. 2021) is among the first to condition a NeRF on image inputs using local CNN features, predicting novel view image in feedforward manner. MVSplat (Chen al. 2024) improves on this by predicting 3D Gaussians from sparse multi-view images with a cost volume for depth estimation, yielding high-quality 3D representations. Subsequent works (Wang al.
2

---

Published as a conference paper at ICLR 2026
Munliie Features
Reference Images
Image Generation
image
Aggregated Actentann
Sec
Q
image
Projected
Image
Targot View Projeces lection
Dencisiing
1488 Loid
Geometry
Projested
Cross-modal Attention Instilaaton
Prediction
Pointmap
Geometry Generation
Geametry Dancising Network
Pontimap
VP
Sec 3.2
K2
Multivice Aggregated Atzension
Guomedtty Metworkk
Imaage
Safherice Cimpppideens Geld
Paintmaps
Figure 2: Training methodology. Our method conducts cross-moda attention instillation, replacing the spatial attention maps of geometry denoising networks with those of image denoising networks, so that the image generation U-Net learns a more robust representation aligned with the geometry completion task. On the other hand, the geometry prediction networks leverage the rich semantics from image features to enhance geometry completion capability.
2024 Leroy et al. 2024 Hong et al. 2023 Ye et al. 2024) tackle the pose-free scenario, where networks predict novel views from unposed images; for example, DUSt3R (Wang al. 2024) and MASt3R (eroy et al.
2024 leverage transformers to output the pointmaps and estimate camera poses, and Noposplat Ye al. 2024) jointly predicts 3D representations and poses from sparse inputs. However, these methods generally lack extrapolative capabilities, unable to synthesize novel geometry or appearance regions unseen or occluded from the reference images.
Generative few-shot NVS. Recent diffusion based NVS approaches (Liu et al. 2023 Voleti et al.)
2024; Shi et al. 2023 Gao et al. 2024) leverage their generative capacity to synthesize novel views from or few images. Zero (Liu al.) 2023) fine-tunes diffusion model directly generate novel viewpoint image for relative pose from single image, and ZeroNVS (Sargent et al.
2023) extends upon this for single-view novel view synthesis. Similarly, MVDream (Shi al. 2023) and CAT3D (Gao et al. (024) employ spatial cross-attention between the generating viewpoints to achieve consistent novel view synthesis at target viewpoints, while ViewCrafter (Yu et al. 2024) delivers high-fidelity performance by finetuning a video diffusion model. Although these approaches offer strong extrapolative capability, they do not provide explicit geometry for the novel viewpoints, necessitating separate optimization process NeRF or 3DGS for full geometry. Moreover, because they receive target view camera pose as a feature embedding, the range of poses they can generate is limited to the training domain, hindering the direct generation of arbitrary novel poses.
Diffusion models (Rombach et al. 2022) have demonstrated strong inpainting capabilities (Lugmayr et al., 2022). which has motivated their application to novel view synthesis (NVS). LucidDreamer (Chung et al., 2023) leverages off-the-shelf monocular depth esti- mators [Bhat et al. 2023 Wang et al.) 2024 2025) to extract geometry from a single image, warp to target viewpoint, and inpaint missing regions, while GenWarp (Seo et al. 2024) uses predicted geometry as a correspondence signal for implicit inpainting. However, because these methods rely primarily 2D image inpainting without comprehensive understanding of 3D structure, they struggle to synthesize scenes with large view differences, particularly in object-centric scenarios where novel view results in warped geometry covering only small portion of the target viewpoints.
baarping-andinnaainn
3

---

Published as a conference paper at ICLR 2026
3 METHOD
Given N unposed and sparse reference RGB images {In E [B Jn= Jn=1, with height H and weight W, the method's objective is the joint prediction of novel view image T, and pointmap P. for target viewpoint Tt, leveraging the diffusion model's generative capabilities for high-fidelity novel view and geometry synthesis. Our method extends the sumping-and-inpaining methodology (Leroy et al. 2024, Seo et al. 2024) from single-image to multi-view settings. Importantly, we generalize this strategy from the image domain to geometry, performing geometry completion at the target viewpoint from partial geometry predicted by off-the-shelf models (Sec. 3.1). To ensure alignment between the target image and geometry, we introduce cross- moss-modal attention distillation (Sec. (Sec.3.22) multitask learning that yields synergistic benefits for both modalities. Finally, to handle noise and artifacts predicted point clouds, we introduce proximity-base mesh conditioning (Sec. 3.3 which prevents erroneous artifacts from degrading generation quality,
3.1
NOVEL VIEW IMAGE GENERATION
Our image generation architecture, as shown in the upper section of Fig. consists of two U-Nets: an image reference network and image denoising network. The image reference network (Hu et al., 2023) extracts semantic reference features from the reference images { E IR #x x denoising network utilizes the features from the reference network to generate novel view image.
In=1- and the
Geometry prediction and pointmap projection. We first leverage an off-the-shelf geometry prediction model (Wang et al., 2024 2025) to obtain the set of corresponding camera poses = n & RAx111 1==1 as well as pointmap {Pr E [=# Jn=1 from reference images. The pointmap Pn (Wang et al. 2024) is 2D grid of 3D point coordinates, where each element represents the predicted world coordinate for the given pixel. Next, the pointmaps of the reference images, Pi, P2. ........ PN. are interpreted as unordered sets of 3D points then merged into a single point cloud P. We then project P onto the target viewpoint TT:
N pll [I(, P P=UP..
resulting in the projected pointmap P. for the target view Tr, where II(-) denotes projection function. When multiple points project onto a single pixel, only the point closest to the target image plane is rendered, in the standard point cloud rasterization procedure (Seo et al. 2024).
Pointmap correspondence conditioning, We leverage the projected pointmap P!i and the reference view pointmaps (P,, ] denoising network to establish the correspondences between the target viewpoint and reference images. Specifically, first encode Pll using positional embedding {-) and concatenate the resulting Fourier feature E with a binary mask M,. which marks the grid pixels without any projected points. This forms the target correspondence condition ct for the image denoising network at target viewpoint Te. Similarly. for each reference viewpoint The we obtain a reference correspondence condition cf which consists of an embedded reference view pointmap Fourier feature ((P)), concatenated with a one-valued tensor mask 1, since every grid pixel in the reference view pointmap has corresponding 3D point due to dense prediction by the off-the-shelf model and therefore marked as As we obtain reference correspondence condition for every reference image J.
sparse geometric correspondence condition, which enables the image
[E(P!),, M,),, c, = (E(,,,, 1]. c [c,
(2)
Similar to Hu et al. 22023). these correspondence conditions are first passed through convolutional
network, resulting in target and reference correspondence condition features. The target correspon-
dence condition feature is added to the target image latent feature from the first convolutional layer of the image denoising network, while each reference correspondence condition feature is similarly added to the features of its corresponding reference image within the image reference network. Such conditioning guides the image denoising network in identifying relevant spatial correspondences from multiple reference images to ensure consistency in novel view generation. Notice that instead of providing explicit pixel-to-pixt tixel correlation (e.g., warped pixel coordinates (Seo et al., 2024)) between reference viewpoints and target viewpoint, we directly provide embedded pointmap condition. This design choice allows the model to associate each spatial location in the target image with multiple potential correspondences, providing the reference images for robust reconstruction.
4

---

Published as a conference paper at ICLR 2026
Aggregated attention. We conduct aggregated attention between the target view image features derived from the image denoising network and the reference features produced by the image reference and image value features V XIX H) from the spatial self-attention layers of the image denoising network. These target view image key and value features are concatenated with N image key and value reference features, resulting in the combined image key and value features, K and V2, respectively. Then we conduct aggregated attention (Seo al. 2024) with K', V and the image target view feature Qf query:
network. We acquire the image key features K E IRLXCx(WXX (W
Q' Qi:
K' [K], Ki, K2, Ky},
r = [V Vi V22 V
(5)
where KJ, vl <R(N-I)xx(WXX)) The spatial attention is computed as follows:
(Q'K""
Attention(Q' (Q'i ,'K v = Softmax
V
(6)
vd
where dj is the dimensionality of the key features. This design enables the image denoising network to simultaneously perform cross-attention across all reference images and self-attention within the target latents, ensuring a unified novel view synthesis.
3.2 ALIGNED NOVEL VIEW GEOMETRY GENERATION
We perform novel view geometry prediction alongside image synthesis using the same architecture as in image generation. The geometry denoising network (U-Net) predicts a target view pointmap, paired with the geometry reference network that receives reference view pointmaps {P,, In Similar to Ke jet al. (2024). our geometry denoising and reference networks are fine-tuned from image denoising U-Nets to predict pointmaps instead of images (ref., Sec. A in the Appendix). As described in the lower half of Fig. [ the geometry generation part predicts a pointmap for the target viewpoint ne, as in the image denoising network, receiving the same conditions, ct and ct. This makes the predicted pointmaps generated by the geometry generation network align with the generated image.
However, we found this naive approach of giving identical conditions was in- sufficient for the aligmment between the generated image and its geometry. The two modalities exhibit different behaviors when completing void regions, with ge- ometry denoising outperforming image in- painting, in Fig. This performance difference can be attributed to the more deterministic nature of geometry comple- tion than image generation, as geometric tasks have stronger structural constraints and less inherent ambiguity, resulting in more consistent and robust outcomes.
Target View Projection
Generation Result
Attention Map
Tanget View
Ref. View
(c)
Figure 3: Effeets of cross-modal atfention instillation.
As shown in Fig. 3 (a-b), when completing partially visible wheel, the image denoising (a) fails establish such correspondences, whereas the geometry prediction (b) correctly attends to other wheel locations for structural reference, producing more realistic completion. Conversely, while the geometry denoising network is better at geometry completion, the spatial attention within these networks struggles to get fine-grained cross-viewpoint correspondences due to the lack of semantic cues, as in (b), where the geometry network's attention is diffusely distributed in comparison to the more focused attention in (a). This complementary relationship motivates a synergistic framework where the geometry network leverages semantic cues from image features via their spatial attention map to achieve more detailed geometry generation, while the structural constraints from geometry completion implicitly guide the image denoising network toward more robust and consistent inpainting.
Cross-modal attention instillation. In this light, we propose cross-modal attention instillation, where the spatial attention maps for the geometry prediction U-Net substituted by the attention
5

---

Published as a conference paper at ICLR 2026
maps from the image denoising U-Net to achieve synergistic effects. Specifically, the key and query features extracted from the image demoising U-Net are leveraged by the spatial attention layers the geometry denoising U-Net. Let K and Q' denote the image key and query features from the image U-Net, respectively, and let VP [V:*, VV,, Vgg. ,.... V& represent the geometry value features from the geometry U-Net that generates pointmap P.. The spatial attention is follows, with dg the dimensionality of the key features:
(2'xz²) (viee
Attention (aaatioon(Q,,a (Q', KK, VP) softmax
VP,,
vde
This method offers several key advantages. As shown in Fig. injecting the attention map across the modalities reinforces the alignment between generated images and their geometries. The image denoising U-Net receives deterministic training signals from the geometry completion network, which regularizes its generation process, yielding enhanced consistency and inpainting capability, as demonstrated in Fig. (c). The geometry prediction U-Net also leverages rich semantic features from the image domain to achieve more accurate geometry completion. Additionally, since attention maps serve only as structural cues for aggregating value features, our architecture avoids the detrimental cross-modal feature mixing often observed in the prior work (He et al. 2024).
3.3 PROXIMITY-BASED MESI CONDITIONING
The off-the-shelf geometry models (Wang et al. 2024 Ke et al. 2024) typically generate a sparse 3D point cloud with noise and errors, which becomes particularly severe when the target viewpoint deviates significantly from the reference viewpoints. Erroneous projections from the sparse point cloud cause misalignment in the generation process, the networks cannot differentiate valid from erroneous, and harm the accuracy of generated images and their geometry.
To address these challenges, we propose proximity-based mesh conditioning. We convert the sparse point cloud into mesh representation with the ball-pivoting algorithm (Bernardini et al. 1999), which reduces erroneous projections and yields dense projections for the generation networks. Therefore, instead of employing the naïve projected pointmap P." as the correspondence condition, we utilize the pointmap derived from the projected mesh, denoted x! as our correspondence condition.
Furthermore, we augment the correspondence condition with the mesh's 's depth and normal map, enabling the network to prioritize reliable correspondences while filtering out noise and erroneous projections. Specifically, we channel-wist concatenate the partial depth map DJ and normal map N! acquired from our converted mesh to the correspondence condition embedding. Accordingly, our final correspondence conditions are expressed as follows:
c² iE(x!), D M.). c₁ =EE(X),,
(
We further refine the conditioning process by applying normal masking to exclude mesh planes whose normals deviate more than 90º from the target viewpoint's 's direction. These planes typically corre- spond to surfaces that have been erroneously projected due to the incomplete nature of the acquired geometry, and therefore should be excluded from the correspondence condition. By masking out these areas, we further ensure that the network is not influenced by erroneous or noisy correspondences.
EXPERIMENT
4.1
IMPLEMENTATION AND EXPERIMENTAL DETAILS
For image denoising networks, we initialize from Stable Diffussion 2.1 (Rombach et al. 2022).
Reference networks share identical architecture but omit timestep embeddings, serving only for feature extraction. We train on RealEstate 10K (Zhou et al., 2018). Co3D (Reizenstein et al. (021). and MVImgNet (Yu et al. 2023) using pseudo ground-truth geometry from VGGT (Wang et al. 2025). During training, only reference pointmaps are used for warping and proximity- mesh conditioning. At inference, VGGT predicts reference camera poses and pointmaps for projection to target viewpoints. For geometry denoising networks, initialize from Marigold (Ke al. 2024);s normal prediction, finetuning it to generate complete geometry from incomplete warped RGB conditioning rather than target images.

---

Published as a conference paper at ICLR 2026
Projection
Completion
Generated Geometry
Projection
Completion
Generated Geametry
Reference images
Reference Images ges
Referencc images
Reference Images
Reference
Reference
DARIS NORRIS
Reference Images
ORRIS
Imapes
Figure 4: Qualitative results. We demonstrate our qualitative results on the Co3D (Reizenstein et al. 2021 dataset, conducting NVS while generating aligned geometry robustly and consistently.
Extrapolative view setting. We define extrapolative camera settings as cases when the target camera position t lies outside the convex hull of reference camera positions of J.V ignoring viewing direction for simplicity. Alternatively, if t, cannot be expressed as a convex combination - on where an
and Σ _ 1. This geometric constraint means that significant portions of the target view may contain regions that are either occluded in reference views or lie beyond the observable scene boundaries, which requires the model to generate plausible scene content based on learned priors about scene structure and appearance to fill in the unknown regions.
4.2 EXPERIMENTAL RESULTS
Results on Co3D. Figure 4 demonstrates our approach on Co3D, reconstructing novel views with aligned geometry from three reference images. Generated images maintain consistency with ref- erences while achieving accurate geometric alignment. Multi-viewpoint pointmap visualizations show well-aligned geometry without scale-and-shift fitting, enabled by formulating depth prediction as completion combined with inpainting. Our denoising network directly generates novel views and geometry at target viewpoints without additional NeRF 3DGS optimization, overcoming limitations of prior diffusion- methods (Gao et al.
2024 Wu et al. 2023 Shi et al. 2023)
Zero-shot results on DTU. Table T and Fig. 5 compare
method against feedforward approaches 2024), MVSplat (Chen et al. 2024), DUSt3R (Wang et al. 2024).
(PixelSplat (Charatan et al.
wnnpiinginpaiiting methods using single views 2023), GenWarp (Seo et al. (024)). Evaluation DTU (Jensen et al. 2014) demonstrates zero-shot generalization capability of our method. For fair comparison, warping methods use identical geometry prediction (VGGT (Wang et al. (2025)). We introduce extrapolative view selection sampling the furthest target cameras. Our method achieves satte-of-the-at performance in both extrapolative and interpolative settings. Qualitative results show model effectively filters point cloud artifacts during warping, producing clean images with aligned geometry, while naive inpainting yields artifacts and inconsistent features.
NopoSplat (Ye et al.) 2024) using two views, and warping inpainting
(LucidDreamerr (Chung et al.
In-domian results on RealEstate10K. Table 2 compares our method against feedforward ap- praches--Pixelpllar (Charatan al. 2024). MVSplat (Chen et al. 2024), DUSt3R (Wang al.
7

---

Published as a conference paper at ICLR 2026
Table 1: Zero-shot quantitative comparison. We compare our model existing feedforward NVS methods (2-view setting) and warping-ande and-inpainting methods (1-view setting) on DTU [Zhou DTU[Zhou et al. (2018) dataset, which is zero-shot setting for all the models. Our method shows superior performance in both extrapolative and original (interpolative) setting in both single-view and view settings.
Extrapolative View
Interpolative View
Views
Method
Pose-free
PSNR1
SSIM
I.PIPS.
PSNR1
SSIM
I.PIPS.
Pixellppaat (Chrratan et al. MVSplar (Creen et a
2024
14.66
0.517
0.334
12.75
0.329
0.637
2-view
NoPoSplat Tee ar.
13.58
12.22
0.393
0.416
0.545
0.423
14.04
13.94
0.414
0.473
0.530
Ours (2-view)
15.58
0.615
0.184
16.58
0.643
0.152
LucidDreamer Chunge et Gen Warp ot 2024
11.14
0.423
0.440
12.09
0.481
0.419
1-view
9.85
0.315
0.527
9.54
0.298
0.538
Ours (I-view)
15.56
0.609
0.184
14.58
0.529
0.202
Reference Images
Warped Puint cloud
LucidDreamer
Ours
GT
Figure 5: Qualitative comparison with inpainting method on DTU (Zhou et al. 2018 dataset. Our qualitative comparison with the naive warping-anddinpainne zero-shot generalization capabilities to unseen data, as well as its ability to robustly handle erroneous warped geometries for geometrically consistent generation.
method demonstrates our model's
Table 2: In-domain comparison, We compare our model to existing feedforward NVS methods in Realestate 10K (Zhou et al. 2018) dataset, our method being superior extrapolative setting.
Extrapolative View
Interpolative View
Method
Pose-free
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
PixelSplat Charatan et al. 2024
X
14.01
0,582
0.384
23.85
0.806
0.185
MVSplut Chen et al. 2024
x
12.13
0.534
0.380
23.98
0.811
0.176
NoPoSplar Ye et
14.36
0.538
0.389
25.03
0.838
0.160
17.41
0.614
0.229
24.23
0.820
0.088
2024), and NopoSplat Ye et al. 2024) on RealEstate (Zhou et al.) 2018). We evaluate inter- polative and extrapolative conditions using two views. For extrapolation, we sample references from the latter third and targets from the first third of video frames. Our model outperforms feedforward methods in extrapolative settings while maintaining competitive interpolative performance. Figure shows conventional models struggle with large missing areas due to limited generative capabilities, whereas our approach effectively infers missing geometry and generates plausible imagery while preserving fidelity. We realistically inpaint partially visible objects with well-aligned geometry, attributed to geometric awareness from attention distillation regularizing structural attention maps.
In Fig 7 we additionally compare our method against recent large model-based novel view synthesis methods, namely LVSM [Jin et al. 2024), ZeroNVS (Sargent et al., 2023), and ViewCrafter (Yu et al. (024). As shown in the figure, our method excels in extrapolative scenarios with significant unobserved regions. While LVSM accurately reconstructs areas with geometric overlap, it fails beyond observable boundaries, producing blurry content in occluded areas. ZeroNVS produces plausible novel views but lacks fidelity and requires manual specification of ofeld-of-view, elevation, and content scale for each scene, with incorrect values leading to convergence failure. More critically, it requires approximately 2+ hours per scene for NeRF distillation via Score Distillation Sampling. ViewCrafter, evaluated with both 16-frame and 25-frame models, produces geometrically degraded
8

---

Published as a conference paper at ICLR 2026
PaxelSplat [3]
MV Sy [4]
DUSL3R [26]
NoPoSpiat [28]
Ous
Ours Depthi
Ground Twith
Images
Inages
Refamans
Imsges
Redanaze
Inneges
Figure 6: Qualitative comparison on extrapolative setting. Our qualitative comparison previous approaches demonstrates model's extrapolative capabilities to plausibly generate locations not seen in reference images while reconstructing faithfully the known regions.
Caroer
Vieoerafter
Reference View
VGGT
LVSM
ZeroNVS
Ours
Target View (GT)
Infermor Time A3020 (PL)
0.05s
0u6Ss
1.42h em4inu
13.28s
206-19%
9.67%
[obata ponewity
Figure 7: Qualitative comparison with large model-based NVS methods. Qualitative comparison of our method to previous approaches demonstrates our method's superior capabilities in conducting geometrically coherent image novel view synthesis with relatively short inference time.
imagery and artifacts under extrapolative settings, requiring 209.19 seconds for 25 frames A6000 GPU. In contrast, our generative approach leverages learned priors to synthesize plausible content for missing regions in only 9.67 seconds average, maintaining high-fidelity detail and

---

Published as a conference paper at ICLR 2026
geometric consistency. This demonstrates our model's competitive performance in both quality and efficiency in comparison to current large modell-aased novel view synthesis methods.
4.3 ABLATION
We conduct quantitative ablation studies on our method using the RealEsttat10KK dataset under the ex- trapolative setting. Our results in Ta- ble [] demonstrate the contribution of each component. The baseline (a) receives no geometric conditioning,
Table 3: Ablation study. We demonstrate how each of our components contributes to enhanced performance in novel view synthesis.
Components
PSNR
SSIM
LPIPS|
(a) Baseline
16.55
0.559
0.260
(b) + Pointmap condition
16.93
0.594
0.243
while (b) introduces naive pointmap
(C) Proximity- mesh
17.01
0.601
0.238
conditioning. When we add proximity-
(d) Cross-modal instillation
17.41
0.614
0.229
based mesh conditioning (c) and rross-modal attention distillation (d), we observe progressive performance improvements. We provide additional qualitative results Fig. [5] our Appendix.
Analysis on the number of input viewpoints. As our model conducts aggregated attention to generate novel views from reference images, it can receive an arbitrary number of input viewpoints for generation. To demonstrate this, in Table and Fig. of of our Appendix, we increase the number of reference viewpoints for a model trained at 2-viewpoint setting and analyze its effects in both image quality and geometric accuracy: the results demonstrate even without being trained on the given number of inputs, our model benefits strongly from additional viewpoints, showing the generalization capability of our aggregated attention architecture to various number of input reference viewpoints.
Table 4: Quantitative analysis regarding number of input viewpoints. We demonstrate improved performance with additional viewpoints inference for both image and geometry generation, despite training only on two-view settings.
Image
Geometry
Geometry (Recon)
Geometry (Inpainting)
Method
PSNR
SSIMt
LPIPS
Abs.Rell
81.25
Abs.Rell
81,25
Abs.Rell
01.25
2-view
17.41
0.615
0.230
0.196
0.715
0.152
0.819
0.308
0.531
3-view
20.02
0.700
0.146
0.143
0.788
0.151
0.849
0.304
0.598
4-view
20.08
0.701
0.144
0.140
0.787
0.113
0.846
0.311
0.594
Table 5: Robustness Gaussian perturbation. Our model shows robustness against Gaussian perturbation to the correspondence condition.
Table 6: Robustness to increase sparsity. Our model shows robustness against increased spar- sity in the correspondence condition.
Perturbation Level
PSNRT
SSIM
LPIPS_
Masking Level
PSNR1
SSIM
LPIPS
No noise
15.580
0.615
0.184
No masking
15,580
0.615
0.184
Gaussian perturb. (a 3%)
14.778
0.520
0.213
30%
masking
14.683
0.577
0.223
Gaussian perturb.
6%)
14,501
0.507
0.225
Gaussiun perturb. (a 10%)
14.129
0.487
0.239
50% masking
13.610
0.468
0.272
Gaussiun perturb. (a 15%)
13.726
0.465
0.262
80% masking
13.000
0.436
0.317
4.4 ANALYSIS ON ROBUSTNESS AGAINST EXTERNAL GEOMETRY PREDICTORS
We demonstrate model's robustness to errors and artifacts from external geometry predictors by adding various perturbations to the correspondence conditions. First, we apply varying levels of Gaussian noise predicted pointmap locations and show that our model exhibits minimal perfor- mance degradation even under high noise levels (Table5)). Second, we randomly mask points from the predicted pointmap to simulate sparse geometry, finding that our model maintains strong performance despite high sparsity in the correspondence condition (Tablel6)) Together, these results confirm that our diffusion-basee framework's generative capability effectively prevents error propagation from external priors, maintaining our pose-free methodology without compromising output quality. We provide qualitative evaluations of the same experiments in Fig. 10 and Fig. I of our Appendix.
CONCLUSION
We propose novel few-shot novel-view synthesis method that overcomes the limitations of existing approaches by jointly predicting images and geometry. By integrating cross-modall attention sharing and geometry- correspondence conditioning into warping-ant aadd-inaiinting framework, our method leverages deterministic cues from geometry completion to regularize the generation process, producing consistent, high-quality novel views even in challenging extrapolative scenarios.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENT
This research was supported by Institute of Information & communications Technology Planning & Evaluation (ITTP) grant funded by the Korea government (MSIT) RRS-2019-11190075, RS-2024- 00509279, 50--025-I1121200
RS-2024-00457882. RS- RS-2025-022144 4479, RS-2025-02217253 and the Culture, Sports, and Tourism R&D Program through the Korea Creative Content Agency grant funded by the Ministryof Culture. Sports and Tourism (RS-2024-003450255 RS-2024-00333060 RS-2023- 00222280, RS- 2023-00266509), and National Research Foundation of Korea (RS-2024-00346597).
RS-2023-00227552,
2025-25441833 RSS-025-25441825
REFERENCES
F. Bernardini, J. Mittleman, H. Rushmeier, C. Silva, and G. Taubin. The ball-pivoting algorithm for surface reconstruction. IEEE Transactions on Visualization and Computer Gruphics, 5(4): 349-359, 1999. doi: 10. 109/2945.817351.
Shariq Farooq Bhat, Reiner Birkl, Diana Wofk, Peter Wonka, and Matthias Müller. Zoedepth: Zero-shot transfer by combining relative and metric depth. arXiv preprint arXiv:2302. 12288, 2023.
David Charatan, Sizhe Lester Li, Andrea Tagliasacchi, and Vincent Sitzmann. pixelsplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 19457-19467, 2024.
Yuedong Chen, Haofei Xu, Chuanxia Zheng, Bohan Zhuang. Marc Pollefeys, Andreas Geiger, Tat-Jen Cham, and Jianfei Cai. Mvsplat: Efficient 3d gaussian splatting from sparse multi-view images. In European Conference on Computer Vision, pp. 370-386. Springer, 2024.
Jaeyoung Chung. Suyoung Lee, Hyeongjin Nam, Jaerin Lee, and Kyoung Mu Lee. Luciddreamer: Domain-fres generation of gaussian splatting scenes. arXiv preprint 1rXiv:2311.1333 2023.
Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3213-3223, 2016.
Ruiqi Gao, Aleksander Holynski, Philipp Henzler, Arthur Brussee, Ricardo Martin-Brualla, Pratul Srinivasan, Jonathan T Barron, and Ben Poole. Cat3d: Create anything in 3d with multi-view diffusion models. arXiv preprint arXiv:2405. 10314, 2024.
Jing He, Haodong Li, Wei Yin, Yixun Liang, Leheng Li, Kaiqiang Zhou, Hongbo Zhang, Bingbing Liu, and Ying-Cong Chen. Lotus: Diffusion- based visual foundation model for high-quality dense prediction. arXiv preprint arXiv:2409. /8/24, 2024.
Sunghwan Hong. Jaewoo Jung, Heeseong Shin, Jiaolong Yang, Seungryong Kim, and Chong Luo. Unifying correspondence, pose and nerf for pose-free free novel view synthesis from stereo pairs. arXiv preprint arXiv:2312.07246 2023.
Li Hu, Xin Gao, Peng Zhang, Ke Sun, Bang Zhang, and Liefeng Bo. Animate anyone: Consistent and controllable image-to-video synthesis for character animation, arXiv preprint arXiv:2311.17117, 2023.
Ajay Jain, Matthew Tancik, and Pieter Abbeel. Putting nerf on a diet: Semantically consistent few-shot view synthesis. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 5885-5894, 2021.
Varun Jampani. Kevis-Kokitsi Maninis, Andreas Engelhardt, Arjun Karpur, Karen Truong, Kyle Sargent, Stefan Popov, André Araujo, Ricardo Martin Brualla, Kaushal Patel, et al. Navi: Category- agnostic image collections with high-quality 3d shape and pose annotations. Advances in Neural Information Processing Systems, 36:76061-76084, 2023.
Rasmus Jensen, Anders Dahl, George Vogiatzis, Engil Tola, and Henrik Aanæs. Large scale multi- view stereopsis evaluation. In 2014 IEEE Conference on Computer Vision and Pattern Recognition,
pp. 406-415 IEFE, 2014.
11

---

Published as a conference paper at ICLR 2026
Haian Jin, Hanwen Jiang, Hao Tan, Kai Zhang, Sai Bi, Tianyuan Zhang. Fujun Luan, Noah Snavely, and Zexiang Xu. Lvsm: A large view synthesis model with minimal 3d inductive bias. arXiv preprint or arXiv:2410.17246 2024.
Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Metzger, Rodrigo Caye Daudt, and Konrad Schindler. Repurposing diffusion- image generators for monocular depth estimation. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9492-9502, 2024.
Bernhard Kerbl, Georgios Kopanas, Thomas Leimküüler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4):139-1, 2023.
Mijeong Kim, Seonguk Seo, and Bohyung Han. Infonerf: Ray entropy minimization for few-shot neural volume rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12912-12921, 2022.
Min-Seop Kwak, Jiuhn Song, and Seungryong Kim. Geconerf: Few-shot neural radiance fields via geometric consistency. arXiv preprint arXiv:2301.1090 2023.
Vincent Leroy, Yohann Cabon, and Jérôme Revaud. Grounding image matching in 3d with mast3r. In European Conference on Computer Vision, pp. 71-91. Springer, 2024.
Zhengqi Li and Noah Snavely. Megadepth: Learning single-view depth prediction from internet photos. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2041-2050, 2018.
Haotong Lin, Sili Chen, Junhao Liew, Donny Y Chen, Zhenyu Li, Guang Shi, Jiashi Feng, and Bingyi Kang, Depth anything 3: Recovering the visual space from any views. arXiv preprint arXiv:2511.10647, 2025.
Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov, Sergey Zakharov, and Carl Vondrick. Zero- 1-to-3 Zero-shot one image to 3d object. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 9298-9309, 2023.
Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11461-11471, 2022.
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1):99-106 106, 2021.
Michael Niemeyer, Jonathan T Barron, Ben Mildenhall, Mchdi SM Sajjadi, Andreas Geiger, and Noha Radwan. Regnerf: Regularizing neural radiance fields for view synthesis from sparse inputs. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp-
5480-5490, 2022.
Jeremy Reizenstein, Roman Shapovalov, Philipp Henzler, Luca Sbordone, Patrick Labatut, and David Novotny. Common objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction. In Proceedings of the IEEE/CVF international conference computer vision, pp. 10901-10911, 2021.
Xuanchi Ren, Tianchang Shen, Jiahui Huang, Huan Ling, Yifan Lu, Merlin Nimier-David, Thomas Müller, Alexander Keller, Sanja Fidler, and Jun Gao. Gen3e: 3d-informed world-consistent vidco generation with precise camera control. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp.
2022.
Kyle Sargent, Zizhang Li, Tanmay Shah, Charles Hermann, Hong-Xin Yu, Yunzhi Zhang, Eric Ryan Chan, Dmitry Lagun, Li Fei-Fei, Deqing Sun, and Jiajun Wu. ZeroNVS: Zero-shot 360-degree view synthesis from single real image. arXiv preprint arXiv:2310. 17994, 2023.
12

---

Published as a conference paper at ICLR 2026
Junyoung Seo, Kazumi Fukuda, Takashi Shibuya, Takuya Narihira, Naoki Murata, Shoukang Hu, Chich-Hsin Lai, Seungryong Kim, and Yuki Mitsufuji. Genwarp: Single image to novel views with semmatic-preserringg generative warping. arXiv preprint arXiv:2405.17251, 2024.
Yichun Shi, Peng Wang, Jianglong Ye, Mai Long, Kejie Li, and Xiao Yang. Mvdream: Multi-view diffusion for 3d generation. arXiv preprint arXiv:2308. 16512, 2023.
Vikram Voleti, Chun-Han Yao, Mark Boss, Adam Letts, David Pankratz, Dmitry Tochilkin, Christian Laforte, Robin Rombach, and Varun Jampani. Sv3d: Novel multi-view synthesis and 3d generation from single image using latent video diffusion. In European Conference Computer Vision, pp. 439-457. Springer, 2024.
Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. arXiv preprint 116.:,503..1661,, 2025.
Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r:
Geometric 3d vision made easy. In CVPR, 2024.
Rundi Wu, Ben Mildenhall, Philipp Henzler, Keunhong Park, Ruiqi Gao, Daniel Watson, Pratul P. Srinivasan, Dor Verbin, Jonathan Barron, Ben Poole, and Aleksander Holynski. Reconfusion: 3d reconstruction with diffusion priors. arXiv, 2023.
Botao Ye, Sifei Liu, Haofei Xu, Xueting Li, Mare Pollefeys, Ming- Hsuan Yang. and Songyou Peng. No pose, no problem: Surprisingly simple 3d gaussian splats from sparse unposed images. arXiv preprint arXiv:2410.24200 2024.
Alex Yu, Vickie Ye, Matthew Tancik, and Angjoo Kanazawa. pixelnerf: Neural radiance fields from one or few images. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4578-4587, 2021.
Wangbo Yu, Jinbo Xing, Li Yuan, Wenbo Hu, Xiaoyu Li, Zhipeng Huang, Xiangjun Gao, Tien- Tsin Wong, Ying Shan, and Yonghong Tian. Viewerafter: Taming video diffusion models for high-fidelity novel view synthesis. arXiv preprint arXiv:2409. 02048, 2024.
Xianggang Yu, Mutian Xu, Yidan Zhang, Haolin Liu, Chongjie Ye, Yushuang Wu, Zizheng Yan, Chenming Zhu, Zhangyang Xiong, Tianyou Liang, et al. Mvimgnet: A large-scale dataset of multi-view images. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9150-9161 2023.
Tinghui Zhou, Richard Tucker, John Flynn, Graham Fyffe, and Noah Snavely, Stereo magnification: Learning view synthesis using multiplane images. arXiv preprint 10X1v:1805.00981,,200 2018. 2018.
13

---

Published as a conference paper at ICLR 2026
APPENDIX
A ADDITIONAL DETAILS
A.1 TRAINING DETAILS
In training procedure, we initialize the image denoising U-Net from Stable Diffusion 2.1 and fine-tune on RealEstate 10K (Zhozuu et al. (2018), Co3D Reizenstein et al. (021). and MVIngNet Yu al. (2023). Reference networks share identical architecture and initial weights with the denoising U-Net but exclude timestep embeddings, serving solely for semantic feature extraction. Our model trained using a batch size of I with gradient accumulation, employing mixed-precision training (bf16) to optimize memory usage and computational efficiency. We utilize 8-bit Adam optimizer with standard hyperparameters: 31 0.9, 32 0.999, weight decay of Ix10-2 and e = 1x10 The learning rate is set to 1x10 with a constant scheduler and minimal warmup of step.
For multi-views training, we sample viewpoints total with reference views and target view (index 1), maintaining a temporal margin of 30 frames between reference and target images to ensure sufficient viewpoint diversity. All training images are resized to 512x512 resolution. We employ the xFormers memory-effieient attention mechanisms to handle the computational demands of multi-view aggregated attention. The noise scheduler follows a scaled linear beta schedule with 1000 training timesteps, Sstart 0.00085, Bend 0.012, and step offset of I, without sample clipping. Validation performed every 2000 training steps to monitor convergence and prevent overfitting.
A.2 COMPUTATIONAL EFFICIENCY
Despite employing a dual-branch architecture with multi-viww attention mechanisms, our model maintains competitive inference speed and memory efficiency. For joint 2-view image and geometry generation, our method requires 9.81 seconds on an A6000 GPU with 28GB memory consumption. Notably, modal attention instillation contributes to reduced memory overhead: the geometry branch reuses pre-computed attention maps from the image branch rather than computing separate attention operations, thereby saving both memory and computation. Furthermore, our image branch can operate independently for image-only novel view synthesis, requiring only 14GB memory and 4.3 seconds inference comparable to standard diffusion models, This flexibility allows users to trade off between full geometric consistency adual-brance mode) and faster image-only generation depending on application requirements.
A.3 IMPLEMENTATION DETAILS
Geometry for correspondence conditioning is generated using VGGT Wang et (2025). with only reference view pointmaps used for warping and proximity-based mesh conditioning. Ground truth geometry for MSE loss supervision is also predicted by VGGT using target images, enabling the model to maintain scale-and-shiff alignment with reference geometry automatically. This approach allows exact reconstruction of known geometry while completing unknown regions alignment with reference observations.
To accelerate and stabilize training, we separately fine-tune tuee the pointmap denoising U-Net and its reference network, initializing from Marigold Ke et al. (202))'s normal prediction weights with identical multiview aggregated attention as the image U-Net. This separate initialization enables both branches to learn robust independent representations before cross-man modall attention distillation, where geometry networks benefit from deterministic image cues, significantly improving prediction consistency. However, discussed in main paper, geometry-on/, training cannot achieve perfectly aligned predictions, necessitating cross-modal attention distillation for optimal performance.
Camera-space pointmap normalization. A key insight in our approach involves normalizing coordinate pointmaps through camera-centric transformation that substantially improves model performance. Specifically, we apply the target viewpoint's 's world-to-camert matrix to convert all predicted pointmap coordinate values into the target camera's local coordinate system. This pre- processing strategy addresses a critical issue in multi-view geometric learning: when coordinate pointmaps retain their original world-space values, the model must simultaneously learn handle
14

---

Published as a conference paper at ICLR 2026
dramatic variations in absolute coordinates while capturing subtle geometrie relationships. Such dual complexity often hampers training convergence and degrades synthesis quality.
Our camera-space transformation eliminates this burden by establishing unified coordinate frame centered on the target view. Within this normalized space, all geometrie information is expressed rela- tive to the target camera's position and orientation, creating more conducive learning environment. The model can then dedicate its representational capacity entirely to understanding the geometric correspondence patterns between reference and target configurations, without being distracted by irrelevant absolute positioning. This coordinate system alignment also ensures numerical consistency across training samples, preventing gradient instabilities that can arise from extreme coordinate ranges. The resulting geometric conditioning leads to more stable training dynamics and enhanced ability to synthesize geometrically plausible novel views across diverse camera poses and scene configurations.
Ref. Image
Target View Projection
Pointmap Normalization
w/o Pointmap Normalization
Figure 8: Ablation on pointmap normalization. Ablation study comparing synthesis results with and without camera-space pointmap normalization. Normalization significantly improves geometric consistency, boundary sharpness, and geometric alignment with projected geometry.
To validate the effectiveness of our pointmap normalization strategy, we conduct a qualitative ablation study comparing synthesis results with and without camera-space coordinate transformation. Our experimental results in Fig. S demonstrates that without normalization, the model struggles to maintain geometric consistency across different viewpoints, producing artifacts such distorted object boundaries, inconsistent depth relationships, and misaligned features between generated images and their corresponding geometry.
B ADDITIONAL RESULTS
B.1 ADDITIONAL. COMPARISON AGAINST LARGG MODELS
We additionally compare against VGGT WWagg et al. 2025). LVSM (Jin et al. 2024) and ViewCrafter Yu et al. 2024). large model-based novel view synthesis models. As shown in Fig. 9 our method excels in extrapolative scenarios with significant unobserved regions. LVSM fails beyond observable boundaries, producing blurry occluded content, while ViewCrafter generates geometrically degraded imagery under extrapolative settings and requires 209.19 seconds for 25 frames on an A6000 GPU. Our approach synthesizes plausible content in only 9.67 seconds on average, maintaining high-fidelity detail and geometric consistency. demonstrating superior quality and efficiency.
Quantitative evaluation. To fur- ther validate our model's gen- eralization capability, we con- duct quantitative evaluation on the DTU dataset under extrap- olative view settings in Table 7 We compare against ViewCrafter and LVSM under both single- view and two-view settings. Re-
Table 7: Comparison to large model baselines. We quantitatively compare our model against recent large-scale models.
Methods
View setting
PSNR1
SSIM
LPIPS
ViewCrafter
1-view
14.04
0.390
0.332
Ours (Single-view)
1-view
15.56
0,609
0.184
LVSM
2-view
15.23
0.499
0.415
Ours (Stereo-view)
2-view
15.58
0.615
0.184
sults demonstrate that our model achieves superior performance across all metrics in both configu-
15

---

Published as a conference paper at ICLR 2026
ViofCrrfttr (16 frumes)
ViewCrafter
Reference View
VGGT
LVSM
125 frames)
Ours
Target View (GT)
Figure 9:
Qualitative comparison with large models Navi (Jampani al. 2023) Dataset. Our model shows competitive performance against large model-based novel view synthesis models.
rations: under single-view input, our model outperforms ViewCrafter by +1.52 dB PSNR, +0.219 SSIM, and 0.148 LPIPS; under two-view input, our model surpasses LVSM by +0.32 dB PSNR, +0.116 SSIM, and 0.231 LPIPS. These quantitative results on DTU corroborate our qualitative find- ings on RealEstate 10k and Navi datasets, confirming our model's competitive performance against state-of-the-art methods across diverse benchmarks.
B.2 ROBUSTNESS TO PERTURBATIONS IN CORRESPONDENCE CONDITION
0.066
0.10
0.15
0-0,66g
0-0:00
0-0.15 e
Ground Truth
Ground Truth
6p Original
Figure 10: Robustness experiments regarding geometric noise. Visualization of MoAl's robustness against noise added the predicted geometry.
In Fig. 10 we present qualitative experiments examining robustness geometric noise in the predicted geometry. We apply varying levels of Gaussian noise (standard deviations of 6%, 10%, and 15% relative to the standard deviation of reference point cloud coordinates) to the point locations in the predicted pointmaps and use these noisy pointmaps correspondence conditions. Despite high noise levels, our model exhibits minimal performance degradation, demonstrating strong robustness against errors and artifacts in the predicted geometry. In Fig. [I provide similar experiments regarding the sparsity the predicted geometry: we randomly mask out a certain percentage of points (30%, 50% and 80%) from the predicted pointmap, and this sparse pointmap the
16

---

Published as a conference paper at ICLR 2026
Masked
30% Masked
50% Masked
80% Masked
Groente Orddiin
Ground Truth
Ground Truth
Gonerte
Figure 11: Robustness experiments regarding sparsity. Visualization of MoAI's robustness against increased sparsity of predicted geometry.
correspondence condition. Despite high sparsity in the correspondence condition, our model shows minimal performance degradation even at 80% masking scenario, demonstrating robustness against sparsity in predicted geometry. These results confirm that our framework's ' generative approach effectively mitigates errors from external priors, maintaining our pose-free methodology without compromising output quality.
B.3 QUALITATIVE EVALUATION WITH IN-THE-WIE WILD DATA
We provide additional results demonstrating our model's strong generalization to unseen domains, including in-the wild and urban data. We conducted experiments
new datasets from different domains: the CityScapes (Cordts et al.) (2066) dataset, containing urban strect-view multi-view images; the MegaDeptth (Li & Snavely 2018) dataset. None of these datasets used during training, thereby validating our model's generalization capabilities. Fig. 12 presents qualitative results demonstrating high-quality novel view synthesis and geometry reconstruction on tn-the-wild data, confirming strong generative performance across diverse domains.
Generated View
Generated Novel
Image
Geometry
Ground Truth
Image
Geometry
Ground Truth
Iie Refenrece Imagess
Re Re-ference Imagees
Refornse Images
Refermne Imuges
Refermee Inages
Referetice Images
Figure 12: Qualitative results on in-the-wild / urban data. We provide generalization results unseen in-the-wild urban datasets (MegaDepth, CityScapes), for which our model is capable of conducting high-fidelity novel view synthesis.
B.4 EVALUATION REGARDING GEOMETRIC CONSISTENCY IN OCCLUSION AND SHADING
in addition to Fig. demonstrates extrapolative view cases where our model generates target views more than 90º away from the reference viewpoints, requiring generation of completely unseen regions. Our model maintains physical consistency when extrapolating beyond 90" view differences, including correct shading and occlusion handling.
Fig. B
17

---

Published as a conference paper at ICLR 2026
Generated Image
Generated Geometry
Reference Image
Warped View
Ground Truth
Figure 13:
Occlusion handling in extrapolative viewpoints. We demonstrate the performance of our model under occlusion lighting / shadow handling extrapolative (greater than 90 degrees)
target camera pose.
Projected Geometry
Completed
Reference View Geometry
Target View Geometry (Gemerated)]
Combined Geometry
Geometry
Ref. Images
+
Projected
Completed Geometry
Reference View Geometry
Target View Geometry
Combined Geometry
Geametry
(Generated]
Ref. Images
+
Figure 14: Alignment visualization with different external model. We demonstrate alignment visualization between reference view geometry and generated target view geometry, evaluated with DepthAnything V3 (Lin al. 2025).
B.5 ALIGNMENT EVALUATION WITH DIFFERENT EXTERNAL POSE PREDICTION MODEL
The alignment of our generated geometry to the predicted geometry is inherently ensured by our ar- chitectural design, regardless of where the target camera pose derives from. Specifically, formulate
18

---

Published as a conference paper at ICLR 2026
geometry generation inpainting (or completion) of the pointmap that has already been projected to the target camera pose. In this formulation, the model learns to leave the projected geometry unchanged and simply generate extended geometry around (similar to the image inpainting task), thereby maintaining alignment within the projected geometry according to the given camera pose. Therefore, geometry completion and alignment take place agnostic the target-camera normalization within our architecture.
To demonstrate this, we provide cross-predictor validation experiment in Fig. 14 where employ an alternative external geometry model (DepthAnything v3 (Lin et al. 2025 at evaluation time to assess alignment between the externally predicted geometry and our model's generated geometry. The visualized results show that our model achieves accurate alignment and high-quality geometry completion regardless of which external prediction model used for geometry and pose prediction.
Pointmap conditioning
Proximity- based mesh
Cross-modal Instillation
Reference Images
Bascline
Ground Truth
Figure 15: Ablation results. Qualitative ablation study across five Co3D scenes demonstrating progressive improvements from naive baseline (spatially incoherent), through pointmap condition- ing (improved depth awareness), mesh-based proximity conditioning (reduced artifacts), and finally cross-modal attention distillation (highest quality with superior consistency). Each component contributes essential capabilities that culminate in state-of-the-arr performance with well-aligned modalities and enhanced realism..
B.6 QUALITATIVE ABLATION
We conduct qualitative ablation experiments across five Co3D scenes in Fig. 15 evaluating four progressive configurations to demonstrate each component's contribution. The totally naive baseline without geometric conditioning struggles with spatial coherence and geometric consistency, producing misaligned features and implausible geometry. Adding pointmap conditioning improves geometric awareness and depth relationships but remains insufficient for complex occlusions and fine-grained details. Mesh-based proximity conditioning yields substantial improvements by providing richer geo- metric cues, enabling Finally, incorporating cross-modal attention distillation produces the highest quality results through synergistic eeamety interaction, ensuring well-aligned modalities with superior consistency
accurate warping and cleaner geometry synthesis while reducing artifacts.
19

---

Published as a conference paper at ICLR 2026
Ref. Image
Projection
Completion
Generated Geometry
Figure 16: Qualitative results single-view extrapolative setting. Our method generate coherent novel view images and geometry from single unposed reference image at extrapolative camera viewpoints, inpainting locations whose information was given in reference images, while faithfully reconstructing the known regions.
and enhanced realism. This progressive enhancement clearly demonstrates how each component contributes essential capabilities that culminate in our method's state-of- state-of-the-an performance.
B.7 ADDITIONAL SCENE COMPLETION RESULTS
Our proposed method demonstrates remarkable flexibility and scalability across varying numbers of input viewpoints Fig. [6, showcasing its robust generalization capabilities beyond the two- view training configuration. In single-view novel view synthesis experiments conducted RealEstate 10K dataset. our model successfully generates high-quality novel views from a single reference image, effectively leveraging the geometric priors learned during training to infer plausible
the
20

---

Published as a conference paper at ICLR 2026
Projection
Completion
Generated Geometry
Projection
Completion
Generated Geometry
Referenca gess
Referense Images.
Reference Imagis
Reference Imasges
Reforence Images
Reference
Reference images
Reference images
Figure 17: Qualitative comparison on two-view extrapolative setting. Our method can generate coherent novel view images and geometry from two unposed reference images extrapolative camera viewpoints, inpainting locations not seen in reference images, while faithfully reconstructing the known regions.
scene structure and appearance for unseen viewpoints. This single-view capability is particularly challenging it requires the model hallucinate significant portions of the target view while maintaining geometric consistency with the limited reference information.
For two-view novel view synthesis, demonstrated Fig. 17 conduct comprehensive evaluations across Co3D Reizenstein et al. (202T) datasets, demonstrating consistent performance improvements when additional reference information becomes available. The model effectively aggregates infor- mation from both reference views through our multi-view attention mechanism, resulting in more accurate geometry estimation and higher-fidelity image synthesis. Our architecture's e's ability to seam- lessly handle two-view inputs during inference, despite being trained on this configuration validates the effectiveness of our correspondence conditioning and attention aggregation strategies.
21

---

Published as a conference paper at ICLR 2026
B.8 ANALYSIS ON THE NUMBER OF INPUT VIEWPOINTS
Our model conducts aggregated attention to generate novel views from reference images, it can receive arbitrary number of input viewpoints for generation, corresponds to simply concatenating additional reference viewpoint's s features within our attention architecture. To demonstrate this, Figure [8] we increase the number of reference viewpoints for model trained at 2-viewpoint setting and analyze its effects in both image quality and geometric accuracy: the results demonstrate even without being trained on the given number of inputs, our model benefits strongly from additional viewpoints, showing the generalization capability of our aggregated attention architecture to various number of input reference viewpoints.
additional reference image
Table 8: Quantitative analysis regarding number viewpoints. Our quantitative analysis con- ducted Co3D dataset shows that model's multi-viee niew aggregated attention enables it to gen- eralize to an arbitrary number of reference images, with performance consistently increasing with additional reference images.
Num. of viewpoints
2
3
4
5
6
PSNR
14.62
16.43
18.65
19.17
22.90
23.11
23.45
View
2 Viows
Vicws
4 Views
5 Views
Views
Vikws
Grond
Figure 18: Analysis on number of reference viewpoints. Our model's multi-view aggregated attention enables it to generalize to arbitrary number of reference images, with performance consistently increasing with additional reference images.
C THE USE OF LARGE LANGUAGE MODELS
Writing assistance. In this work, large language models (LLMs) were used exclusively for grammati- cal refinement and polishing of the manuscript's 's writing. All usages of LLMs only sentence-level or lower, proofreading for removal grammatical errors, or rephrasing of certain phrases to enhance the conciseness or academic flair of the sentences originally written by the authors. All technical content, research ideas, experimental design, analysis, and scientific conclusions were developed independently by the authors without LLM assistance. The LLMs did not contribute research ideation, methodology development, or interpretation of results. We take full responsibility for all content in this paper, including any LLM-assisted grammatical improvements.
22


Reviews:
[{"review_id": "pd7nMYwiHJ", "reviewer": "Reviewer_zYBJ", "summary": "This paper proposes a diffusion-based framework for few-shot novel view synthesis (NVS) that generates both aligned novel view images and corresponding geometries. Unlike prior approaches (e.g., PixelSplat, MVSplat, or diffusion-based GenWarp) that require dense camera poses or handle only interpolative view synthesis, this work aims to address the pose-free extrapolative NVS setting.\n\nThe key innovation is Cross-Modal Attention Instillation (MoAI): a mechanism that transfers attention maps from an image diffusion branch to a geometry diffusion branch. This encourages alignment between synthesized appearance and predicted geometry, mitigating inconsistencies that arise in conventional inpainting-based pipelines.", "questions": "- How does MoAI compare to simply concatenating geometry and image features before attention?\nCould this mechanism be replaced with a more general form of cross-attention sharing?\n\n- What happens if the pretrained geometry network is replaced with a lightweight learned depth prior, does performance drop significantly?\n\n- Does the model maintain physical consistency (e.g., shading or occlusion correctness) when extrapolating beyond 90° view differences?\n\n- Could the method be extended to temporal coherence (e.g., video novel view synthesis)?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- Novel and technically elegant contribution:\nThe cross-modal attention instillation mechanism is conceptually clean and effectively bridges the gap between image synthesis and geometry completion. It’s a natural extension of diffusion-based correspondence learning and could influence future cross-domain conditioning designs.\n\n- Comprehensive evaluation:\nThe experiments cover both extrapolative and interpolative regimes, multiple datasets (Co3D, DTU, RealEstate10K), and ablation analyses (Table 3–5) that clearly isolate contributions of each design component (pointmap conditioning, mesh proximity, MoAI).\n\n- Strong empirical results:\nThe method consistently outperforms existing approaches across metrics (e.g., PSNR↑17.4 vs 14.3 on DTU extrapolative), demonstrating robust generalization, especially in pose-free and single-view setups.\n\n- Interpretability and coherence:\nThe qualitative figures (Fig. 3, 6, 9) convincingly show improvements in structural alignment, artifact reduction, and cross-view consistency. The framework maintains geometric realism without explicit NeRF optimization.\n\n- Scalability and flexibility:\nThe model generalizes to variable numbers of input views (Table 5), highlighting strong architectural modularity and robustness.", "weaknesses": "- Limited exploration of generalization across domains and semantics:\nIt remains unclear whether the approach scales to in-the-wild scenes (e.g., urban/street-level data) or semantic diversity (humans, animals, etc.).\n\n- Dependence on pretrained geometry estimators (e.g., VGGT, Marigold):\nThe framework relies heavily on the quality of external predictors. Errors in these priors can propagate, which partially undermines the claim of being “pose-free” or “self-contained.”\n\n- Lack of detailed comparison to recent large-scale diffusion baselines:\nWhile comparisons to Zero123, GenWarp, and MVDream are discussed, newer world-consistent 3D diffusion models should be quantitatively evaluated to position the work more clearly.\n\n- Compute complexity and practicality:\nTraining with dual-branch diffusion networks (image + geometry) and multi-view attention is computationally expensive; inference speed and memory footprint are not reported.\n\n- Missing discussion of failure cases:\nThe paper does not analyze scenarios where MoAI might fail, e.g., inconsistent shadows or hallucinated surfaces due to semantic-geometry conflicts.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xIaROOYTOS", "reviewer": "Reviewer_EvEn", "summary": "The paper presents a diffusion-based warping-and-inpainting framework that jointly synthesizes novel-view images and geometry from unposed references. It uses off-the-shelf pose/pointmap predictors to project partial 3D into the target view, then completes both modalities via conditional denoising. The core idea, cross-Modal Attention Instillation (MoAI), replaces the geometry U-Net’s spatial attention with that of the image U-Net during training and inference to enforce image–geometry alignment and leverage semantic correspondences. To handle noisy projections, proximity-based mesh conditioning converts sparse point clouds to meshes, augments conditions with depth/normal cues, and masks implausible surfaces. Experiments on DTU and RealEstate10K show strong extrapolative NVS, competitive interpolation, aligned pointmaps enabling 3D completion, and consistent gains from pointmap/mesh conditioning and MoAI.", "questions": "- Scope of “pose-free”: The method estimates poses/pointmaps from references via an external model and then conditions the diffusion process. In practical deployments, how are target cameras specified? Are you assuming the user provides an arbitrary camera, or do you sample target poses? If target intrinsics/extrinsics are perturbed, how robust is synthesis?\n- MoAI mechanism and design choices: Which attention layers are instilled (early/mid/late; which resolutions)? Did you try partial instillation (e.g., only shallow or only deep blocks) or mixing a fraction of the image U-Net attention instead of full replacement? Please include layer-wise ablations and attention visualizations before/after MoAI on several scenes to clarify what correspondences are transferred and where the largest gains arise.\n- Comparisons to more baselines: Could you add AnySplat and a DUSt3R+PnP+splatting pipeline as additional pose-free baselines under 2–3 views, at least on DTU/RE10K subsets? If they fail at larger extrapolation, a controlled comparative failure analysis (with runtime and failure modes) would strengthen your positioning.\n- Failure cases and qualitative diagnostics: A short taxonomy of failure modes and their link to upstream geometry errors vs. diffusion inpainting would be very helpful, along with attention visualizations that reveal where MoAI succeeds or fails.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Joint diffusion for images and geometry with cross-modal attention instillation (MoAI) is a simple, original coupling; using image attention to guide geometry and vice versa is creative, and proximity-based mesh conditioning is a practical improvement over raw pointmaps.\n\n- Empirical results are strong in extrapolative settings on DTU and RealEstate10K, with clear additive gains in ablations (pointmap → mesh → MoAI) and the ability to benefit from more input views at test time despite two-view training.\n\n- The method is clearly described with equations and rationale; the motivation for MoAI is well illustrated, and training/inference details are transparent enough to reproduce.\n\n- The work is significant for pose-free extrapolative NVS and image–geometry alignment, producing aligned outputs that enable 3D completion and benefit downstream reconstruction and content creation.", "weaknesses": "- Reliance on off-the-shelf geometry/pose predictors   \nThe pipeline depends on VGGT/MASt3R-style predictors for both training supervision (pseudo GT) and inference conditioning. This creates a ceiling tied to those models’ biases and errors, and complicates fairness: improvements may partially reflect better use of VGGT rather than intrinsic advances. Please include: (i) a robustness study under degraded pointmaps/poses (noise, sparsity, biased scale), (ii) an alternative backbone (e.g., DUSt3R/MASt3R vs. VGGT) to show generality.\n\n- MoAI ablations are underspecified   \nThe paper replaces geometry U-Net attention with image U-Net attention, but lacks: (i) analysis of which layers/stages matter most, (ii) partial vs. full instillation, and (iii) compute/memory overhead.\n\n- Geometry alignment claims hinge on pseudo GT and camera-space normalization  \nDepth/pointmap “alignment” is evaluated with pseudo labels derived from the same family of predictors, and benefits from target-camera-space normalization. To avoid confirmation bias, add: (i) cross-predictor validation (e.g., train with VGGT pseudo GT, evaluate with MASt3R/Colmap), and (ii) ablations that remove target-camera normalization to quantify true contribution and residual misalignment.\n\n- Limited analysis of multi-view scaling and consistency across targets  \nThe model can accept more reference views at test time, but consistency across multiple simultaneously generated target views is not evaluated. Add: (i) cross-target consistency metrics (e.g., photometric and geometric cycle consistency), and (ii) performance scaling plots vs. number of references with compute trade-offs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9LsE9Jz4KH", "reviewer": "Reviewer_wJZj", "summary": "This paper proposes a diffusion-based inpainting framework to enhance novel view synthesis (NVS) quality, leveraging the 3D foundation model as geometric prior. Specifically, the method utilizes the off-the-shelf geometry predictors and fills both the geometry and image by framing NVS as a completion problem. To ensure alignment between the generated image and geometry, it presents MoAI where the attention maps from the diffusion branch are injected into a parallel geometry diffusion branch. The experiments validate the effectiveness of the proposed approach on several datasets.", "questions": "Please refer to the weaknesses in the weaknesses part. I am open to being persuaded based on the feedback from the authors.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "* The paper is well structured and easy to follow.\n* Experiments on Co3D and DTU datasets demonstrate the effectiveness of the proposed method.\n* The cross-modal attention installation is interesting to me and has been shown to be effective via the ablation study.", "weaknesses": "* The integration of diffusion priors to help NVS has been extensively explored in recent works, like diffusion-aided NeRF/3DGS  reconstruction (Deceptive-NeRF/3DGS [Liu et al.]) and unified frameworks such as ReconX, ViewCrafter, and ZeroNVS. Moreover, the proximity with mesh conditioning is quite standard.  Although the MoAI is interesting, it cannot support the whole paper for a top conference. \n* The paper compares primarily against classical or earlier NVS methods (e.g., NeRF variants), but omits several strong diffusion-based or geometry-conditioned NVS works, like ZeroNVS, ViewCrafter, Rand econX. \n* Only the quantitative comparison with LVSM is also missing in the paper, which makes it difficult to assess the actual progress relative to the state of the art.\n* The runtime and memory consumption are missing.\n* The mesh generation with incomplete and noisy points is not easy itself. The robustness of the method is not well evaluated, especially considering that it adopts a rather classical mesh generation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "N5cTIMyGWa", "reviewer": "Reviewer_GgBm", "summary": "This paper looks at how to perform improved view synthesis, especially in the case of more extreme viewpoint changes where older methods would usually break down (due to slightly incorrect geometry or large reasons for outpainting).\n\nThe authors solution is to leverage a predicted 3D point cloud and image generation methodology together in order to improve on the quality of either in isolation by use of a x-attention map that leverages information from both modalities.\n\nThe authors propose a pipeline, leveraging prior work (e.g. for point cloud extraction and viewpoint prediction) in order to achieve this.\n\nThey achieve good results in comparison to prior work.", "questions": "Please see above.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper is clearly written and easy enough to follow.\n\n2. The idea is reasonable and makes sense -- to enforce consistency between predicted geometry and viewpoint synthesis. They also propose to use 3D techniques (e.g. conversion to a mesh w/ corresponding normals and depth to allow the model to determine which points are more reliable.\n\n3. The authors evaluate both in/out of distribution and include ablations and standard baselines.", "weaknesses": "1. Some of the described intuition is unclear to me: \"The image denoising U-Net receives deterministic training signals from the geometry completion network,\" --> This implies to me that the image-inpainting and geometry prediction models are trained separately but from reading the paper, I don't think that they are. I'm also unsure as to what losses are used to train the geometry prediction model and what it is expected to look like -- my understanding is the loss is on the depmap which the point cloud is projected to. But this is unclear to me from reading the paper.\n\n2. Ablations on the geometry results.\nThe paper has an ablation in Table 3 on the impact of the geometry losses + cross model instillation for the view synthesis but not on how those same choices impact the geometric results in Table 4.\n\n3. Further comparisons to SOTA\nThe authors could have included results / comparisons (in terms of speed / performance) to the methods that they say they are improving against by not having to do a separate NERF optimization (e.g. CAT3D). In that case, it would be imp. to compare the performance by considering the need of the authors' work to generate the 3D point cloud in the first place using the other method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)