Efficient-LVSM: Faster, Cheaper, and Better Large View Synthesis Model via Decoupled Co-Refinement Attention
Authors: Xiaosong Jia
Yihang Sun
Junqi You
Songbur Wong
Zichen Zou
Junchi Yan
Zuxuan Wu
Yu-Gang Jiang
Venue: ICLR 2026 Poster (2026)
Keywords: novel view synthesis
novel view synthesis
transformer
large model
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Feedforward models for novel view synthesis (NVS) have recently advanced by transformer-based methods like LVSM, using attention among all input and target views. In this work, we argue that its full self-attention design is suboptimal, suffering from quadratic complexity with respect to the number of input views and rigid parameter sharing among heterogeneous tokens.  We propose Efficient-LVSM, a dual-stream architecture that avoids these issues with a decoupled co-refinement mechanism. It applies intra-view self-attention for input views and self-then-cross attention for target views, eliminating unnecessary computation. Efficient-LVSM achieves 29.86 dB PSNR on RealEstate10K with 2 input views, surpassing LVSM by 0.2 dB, with 2× faster training convergence and 4.2× faster inference speed. 
Efficient-LVSM achieves state-of-the-art performance on multiple benchmarks, exhibits strong zero-shot generalization to unseen view counts, and enables incremental inference with KV-cache, thanks to its decoupled designs.

Published as a conference paper at ICLR 2026
EFFICIENT-LVSM: FASTER, CHEAPER, AND BETTER LARGE VIEW SYNTHESIS MODEL VIA DECOUPLED CO-REFINEMENT ATTENTION
Xiaosong Jia Yihang Sun** S* Junqi You",, Songbur Wong", Zichen Zou!i²,
Junchi Yan Zuxuan Wu!.2t, Yu-Gang Jiang!?
l Institute of Trustworthy Embodied AI (TEAI), Fudan University
2 Shanggaai Key Laboratory of Multimodal Embodied AI
SSh.. of Computer Science & Sch. of Artificial Intelligence, Shanghai Jiao Tong University Conrespondence Authors https://effic:: eeriiiieeettiiii enteenttllle lvsm. g thub. 1o/
Equal Contributions
ABSTRACT
Feedforward models for novel view synthesis (NVS) have recently advanced by transformer-basee methods like LVSM, using attention among all input and tar- get views. In this work, argue that its full self-attention design is suboptimal, suffering from quadratic complexity with respect to the number of input views and rigid parameter sharing among heterogeneous tokens. We propose Efficient- LVSM, dual-stream architecture that avoids these issues with decoupled co- refinement mechanism. It applies intra-view self-attention for input views and self-then-cross attention for target views, eliminating unnecessary computation. Efficient-LVSM achieves 29.86 dB PSNR on RealEstate 10K with 2 input views, surpassing LVSM by 0.2 dB, with 2x faster training convergence and 4.4x faster inference speed. Efficiint-LLVS achieves state-of-the-art performance on multi- ple benchmarks, exhibits strong zero-shot generalization to unseen view counts, and enables incremental inference with KV-cache, thanks to its decoupled designs.
INTRODUCTION
Reconstructing 3D scenes from collection of 2D images remains a cornerstone challenge in com- puter vision. The field has witnessed remarkable evolution, moving from classical photogramme- try systems per-scene optimized neural representations like NeRF (Mildenhall et al. 2020) and 3DGS (Kerbl et al. 2023), which achieve high-quality reconstruction, but require dense inputs and costly optimization for each Models (LRMs) (Hong et al. 2024 Wei et al. 2024 Zhang et al., 2024), which learn generalizable 3D priors from vast datasets. A recent paradigm shift, pioneered by models like LVSM (Jin al., 2025), has further advanced the field by minimizing hand-crafted inductive biases, where it directly synthesizes novel views from posed images. It eliminates the need for predefined 3D structures or rendering equations and achieves surprisingly good rendering quality with flexibility.
scene. A significant advance came from Large Reconstruction
Despite the success, its monolithic self-attention mechanism. where all input and target tokens are concatenated into single sequence, leads to two primary drawbacks: (1) Low efficiency: full self- attention leads to quadratic complexity with regard to the number of input views (Jia al., 2023d). Furthermore, when generating multiple target views with the same input views, input representation can not be re-used. (2) Limited performance: full self-attention enforces parameter sharing for heterogencous tokens content-rich input views and pose-only target queries. It hinders the model's ability to learn specialized representations for their distinct tasks, i.c., understanding the semantics & 3D structure of the scene for input tokens and rendering the novel view for target tokens.
This work was supported by the Science and Technology Commission of Shanghai Municipality (No. 10311103100) and the New Cornerstone Science Foundation through the XPLORER PRIZE. This work was supported by Ant Group through CCF-Ant Research Fund, This work was also in part supported by Scientific Research Innovation Capability Support Project for Young Faculty (U40) of the Ministry of Education of China, SRICSPYT-2Y2050019,

---

Published as a conference paper at ICLR 2026
Input
Input
Latent
Input View
Input View
Target View
View
Tokens
Latent Tokens
Target View
Input View
Input View
Input View 2
Target
View
Latent
Input View
Input
View
View
Tokens
Latent
Target
View
Target
Input View
View
Input
Duplicatex
Latents
KV
Encoder
Decoder
Decoder
Encoder
Duplicate
Input Views
Latents
Target Viewsx NV
Input Views
Input Views
Target Views
Encoder-Decoder LVSM
Decoder-only LVSM
c) Efficient LVSM
Figure 1: Latent Novel View Synthesis Paradigms Comparison. The proposed decoupled archi- tecture disentangles the input and target streams with lower O(Ni)) complexity and no duplication of tokens.
In this work, systematically analyze these trade-offs and derive Efficient-LVSM. Transformer- based architecture designed to resolve these limitations. The key insight is to decouple the process of input view encoding from target view generation (Jia et al. 2023b; Wu et al., 2022). To realize this, Efficient-LVSM employs a dual-stream architecture. First, a dedicated Input Encoder is solely responsible for processing the source views. It uses intra-view self-attention to independently build a representation of each input's content and geometry. Second, a Target Decoder is tasked with synthesizing the novel view. The Target Decoder engages in continuous dialogue with the encoder: at each layer, the target tokens first refine their own understanding of the target view's structure through self-attention, and then query the features from the corresponding layer of the Input Encoder via cross-attention.
Specialized Attention Pathways. Our architecture utilizes distinct modules for input and target tokens (Jia et al. 2023c). In the input encoder, only input view is processed. In the target decoder, target tokens act as queries and input tokens serve as keys and values cross-attention, avoiding the use of shared parameters for heterogeneous information.
Robustness Variable View Counts. The input self-attention processes each view separately, making the transformation of one view independent of others. This per-view processing strategy allows the model to generalize better than LVSM to a variable number of input views at test time.
Computational and Memory Efficiency. The input encoder processes each input view sepa- rately and the target decoder adopts cross-attention, both reducing the computational complexity with respect to the number of input views from quadratic O(N2) to linear ONN...
Incremental Inference via K KV-Cache. The decoupled structure enables K KV-cache of input view features. When a new input view is provided, only that view needs to be processed. When new target view is required, the KV-cache could be directly re-used. In summary, the cost of adding new input views and target views is nearly constant and thus enables incremental inference.
We conduct comprehensive evaluations for Efficient-LVSM. sets a new state-of-the-arr, outper- forming LVSM by 0.2dB PSNR and GS-LRM by 1.7dB PSNR on the RealEstate 10K benchmark with 50% training time and achieves times speed acceleration terms of both training iteration and inference. It exhibits strong zero-shot generalization unseen numbers of input views.
2
METHOD
In this section, we present step-by-step analysis that derives the design of Efficient-LVSM.
2.1 PRELIMINARY
Given N input images with known camera poses and M target view camera poses, novel view synthesis (NVS) aims to render M corresponding target images. Specifically, the input {L, E,, Ki 1, 2, ..., .) } and {EE, Kj)lj 1,2, M. where E x x3 is the input RGB image, H and W the height and width, E, K I camera extrinsic and intrinsic. The output is rendered target images, denoted {i, 1,2, ... M. i E Hx
Task Definition:
2

---

Published as a conference paper at ICLR 2026
Decoder
X N
Linear & Unpachity
Encoder
K.
Cross-Attn
Self-Attn
Self-Attn
Self-Attn
Self-Attn
Self-Attn
Synthesized Novel Views Figure 2: Efficient-LVSM Model Structure. Efficient-IVSSM patchifies posed input images and target Plücker rays into tokens. Input tokens pass separately through an encoder to extract context, while target tokens cross-attend to generate new views. Asterisks indicate shared parameters.
Input Views & Plücker Rays
Target View Plücker Rays
adopt LVSM (Jin et al., 2025) end-to-end paradigm. For M the input {(L, E, k and ((E,, k.) all camera poses are encoded using Plicckr ray embedding (Plucker [865) while input images are patchified in ViT (Dosovitskiy et al. 2020). We obtain the input tokens {S by concatenating its RGB patches and Plücker ray patches in the hidden dimension and passing through an MLP. We obtain the target tokens (T3 J.M. by feeding its Plücker ray patches into another MLP.
Feedforward NVS Framework:
Next, input and target tokens pass through a set of transformer blocks to extract features, which is the key component of the framework: (R (((S)) ( A where & represents the {T)
transformer blocks and { the final features of target views.
The output layer transforms the final features of target views {Ry into RGB value by a lin- ear layer followed by a sigmoid function. The resulting RGB patches are then unpatchified and assembled to form the corresponding target images If. producing the final synthesized outputs:
Smpatchify(Sigmoidd .ineareeneer (R,)) mxxxx
Structure
Overall Complexity
Component
Complexity
Table 1: Comparison of Model Structure Complexity. The pro- posed Efficient-LVSM obtains lower complexity than LVSM and thus achieves significant speed up, as idenced in Sec. 3.4
LVSM
Eacoder
O(N2)
O(N M))
Encoder- Eneoder-Deeoder Decoder
Decoder
O(M)
LVSM
Decoder-Only
O(M(N + 1)))
Decoder
O(M(N + 1)))
Efficient-LVSM (Ours)
ONN + N)
Eacoder
O(N)
Decoder
((NM)
2.2
ANALYSIS OF LVSM'S FULL SELF- ATLENAITNN PARADIGM
LVSM deocder-only model employs full self-attention on all input and target tokens, which intro- duces the following two limitations:
Entangled Representation. From the content perspective, input tokens contain both semantic and geometric information, while target tokens only have geometric information. From the system per- spective, they bear distinct tasks: input tokens are to understand the semantics & 3D structure of the scene (Jia et al.) 2023a) and target tokens are to render the novel view. However, shared self-attention parameters do not distinguish the difference (Yang et al. 2025a), hampering the generalization abil- ity, as evidenced experiments in Table and visualizations in Fig. [cc
Computation and Memory Costs. Consider sample (S,, T3) with the shapes of NP x d and MP x d, where N and M are the numbers of input and target views, and P = HW/p² is the number of patches (p represents the patch size). LVSM decoder-only model constructs M separate sequences for M target views, where each sequence is a concatenation of the entire set of input tokens and the tokens of a single target view. These sequences are processed by full self-attention across multiple transformer layers. For a given layer indexed by the operation is defined as:
V concat (S, S2',... ...,,'',,
Ty'): S'y,
v V, [-1 Self-Attnaan (Vi (V
The shape of V, is M x (NP + P) x d. LVSM repeats the computation of one target view for M times. Thus, the temporal complexity of LVSM decoder-only model is M O(N2 p2) O(N2M),
3

---

Published as a conference paper at ICLR 2026
shown in Fig. | and Table | The quadratic complexity with regard the number of input views hampers the efficiency and the repetition of tokens introduces severe computational cost.
LVSM encoder-decoder structure avoids the repetition issue by using encoder compress all input views into one latent vector first. However, this design introduces loss of information, signif- icantly limiting the reconstruction quality, which is acknowledged in LVSM paper Ji et al. (2025)
2.3 DUAL- STREAM PARADIGM
Based on the observation above, propose dual-stream structure, where distinct modules applicd on input and target tokens to decouple the information flow, as in Fig. 2
Input Encoder:
To maintain the independency of different input views and improve efficiency, we limit the scope of self-attention to patches within the same input view. Each input view is processed separately, which enables efficient inference when new input view is provided (incremental in- ference). Instead of constructing a single, prohibitively long attention sequence containing tokens from all input views, we propose to process N shorter sequences. Specifically, let S represent the tokens of the ith input view. They are updated by an intra-view self-attention block layer
S = St 1- Sclf- mpp
(S ");; S = S, FFN! apu (S
(3)
Target Decoder: To allow efficient KV-cache for features of input views, target decoder employs cross-attention, letting output tokens Ty' attend to input tokens S; from the last layer of encoder:
Ty = TT' Cross-Atth (T)', (T)'S'" ". S... = FFN ( (Ty))
(4)
This design decouples the parameters for input and output tokens with dual-stream structure and allows rendering multiple target views with the same input KV-cache. While this approach shares conceptual similarities with recent architectures like MM-DiT (Esser et al.) 2024), which use dif- ferent projections for heterogeneous inputs, key distinction lies in our architectural choice. We argue that our tokens differ not just in content but in their fundamental computational roles: inputs content-rich providers, while targets are content-agnostic queries. This asymmetry motivates our use of fully decoupled dual-stream architecture, in contrast to MM-DiT's unified, full self-attention block.
Assuming the hidden dimension and the number patches per image are constants, the tempo- ral complexity of the Target Decoder are O(NM), while complexity of LVSM decoder-only is O(M(N + 1))), as in Table
2.4 INTRA- VIEW ATTENTION OF TARGET TOKENS IN DECODER
The aforementioned cross-attention only decoder design introduces a drawback: each target token has to store the information of the whole scene by their own since there is no scene-level interac- tion in input encoder, limiting the capacity. To this end, we propose to add intra-view self-attention target decoder alternatively with the original cross attention
T T Self-Atth (T)
T T Cross-Atttange (T, S1 S2 ..y)
(S)
T T FFN (T,²)
In this way, the intra-view self-attention in decoder allows to integrate scene-level information from other target tokens while still maintaining KV-cache ability. Experiments Table 6 demonstrates 6+6 layers self-then-cross attention performs better than 12 layers cross-attention.
2.5
CO-REFINEMENT OF SECODER-DECODER
One widely observed phenomenon for deep neural network is that different layers of features represent different abstract level of informantion (Zeiler & Fergus 2013): early layers captur- ing fine-grained details such as textures, and later layers encoding high-level semantics. vanilla encoder-decoder, only last layer features used, as in Fig. (a), which limits its capacity.
To this end, propose dual-stream co-refinement structure, illustrated in Fig. (b), where each layer of the encoder provides information to corresponding layer decoder. layer I, input tokens Si are first updated by self-attention and then the target decoder queries these updated tokens
4

---

Published as a conference paper at ICLR 2026
Vanilla Encoder-Decoder Decoder
Dual-Stream Co-telinement
Target Image
Venilla Encoder-Decodee
Dual Stream Co-refinmmnt
KV
Sel-AAin
Self-Cross-Attn
Self-Alln
Self-Cross-Ath
Sel-Aüti
Self Cruss
Self-Alin
Self Cross
Bell-Cross-Alln
Belt-Cross-Alin
Set-Atth
Salt Calt-Gross-ae Altn
Self Altn
Selt Cross-Arth
Views
Input
Targel View
Structure Comparison
Feature Map Comparison
Figure 3: Vanilla Ancoder-Deecodee vs. Dual-Stream Co-refinement. (a) Hidden features in wasted while the dual-stream co-refinement structure
middle layers in vanilla encoder-decoder utilizes these features to extract more information. (b) Feature maps indicate that co-refinement
ctructura catchor more clttalle of the tarnet vriow
Rendered Target
KV
Target GT
Fremrines
Input View
Self-Att
Self-Cross-Atm
Loss
Seft-Atth
Selt-Cross Attn
Selt-Attn
Self-Cross-Atm
Encoder
Decoder
Loss
Target View
Pretrained
Encoder
Without REPA
With REPA
Images
Input Views
Target View Plückar Rays
Placker
Structure REPA
b) Map Comparison
Figure 4: Applying REPA into Efficient-LVSM. (a) Pretrained vision encoders and MLP projectors discarded inference. (b) Feature maps indicate that REPA helps the model extract semantics. refine its own representation T
Ty Ty Self-Attre Self-Atthinge (T (-1)
Tj T Cross Cross-Attt Attolage ( (T) S1 S2' .S.'...,SS')
(6)
T T FFN" a (T,')
By querying the encoder's representations the middle layers, the decoder can synthesize own features using both the fine-grained details from early layers and the rich semantic context from later ones. Fig. (b) demonstrates that the co-refinement model generate more detailed and high-quality features compared to vanilla decoder structure.
2.6 DISTILLATION WITH REPA
With the decoupled attention for different views, natural thought is to utilize those powerful pre- trained vision encoder. To utilize visual features without sacrificing inference speed, we employ REPA Yuu et al., 2025) to distill visual features from DINOv3 (Siméoni et al., 2025). Formally, consider a clean image I and (Xx)x) is the projection of hidden teatures of layer k. where he is trainable projector and X represents the input tokens or target tokens of layer k: X, Sy or X₁ Tk. Let represent the pretrained encoder such DINÓv3. The goal is to align the projec- tion of layer output h (xx) with encoded images S(I) by maximizing the patch- wise similarities:
CREPA
Σ
ho
N
We find that improvement with REPA is conditional. Experiment in Table 6 that LVSM ben- efits much less compared to Efficient LVSM's dual-stream co-refinement design structure, possibly due to its full self-attention design entangles feature maps of different views.
2.7 KV-CACHE & INCREMENTAL INFERENCE
A key advantage of the decoupled dual stream design is its natural compatibility with KV caching during inference illustrated in Fig. 10 The key and values of all input views, (S.). can be
5

---

Published as a conference paper at ICLR 2026
Table 2: Scene-level View Synthesis Quality. We test on the same validation set proposed in pixelSplat.
Table 3: Object-level View Synthesis Quality. We test at 512 and 256 resolution on both input and rendering. "Enc" means encoder and "Dec" means decoder.
RealEstate 10k Zhou
2018
ABO Collins et al.
2022
GSO
Downs et al.
2022
PSNR
SSIM
LPP5 3
PSNR
SSIM
LPIPS
PSNR
7 SSIM
LPIPS
Next
20.43
0.589
0.550
ViewCrafter
Triplane-I RM (Res-512)
27.50
0.896
0.093
26.54
0.893
0.044
GPNR
24.11
0.793
0.255
GS-LRM (12s-512)
0.925
0.085
30.52
0.952
SEVA
25.66
0.841
0.139
Du
INSM Fne-Dee (Res-512)
29.86
0.913
0.065
29.32
0.933
0.052
pixelSplat
26.09
0.863
0.136
LVSM Dec-Only (12)
32.10
0.938
0.045
32.36
0.962
0.028
DepthSplar
27
0.889
0.118
Ours (Rcs-512)
32.65
0.951
0.042
32.92
0.973
0.021
MVSplat
26.39
GS-LRM
28.10
0.892
0.114
LGM (Res-256)
20.79
018113
0.158
21. /4
0.832
0.122
LVSM Enc- Eac-Deelres-256
28.58
0.893
0.114
LVSM Dec Onlyires- 156)
29.67
0,906
0.098
GS-LRM (56)
28.98
0.926
0.074
29.59
0,944
0.051
Ours(res-256)
0.102
INSM Ein--De (Ros-256)
0.923
0.052
29.19
0.932
LVSM Eac- Declres-5122 512)
28.55
0.894
0.173
LVSM Dec-Only (Res-256)
32.47
0.944
0.037
31.71
0.957
0.027
LVSM Dec- Onlyires-512))
29.53
0.904
0.141
Ours (Res-256)
33.13
0.960
0.035
32.73
0.969
0.022
Ours(res-512)
29.86
0,905
0.147
computed once and stored. When a new target view is required. the decoder could directly utilize the stored cache {S 1 for rendering. When a new input view IN+1 is introduced, only this new view needs to be processed and appended into the cache. As a result, it enables efficient incremental inference, which could be used in interactive application scenarios.
EXPERIMENTS
3.1 DATASETS
Scene-level Datasets. We use the widely used RealEstate dataset (Zhou al. 2018). It contains 80K video clips curated from 10K YouTube videos, including both indoor and outdoor scenes. We follow the training/testing split applied in LVSM (Jin et al. 2025).
We use the Objaverse dataset (Deitke et al. 2023) to train our model. Follow- ing the rendering settings in GS-LRM (Zhang et al. (2044). we render 730K objects, and each object contains 32 random views. We test our object-level model on Google Scanned Objects (Downs)
Object-level Dataset.
eet al. 2022) (GSO) and Amazon Berkeley Objects (Collins et al., 2022) (ABO), containing 1099
and 1000 objects respectively. Following Instant-3D (Li et al. 2023b) and LVSM (Jin et al. 2025 render 4 structured input views and 10 random target views for testing.
3.2
IMPLEMENTAION DETAILS
Model Details. Following LVSM (Jin et al. 2025 use a patch size of 8 8 for the image tokenizer with 24 transformer layers (12-layer encoder and 12-layer decoder) and the dimension of hidden feature 1024. Following REPA YYu et al. 2025), we select a 3-layer MLP as the alignment projection layer.
Protocols.
Following the settings in LVSM, select 4 input views and 8 target views in the object- level dataset. We select 2 input views and 3 target views in scene-level dataset.
More implementation details are provided in Appendix C
3.3
COMPARISON WITH START-OF-THEE-ARY MODELS
Scene-Level Comparison. We compare on scene-level inputs with RiXeINeRF YYu et al. 2021)
ViewCrafter (Yu et al. 2024), GPNR (Suhail et al., (002), SEVA (Zhou et al. 2025a), Du et al. (Du al. 2023). pixelSplat (Charatan et al. 2024). ,, DepthSplat (Xu et al. 2025), MVSplat (Chen al.] 2024 GS-LRM (Zhang et al. 2024). LVSM encoder-decoderam LV decoder-only (Jin et al.
2025). J. As in Table [2] our model establishes a new ofate-of-the-art on the RealEstate bench- mark, outperforming the previous leading method, LVSM decoder-only, by significant margin of 0.2 JB PSNR. This corresponds to an 4.5% reduction in Mean Squared Error (MSE), indicating a substantial improvement in reconstruction fidelity. This quantitative lepp is supported by our quali- tative results in Figure where our model produces noticeably sharper renderings and demonstrates superior geometric accuracy, particularly when synthesizing near-field objects where LVSM often introduces artifacts. Notably, this ofte-off-hheart performance is achieved with remarkable effi- ciency. Our model was trained for just days 64 A 100 GPUs, which is half the training time

---

Published as a conference paper at ICLR 2026
Input images
LVSM
Encoder-Decoder
LVSM
Decoder-Only
Ours
Ground
Truth
Scen--eeel evaluation
b) bbjet-leevel evaluation
Figure 5:
NVS Visual Comparison. We compare with LVSM (Jin al. 2025 in
RealEstate 10K Zhou et al. 2018) and Amazon Berkeley Objects (Collins al. 2022 Images rendered by our model have less blur details.
Ours
LVSM-Encoder-Decoder
LVSM-Decoder
230
20000
MemaryMMil
2000
Timelms)
5,0000
3400
1. faster
90% less
12
16
Target Number
Target Views Number
Target Views Numbor
700
10000
75000
Timelmod
30000
15000
MenoryMM))
200
10000
88%
2000
33% less
5000
2000
Input
Input
Input Views Number
Figure 6: Inference Speed Comparison. We compare the inference time (ms) against (a) the num- ber of target views and (b) the number of input views. Our model achieves consistently low latency. The performance of the LVSM baselines, particularly LVSM Decoder- Only, degrades severely as view counts increase. This highlights our model's ''' significant computational efficiency, achieving up to 14.9x speedup over LVSM Decoder-Only.
required by LVSM. In essence, EfficienttLLSM not only surpasses the previous state-of-the- in quality but does while requiring only 50% of the training budget.
Object-Leevl Comparison. Similarly, Efficient-LLSM achieves state-of-the-ard performance.
3.4 EFFICIENCY ANALYSIS
We evaluate the efficiency from three perspectives: vanilla inference latency, incremental inference latency, and training convergence speed. For fair comparison, keep the number of layers (12+12) and hidden dimension (1024) the same with LVSM. For the convergence analysis, smaller variants used for fast verification save computational resource.

---

Published as a conference paper at ICLR 2026
600
LVSM Decoder-Only
LVSM Encode-Decoder
Memory Peak (GB)
500
5.0
Time (ms)
400
4.5
300
200
3.5
100
2.5
10
12
10
12
16
Number of Input Views
Number of Input Views
(a) Incremental Inference Experiments. We compare the inference latency and memory consumption when input view fed one by one. observe that Efficient LVSM achieves near constant latency and memory consumption due to KV-cache ability.
28
30
2x faster
P5NR
PSNR
28
24
22
Ours REPA LVSM
Ours
Ours wio REPA
24
LVSM Decoder-Only
2
4
6
10
12
14
16
GPU Hours
Number of Input Views
(b) Training Speed Comparison. Efficient IVSM delivers roughly 2x faster training and consistently achieves higher PSNR.
(c) Zero-Shot Generalization to Input View Count. Trained with 4 input views and tested on varying numbers of input views.
Figure 7: Overall Comparison.
Table 4: Comparison Different Novel View Synthesis Methods.
Type
Model
Param.
Latency (ms) + GFLOPS 1 PSNR
pixelNeRF
28M
2500+
20.43
Optimization- Bassed
GPNR
10M
6000+
24.11
pixelSplat
125M
50.52
1934
26.09
Feed-forward GS
MVSplat
12M
10.23
583
26.39
GS-LRM
307M
88.24
5047
28.10
SEVA
2333M
29000
27.46
Diffusion-based
ViewCrafter
2609M
38000
21.63
LVSM Enc-Dee
177M
70.88
6042
28.58
Feed-forward Latent
LVSM Dec-Only
177M
109.37
8523
29.67
Ours (inference)
199M
24.78
1325
29.82
Vanilla Inference Speed. We analyze the inference by measuring latency, memory peak, and total GFLOPS as function of both input and target view counts. As shown in Fig. 6 our model's resource consumption exhibits a slow growth, maintaining high efficiency even with many views. In contrast, while the LVSM Encoder-Decoder shows a moderate increase in cost, the LVSM Decoder- Only variant suffers from a severe computational growth. This efficiency gap substantial and becomes increasingly pronounced as the number of views grows due Specifically, with 16 input views, our model is approximately 14.9x faster and consumes 50% less memory than LVSM Decoder-Only. This demonstrates that decoupled attention mechanism effectively removes the computational bottleneck caused by quadratic complexity.
linear complexity design.
Incremental Inference. Fig. indicate that the time and memory required for the incremental coming input views is nearly constant for Efficient-LVSM. Conversely, both LVSM baselines exhibit clear growth in latency and memory consumption.
8

---

Published as a conference paper at ICLR 2026
Table Ablation Study of REPA Distillation.
Category
Configuration
PSNR f
LPIPS +
SSIM 1
Without REPA Distillation (Baseline)
26.02
0.1481
0.8483
Ablation on REPA Hyperparannteers
Smooth L1
26.81
0.1349
0.8562
Loss Function
L2
26.39
0.1366
0.8571
Cosine
26.30
0.1374
0.8542
Input Tokens Only
26.35
0.1367
0.8569
Distillation Target
Target Tokens Only
26.27
0.1452
0.8536
Both Input & Target
26.60
0.1256
0.8642
Layer 8
26.60
0.1256
0.8642
DINOv3 Source Layer
Layer 10
26.28
0.1441
0.8540
Layer 12
26.11
0.1416
0.8503
Table 6: Ablation Study.
(a) Architectural Components Ablation.
(b) Effect REPA
Arch.
PSNR f
SSIM
LPIPS +
Arch./Variant
PSNR
SSIM
LPIPS
LVSM Dec-Only
25.52
0,8385
0.1541
Cross-Attenion Only
24.18
0.7908
0.1982
LVSM Dec-Only REPA
25.68
0,8410
0.1515
Self-Cross Attention
24.97
0.8201
0.1628
Ours wlo REPA
26.02
0.8483
0.1481
Co-Refinement
26.25
0.8462
0.1490
Ours w REPA
26.81
0,8628
0.16
(c) Effect Model Sizes
Models
Parameters
PSNR
SSIM 1
LPIPS
Latency(ms)]
GFLOPS.
Memory
Fne(12) Dec(12)
199M
28.32
0.8892
0.1106
24.78
1325
3032
Enc(6) Dec(6)
101M
27.77
0.8871
0.1149
17.58
647
1802
Ene(3) Dec(3)
53M
26.43
0.8609
0.1377
10.16
310
1321
Training Speed. As in Fig. (b), Efficient-LVSM demonstrates steeper learning
curve. It successfully reaches the final performance plateau of the LVSM baseline while consuming only half the computational budget (GPU hours).
To further assess the efficiency of Efficient LVSM within the broader landscape of novel view synthesis, we compare it against representative methods across four paradigms, as summarized in Table 4 As shown in the table, optimization based meth- ods (e.g.. PixelNeRF (Yu al., (02T)) require per-object optimization before rendering, resulting in substantial computation time. Diffusion-based models (e.g., ViewCrafter (Yu et al. (024)) suffer from prohibitive latency (ranging from seconds to minutes) due to the large number of sampling steps, making them unsuitable for real-time applications. Although feed-forward Gaussian Splat- ting approaches such as MVSplat (Chen et al. 2024) provide extremely low latency, they typically sacrifice reconstruction quality.
Comparison with State-of-the-art Paradigms.
Notably, our model surpasses GS-LRM ZZhang et al. 2024) in both quality and efficiency, requir- ing only 26% of its GFLOPS. Compared to LVSM variants, Efficient-LVSM maintains the high- quality rendering characteristic of large latent models and reduce inference latency to 24.78 ms. This demonstrates that decoupled attention design successfully mitigates the computational bot- tleneck of monolithic transformers without sacrificing performance.
3.5
ZERO- SHOT GENERALIZATION TO THE NUMBER OF INPUT VIEWS.
As in Fig. 7c Efficient-LVSM and LVSM both could benefit from more views even not trained under such data, thanks to the operator Transformer. Efficient LVSM constantly outperforms LVSM under all view settings while the gap is gradually reduced, since the reconstruction becomes easier with more input views.
3.6 ABLATION STUDIES
All ablation experiments smaller 6+6 encoder-decoder configuration budget.

---

Published as a conference paper at ICLR 2026
Co-refinement of Encoder-Decoder Structure. As in Table 6 (a), self-then-cross attention yields 0.79 dB PSNR improvement compared to cross-attention only in decoder. Further, adopting encoder-decoder co-refinement gives 1.28 dB PSNR gains.
Applicability REPA Distillation. As Table 6 (b), applying REPA to Efficient-LVSM brings a substantial gain of 0.8 dB while applying LSVM M only brings 0.16 dB improvement. In Table 5 we study the configuration absolute approximation to DINOv3 features instead of relative approximation as cosine similarity. We confirm that distillation for both input and target are useful. DINOv3's middle layer features instead of the final layers are most helpful, aligning with findings Siméoni
REPA. We find that Smooth LI loss works the best, possibly due to its
(2025).
Influcne of Model Size. As in Table (c), increasing model size consistently improves reconstruc- tion quality, aligning with Jii al. (2025). demonstrating the potential of feedforward models.
P Plücker features
Edges
Semantics
Target View Plücker rays
Render
KV
KV
Input Images Plücker rays
Pixel-level features
T Target View Images
Edges
Semantics
KV
kV
KV 1
Target View Plücker rays
Render
Plücker features Layer 1
Edges Layer 6
Semantics
Layer 12
Figure 8: PCA Visualization of Input and Target Views Features Different Layers.
4 VISUALIZATION
In Fig. 8 we visualize the features of Efficient-LVSM trained on RealEstatee We could observe that from the initial layer (Layer I) to the middle layer (Layer 6), the features contain more and more semantics. From the middle layer (Layer 6) to the last layer (Layer 12), the features becomes similar to the final output- RGB images. The evolving process demonstrates the effectiveness of the proposed co-refinement structure to extract features from all levels. By bridging the encoder and decoder at each layer, the model ensures that fine-grained structural details are preserved while progressively extracting information of the scene.
CONCLUSION
In this work, we present systematic analysis for issues of existing Transformer based NYS feed- forward model, identifying the bottlenecks of monolithic attention, Based on the analysis, we de- rive Efficient LVSM, decoupled dual-stream architecture that integrates an encoder-decoder co- refinement mechanism. Comprehensive experiments demonstrate that the proposed structure not only performs better but also achieves significant speed up for training convergence and inference latency. By successfully reconeiling the trade-off between quality and cost, approach paves the way for more scalable and interactive 3D) generation applications.
10

---

Published as a conference paper at ICLR 2026
Ethics Statement. Our research aims advance the field of computer vision and does not present immediate, direct negative social impacts. We believe our work has the potential for a positive impact by improving. The dataset used in this study is publicly available and have been widely adopted by the community for academic research. All data was handled in accordance with their specified licenses and terms of use. We did not any personally identifiable or sensitive private information. We have focused our evaluation on standard academic benchmarks. We encourage future research building upon our work to consider the specific ethical implications of their target applications.
Reproducibility Statement. To ensure the reproducibility of our research, we provide a comprehen- sive description of our methodology, implementation details, and experimental setup in the paper. Furthermore, we commit to making our code, pre-trained models, and experiment configurations publicly available upon publication of this paper.
REFERENCES
Yoonwoo Jeong amd Jinwoo Lee, Chiheon Kim, Minsu Cho, and Doyup Lee. Nvs-adapter: Plug- and-play novel view synthesis from a single image, 2024.
David Charatan, Sizhe Lester Li, Andrea Tagliasacchi, and Vincent Sitzmann. pixelsplat: 3d gaus- sian splats from image pairs for scalable generalizable 3d reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19457-19467, 2024.
Yuedong Chen, Haofei Xu, Chuanxia Zheng, Bohan Zhuang, Marc Pollefeys, Andreas Geiger, Tat- Jen Cham, and Jianfei Cai. Mvsplat: Efficient 3d gaussian splatting from sparse multi-view images, 2024. URL https:://axxi...rrr// abs
14627
Inchang Choi, Orazio Gallo, Alejandro Troccoli, Min H Kim, and Jan Kautz. Extreme view synthe- sis. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7781-
7790, 2019.
Jasmine Collins, Shubham Goel, Kenan Deng, Achleshwar Luthra, Leon Xu, Erhan Gundogdu, Xi Zhang. Tomas F. Yago Vicente, Thomas Dideriksen, Himanshu Arora, Matthicu Guillaumin, and Jitendra Malik, Abo: Dataset and benchmarks for real-world 3d object understanding. 2022.
org/ btrgg/bba abs 2210 96199
Paul E. Debevec, Camillo Jose Taylor, and Jitendra Malik. Modeling and rendering architecture from photographs: A hybrid geometry- and image-based approach. Seminal Graphics Papers: Pushing the Boundaries, Volume 2, 1996. URL htt.h:ttss//// httpps::/ap...smmttttcttslto www.nticsccoool.................
[CoppuSI:: 26094115
Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani. Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of anno- tated 3d objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13142-13153, 2023.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recogmition
at scale. ArXiv, abs/2010.11922 11929, 2020. URL https: /tts://apl. /api. www.mtticsccollls........ oorg/
CorpusID:22503990
Laura Downs, Anthony Francis, Nate Koenig, Brandon Kinman, Ryan Hickman, Krista Reymann, Thomas B. McHugh, and Vincent Vanhoucke. Google scanned objects: high-quality dataset of 3d scanned houschold items, 2022. URL httpthtttts/iiiiie www.wwwwww...a.oo.aaoooooooooooa abs 2204. 11918
Yilun Du, Cameron Smith, Ayush Tewari, and Vincent Sitzmann. Learning to render novel views 08463
from wide-baseline stereo pairs, 2023. URL
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-fire international conference on machine learning, 2024.
11

---

Published as a conference paper at ICLR 2026
Cunxin Fan, Xiaosong Jia, Yihang Sun, Yixiao Wang, Jianglan Wei, Ziyang Gong, Xiangyu Zhao, Masayoshi Tomizuka, Xue Yang, Junchi Yan, et al. Interleave-vla: Enhancing robot manipulation with interleaved image-text instructions. arXiv preprint arXiv:2505.02152, 2025.
Ruiqi Gao*, Aleksander Holynski Philipp Henzler, Arthur Brussee, Ricardo Martin-Brualla, Pratul Srinivasan, Jonathan T. Barron, and Ben Poole*. Cat3d: Create anything in 3d with multi-view diffusion models. Advances in Neural Information Processing Systems, 2024.
Will Gao, Dilin Wang, Yuchen Fan, Aljaz Bozic, Tuur Stuyck, Zhengqin Li, Zhao Dong, Rakesh Ranjan, and Nikolaos Sarafianos. 3d mesh editing using masked Irms. arXiv preprint arXiv:2412.08641, 2024.
Ross Girshick. Fast r-cnn. In Proceedings of the IEEE international conference computer vision, pp. 1440-1448, 2015.
Steven Gortler, Radek Grzeszezuk, Richard Szeliski, and Michael F. Cohen. The lumigraph. Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, 1996. URLhttps: stt.app.sssmmtttcccccccleeoeoeee ott.ccooo....ooo//ooorrrata.....
Peter Hedman, Julien Philip, True Price, Jan-Michael Frahm, George Drettakis, and Gabriel Bros- tow. Deep blending for vree-viewpoint image-based rendering. ACM Transactions on Graphics 376)1111, 2018.
Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, and Hao Tan. Lrm: Large reconstruction model for single image to 3d, 2024. URL https: //axxiv org/ wttg/a//////oooo /alss/23111 04400
Yihan Hu, Jiazhi Yang, Li Chen, Keyu Li, Chonghao Sima, Xizhou Zhu, Siqi Chai, Senyao Du, Tian- wei Lin, Wenhai Wang, Lewei Lu, Xiaosong Jia, Qiang Liu, Jifeng Dai, Yu Qiao, and Hongyang Li. Planning-orientee autonomous driving. Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition, pp. 17853-17866 2023.
Xiaosong Jia, Li Chen, Penghao Wu, Jia Zeng, Junchi Yan, Hongyang Li, and Yu Qiao. Towards capturing the temporal dynamics for trajectory prediction: coarse-to-fine approach. In Confer- ence on Robot Learning, pp. 910-920. 920. PMLR, 2023a.
Xiaosong Jia, Yulu Gao, Li Chen, Junchi Yan, Patrick Langechuan Liu, and Hongyang Li. Driveadapter: Breaking the coupling barrier of perception and planning in end-to-end autonomous driving. Proceedings of the IEEE/CVF International Conference on Computer Vision, pp- 7953-7963, 2023b.
Xiaosong Jia, Penghao Wu, Li Chen, Yu Liu, Hongyang Li, and Junchi Yan. Hdgt: Heteroge- neous driving graph transformer for multi-agent trajectory prediction via scene encoding. IEEE transactions on pattern analysis and machine intelligence, 45(1 (31111113800-3888
2023c.
Xiaosong Jia, Penghao Wu, Chen, Jiangwei Xie, Conghui He, Junchi Yan, and Hongyang Li. Think twice before driving: Towards scalable decoders for end-to-end autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 21983-21994, 2023d.
Xiaosong Jia, Shaoshuai Shi, Zijun Chen, Li Jiang, Wenlong Liao, Tao He, and Junchi Yan. Amp: Autoregressive motion prediction revisited with next token prediction for autonomous driving. arXiv preprint arXiv:2403. 1334403..33,,, 13331, 2024a.
Xiaosong Jia, Zhenjie Yang. Qifeng Li, Zhiyuan Zhang, and Junchi Yan. Bench2drive: Towards multi-ability benchmarking of closed-loop end-to-end autonomous driving. In NeurIPS 2024 Datasets and Benchmarks Track, 2024b.
Xiaosong Jia, Yanhao Liu, Junqi You, Renqiu Xia, Yu Hong, and Junchi Yan. Drivevggt: Visual geometry transformer for autonomous driving, 2025a. URL https: //tp///avvv////////////taa//..at org/ abs// 2511 22264
12

---

Published as a conference paper at ICLR 2026
Xiaosong Jia, Junqi You, Zhiyuan Zhang, and Junchi Yan. Drivetransformer: Unified transformer for scalable end-to-end autonomous driving. In International Conference on Learning Represen- tations (ICLR), 2025b.
Xiaosong Jia, Chenhe Zhang, Yule Jiang, Songbur Wong, Zhiyuan Zhang, Chen Chen, Shaofeng Zhang, Xuanhe Zhou, Xue Yang, Junchi Yan, al. Spatial retrieval augmented autonomous driving. arXiv preprint Sedi::2122.066666 2025c.
Haian Jin, Hanwen Jiang, Hao Tan, Kai Zhang, Sai Bi, Tianyuan Zhang, Fujun Luan, Noah Snavely, and Zexiang Xu. Lysm: A large view synthesis model with minimal 3d inductive bias. In The Thirteenth International Conference on Learning Representations, 2025. URL net www.:eoooni...-ooee.e....
https:
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splat- ting for real-time radiance field rendering. ACM Transactions Graphics, 42(4), July 2023. URL https://repo: sam. inria. wttwfgggaaf////dtttccoooo gassdianaaaannnnna splatting/
Hongyang Li, Chonghao Sima, Jifeng Dai, Wenhai Wang, Lewei Lu, Huijie Wang, Jía Zeng, Zhiqi Li, Jiazhi Yang, Hanming Deng, Hao Tian, Enze Xie, Jiangwei Xie, Li Chen, Tianyu Li, Yang Li, Yulu Gao, Xiaosong Jia, Si Liu, Jianping Shi, Dahua Lin, and Yu Qiao. Delving into the devils of bird's seey---we perception: A review, evaluation and recipe. IEEE Transactions Patlern Analysis and Machine Intelligence, 46(4):2151-2170, 2023a.
Jiahao Li, Hao Tan, Kai Zhang, Zexiang Xu, Fujun Luan, Yinghao Xu, Yicong Hong, Kalyan Sunkavalli, Greg Shakhnarovich, and Sai Bi. Instant3d: Fast text-to-3d with sparse-view gen- eration and large reconstruction model, 2023b. URL httts:///aaxiiiii http///awww...rr/b/////111/11111 06214
Qifeng Li, Xiaosong Jia, Shaobo Wang, and Junchi Yan. Think2drive: Efficient reinforcement learning by thinking with latent world model for autonomous driving (in carla-v2). In European Conference on Computer Vision, pp. 142-158. Springer, 2025.
Han Lu, Xiaosong Jia, Yichen Xie, Wenlong Liao, Xiaokang Yang, and Junchi Yan. Ac- tivead: Planning-oriented active learning for end-to-end autonomous driving. arXiv preprint arXiv:2403.02877, 2024.
Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes neural radiance fields for view synthesis, 2020. URL
Julius Plucker. Xvii. on a new geometry of space. Philosophical Transactions of the Royal Society of London, pp. 725-791, 1865.
Ben Poole, Ajay Jain, Jonathan T. Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv, 2022.
Mehdi SM Sajjadi, Henning Meyer, Etienne Pot, Urs Bergmann, Klaus Greff, Noha Radwan, Suhani Vora, Mario Lučić, Daniel Duckworth, Alexey Dosovitskiy, et al. Scene representation trans- former: Geometry-free novel view synthesis through set-latent scene representations. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 222996238, 2022.
Oriane Siméoni, Huy Vo, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Mare Szafraniec, Seungeun Yi, Michaël Ramamonjisoa, Francisco Massa, Daniel Haziza, Luca Wehrstedt, Jianyuan Wang, Timothée Darcet, Théo Moutakanni, Leonel Sentana, Claire Roberts, Andrea Vedaldi, Jamie Tolan, John Brandt, Camille Couprie. Julien Mairal, Hervé Jégou, Patrick Labatut, and Piotr Bojanowski. DINOv3, 2025. URL
orgg/
abs 2 220/2258..0 10104
Mohammed Suhail, Carlos Esteves, Leonid Sigal, and Ameesh Makadia. Generalizable patch-based neural rendering. In European Conference on Computer Vision, pp. 156-174. Springer, 2022.
13

---

Published as a conference paper at ICLR 2026
Ashish Vaswani, Noam M. Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. In NeurIPS, 2017. URL https: //api. www.ssssaaniccchllaaatoooooo
Daniel Watson, William Chan, Ricardo Martin Martin-Brualla, Brualla, Jonathan Ho, Andrea Tagliasacchi. and Mo- hammad Norouzi. Novel view synthesis with diffusion models. arXiv preprini urXiv:22
2022.
Xinyue Wei, Kai Zhang, Sai Bi, Hao Tan, Fujun Luan, Valentin Deschaintre, Kalyan Sunkavalli, Hao Su, and Zexiang Xu. Meshlrm: Large reconstruction model for high-quality mesh. arXiv preprint arXiv:2404. 12385, 2024.
Penghao Wu, Xiaosong Jia, Li Chen, Junchi Yan, Hongyang Li, and Yu Qiao. Trajectory-guided control prediction for end-to-end autonomous driving: A simple yet strong baseline. Advances in Neural Information Processing Systems, 35:6119- 11-622991,,,, 2022.
Penghao Wu, Li Chen, Hongyang Li, Xiaosong Jia, Junchi Yan, and Yu Qiao. Policy pre-training for autonomous driving via self-supprvised geometric modeling. In International Conference on Learning Reprexentations, 2023a.
Rundi Wu, Ben Mildenhall, Philipp Henzler, Keunhong Park, Ruiqi Gao, Daniel Watson, Pratul P. Srinivasan, Dor Verbin, Jonathan T. Barron, Ben Poole, and Aleksander Holynski. Reconfusion: 3d reconstruction with diffusion priors. arXiv, 2023b.
Haofei Xu, Songyou Peng, Fangjinhua Wang, Hermann Blum, Daniel Barath, Andreas Geiger, and Mare Pollefeys. Depthsplat: Connecting gaussian splatting and depth. In CVPR, 2025.
Zhenjie Yang, Xiaosong Jia, Hongyang Li, and Junchi Yan. Llm4drive; A survey of large language models for autonomous driving, 2023.
Zhenjie Yang, Yilin Chai, Xiaosong Jia, Qifeng Li, Yuqian Shao, Xuckai Zhu, Haisheng Su, and Junchi Yan. Drivemoe: Mixture-of-experts for vision- vision-language-ncti language-action model in end-to-end au- tonomous driving. arXiv preprint arXiv:2505. 16278, 2025a.
Zhenjie Yang, Xiaosong Jia, Qifeng Li, Xue Yang. Maoqing Yao, and Junchi Yan. Raw2drive: Reinforcement learning with aligned world models for end-to-end autonomous driving (in carla v2). Advances in Neural Information Processing Systems, 2025b.
Junqi You, Xiaosong Jia, Zhiyuan Zhang, Yutao Zhu, and Junchi Yan. Bench2drive-r: Turning real world data into reactive closed-loop autonomous driving benchmark by generative model. arXiv preprint arXiv:2412.0000 22.444947777 2024.
Junqi You, Chich Hubert Lin, Weijie Lyu, Zhengbo Zhang, and Ming-Hsuan Hsuan Yang. Instainpaint: Instant 3d-scene inpainting with masked large reconstruction model. NeaulPS, 2025.
Alex Yu, Vickie Ye, Matthew Tancik, and Angjoo Kanazawa pixelnerf: Neural radiance fields from one or few images, 2021. URLPttte /wt.aaaiiiv////////v//eaa//aaaaa org/absg.......o..o.. 2012.0200000 021 100
Sihyun Yu, Sangkyung Kwak, Huiwon Jang, Jongheon Jeong, Jonathan Huang, Jinwoo Shin, and Saining Xie. Representation alignment for generation: Training diffusion transformers is easier than you think. In International Conference Learning Representations, 2025.
Wangbo Yu, Jinbo Xing, Li Yuan, Wenbo Hu, Xiaoyu Li, Zhipeng Huang, Xiangjun Gao, Tien- Tsin Wong, Ying Shan, and Yonghong Tian. Viewcrafter: Taming video diffusion models for high-fidelity novel view synthesis. arXiv preprint arXiv:2409.0200 2024.
Matthew D. Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. ArXiv, abs/1311.290 2013. URL
3960646
Kai Zhang, Sai Bi. Hao Tan, Yuanbo Xiangli, Nanxuan Zhao, Kalyan Sunkavalli, and Zexiang Xu. Gs-Inm: Large reconstruction model for 3d gaussian splatting, 2024. URL https: : //arxiv. org/abs/ 2404. 19702
14

---

Published as a conference paper at ICLR 2026
Jensen (Jinghao) Zhou, Hang Gao, Vikram Voleti, Aaryaman Vasishta, Chun-Han Yao, Mark Boss, Philip Torr, Christian Rupprecht, and Varun Jampani. Stable virtual camera: Generative view synthesis with diffusion models. arXiv preprint arXiv:2503.14488 2025a.
Sizhuo Zhou, Xiaosong Jia, Fanrui Zhang, Junjie Li, Juyong Zhang, Yukang Feng, Jianwen Sun, Songbur Wong, Junqi You, and Junchi Yan. Lagen: Towards autoregressive lidar scene generation, 2025b. URL https: /arxiv. org/ hbtp//1bs22111... 21256
Tinghui Zhou, Richard Tucker, John Flynn, Graham Fyffe, and Noah Snavely. Stereo magnification: Learning view synthesis using multiplane images. In SIGGRAPH, 2018.
Yutao Zhu, Xiaosong Jia, Xinyu Yang, and Junchi Yan. Flatfusion: Delving into details of sparse transformer-basee camera-lidar fusion for autonomous driving. IEEE International Conference on Robotics and Automation (ICRA), 2025.
A USE OF LARGE LANGUAGE MODELS (LLMS) STATEMENT
During the preparation of this manuscript, we utilized Large Language Models (LLMs), writing assistance tool. The use of L.MSs was limited to improving the grammar, clarity, and readability of the text. This includes tasks such rephrasing sentences for better flow, correcting spelling and grammatical errors, and ensuring stylistic consistency. The core scientific ideas, experimental design, results, and conclusions presented in this paper are entirely our own. LLMs were used to generate any of the primary scientific content or interpre- tations. The final version of the manuscript was thoroughly reviewed and edited by all authors, who take full responsibility for its content and originality.
B RELATED WORKS
Generalizable Novel View Synthesis. The ability to synthesize novel views from sparse set of images is a long-standing goal in computer vision. Pioneering approaches such as image-baser rendering (IBR) blend reference images based on proxy geometries (Debevec et al. 1996 Gortler
ct al.
1996). Early deep learning based methods predict blending weights or depth maps (Hedman
al.
2018
Choi et al., 2019). Generalizable neural radiance fields models like PixeINeRF (Yu
et al.
2021
and MVSNeRF (Chen et al.) 2024) pioneered the use of 3D-specific inductive biases.
Transformer-based Large Reconstruction Models. The Transformer architecture (Vaswani al., 2017). originally proposed for natural language processing, has demonstrated remarkable versatility and scalability across diverse domains (aa al. 2025b al Yang et al., 2025a 2023 Fan et al,, perception and spatial
2025). Transformer representations have been studied for geometry- reasoning under sparse multi- sensor settings Li et al. (2023)); Zhu et al. (2025); Jia et al. (2025c), suggesting that explicit handling of heterogeneous geometric cues often Ceneficial.Bulddiin its success, recent research has focused on developing Transformer bassdd Large Reconstruction Models (LRMs)(Hong et al. 2024 WWii al. 22224]] Li et al.) 20236 Gao al. [0024] You al.) 2025) to capture robust and generalizable 3D priors from extensive datasets. These models are trained on vast datasets learn generic 3D priors For instance, Triplane-LRM(LLi et al. 2023b) and GS-LRM (Zhang et al., 2024) learn to map sparse input images to explicit 3D representations like triplane NeRFs or 3D Gaussian Splatting primitives. Recent studies show that the choice of tokenization scheme can significantly affect the efficiency-quality trade-off in sequence modeling and generation Jia et al. (024a))
on
View Synthesis without Explicit 3D Representations. A compelling line of research explores the possibility of performing novel view synthesis purely "geometry-free" manner. Early attempts such as Scene Representation Transformers (SRT) (Sajjadi et al.) 2022). introduced the idea of using a Transformer learn a latent scene representation. Large View Synthesis Model (LVSM) (Jin et al., 2025) employs single, monolithic Transformer to process all input and target tokens.
Novel View Synthesis. Parallel to deterministic approaches, gen- erative models, particularly those based on diffusion (Watson et al. 2022 Zhou et al. 2025a). have gained traction for their ability to plausibly complete unobserved regions. A key challenge has been ensuring multi-view consistency. Solutions include test-time distillation into 3D representations like NeRFs (Poole et al. 2022; Wu et al.) 2023b), or building consistency directly into the model using video backbones and cross- view attention (Gao* et al. 2024
Generative and Diffusion--aan
Zhou et al. 2025a). This research
15

---

Published as a conference paper at ICLR 2026
area also includes lightweight methods like NVS-Adapter (amd Jinwoo Lee al. 2024 which adapts pre-trained 2D models for the single-view view NVS task. Related efforts also explore autoregres- sive scene generation in other 3D sensing modalities, highlighting the broader interest in scalable generative scene modeling /Zhou et al. (2025b).
C TRAINING AND IMPLEMENTATION DETAILS
Training Setup. We train Efficient-LVSSM with constant learning rate of 4e-4 with a warmup of 2500 iterations. Following LVSM (Jin et al. 2025). we use AdamW optimizer and the 31 and 32 are 0.9 and 0.95 respectively. We also employ a weight decay of 0.05 on all parameters of the LayerNorm layers. Unless noted, our models have 12 encoder layers and 12 decoder layers, which the same as LVSM.
-Speeic Schedules. For the object-level dataset, we use input views and 8 target views, training on 64 A100 80G GPUs We first train at resolution of 256, using batch size of 12 for 3 days, totaling 250k iterations. We then finetune the model at a resolution of 512 for days, with batch size of per GPU and 100k iterations, using learning rate of 10-5, For the scene- level dataset, we use 2 input views and 3 target views. Training begins at a resolution of 256 with batch size of 16 per GPU for days, completing 650k iterations. We then finetune the model resolution of 512 for 1 day, using a batch size of 4 and 200k iterations. In the ablation studies, we train and evaluate on the RealEstate 10K dataset using 2 input views and 3 target views. These models are trained on 2 A A100 80G GPUs for 10 hours, with the per-GPU batch size set as large as permitted by memory capacity. For the model-size ablations, training extended to 24 hours on GPUs.
REPA Distillation Details. We use the DINOV6-ViT-B/16 model (Siméoni et al. 2025) as the pre-trained teacher. We use the output features from the 8th transformer layer of DINOv3 as the distillation target. These teacher features are aligned with the output of specific layer our student model, which varies by its size: for main 12+12 layer models, we align with the 3rd layer's output. while for the smaller 6+6 layer models used in ablations, we align with the 2nd layer. The alignment is performed via 3-layer MLP projector and optimized using the Smooth L1 loss (Girshick 2015).
D ANALYSIS OF ALTERNATIVE ARCHITECTURAL DESIGNS
In this section, we evaluate our dual-stream co-refinement architecture by comparing it against sev- eral alternative designs. The idea of decoupling heterogeneous roles (e.g., context providers versus query executors) has been explored in prior work Hu et al. (2023); Lu et al. (2024) Wu et al. (2023a). Notably, the datapath of our architecture can, in principle, be reproduced using customized attention masks, and the separation of token roles (input target tokens) may be approximated by assign- ing distinct projection layers within the attention module. To provide a comprehensive analysis, we implemented and assessed such alternatives. The alternative architectures described below:
LVSM w/ Mask:
We add a custom attention mask on a single Transformer backbone to emulate the information flow of our model: input tokens are restricted self-attend only within themselves, while target tokens are permitted both self-attention and cross-attention over all input tokens.
LVSM w/ MMDiT-style: Inspired by multi-modal architectures (Esser et al., 2024). this variant uses shared Transformer block but introduces separate projection layers (W g: W W. W.)) and feed- forward networks (FFNs) for input and target tokens, allowing the model to handle their distinct characteristics.
LVSM w/ Mask + MMDiT-style: A hybrid design that combines the custom attention mask with the specialized per-token-tyee type projection layers within a single Transformer block.
We trained all variants on the RealEstate 10K dataset while keeping the parameter budget compa- rable as possible. Quantitative results are summarized in Table
The results clearly highlight the advantages of our dual-stream co-refinement architecture. Although the LVSM w/ Mask variant uses fewer parameters, it is more than 7x slower. This slowdown arises because irregular attention masks disrupt the highly optimized contiguous memory access patterns expected by underlying implementations: realizing any theoretical speedup would require custom CUDA kernels, undermining the simplicity and generality of the approach. Similarly, MMDiT-style
16

---

Published as a conference paper at ICLR 2026
Table 7: Comparison of Different Architectures.
Arch.
Parameters
PSNR
SSIM
LPIPS
Latency (ms)
GFLOPS
Memory
Corefinement
101M
26.02
0.8483
0,1481
17.58
647
1802
LVSM
25.24
0.8286
0.1545
74.45
3487
3114
LVSM mask
86M
24.13
0.7925
0.1793
125.13
1050
LVSM w/ MMDIT
164M
24.37
0.8021
0.1607
78.58
3487
3857
LVSM w/ MMDIT
164M
23.24
0.7601
0.1913
130.76
1050
6544
Table 8: Latency and Speedup with KV-Cache.
Number of Input Views
Ours w/ KV-Cache (ms) 4
LVSM Dec-Only (ms) 4 Specdup Factor 1
24.37
123.1
5.1x
8
28.62
286.0
10.0x
16
42.96
801.7
18.7x
32
72.84
2592
35.6x
103.26
5408
52.4x
64
138.43
9231
66.7x
variants incur significantly higher parameter counts and computational costs due to the duplicated projection and FFN layers for each token type.
This ablation study demonstrates that our dual-stream co-refinement architecture achieves an effec- tive balance of reconstruction quality, inference speed, computational efficiency, and implementation simplicity for novel view synthesis.
E DETAILS ON K KVCCACINGG
Many interactive applications require reusing computation across iterative queries, motivating cache-friene friendfy architectures and incremental inference pipelines that have been explored in other se- quential decision and closed-loop settings Li et al. (2025); Yang al. (2025b); Jia et al. (2024a b); You et al. (2024). In contrast to repeatedly recomputing context, our decoupled design enables KV-caching for input features and supports near-constant-cost updates when adding new inputs targets. As discussed in the main paper, our architecture's is compatibility with KV-caching key its efficiency. Figure I0 provides a visual workflow of this incremental inference process and contrasts it with the LVSM baseline.
F DETAILED MODEL ARCHITECTURE
Fig. 9 displays the detailed model architecture and the data path.
G LIMITATIONS AND FUTURE WORK
Despite the progress, we recognize several limitations that offer promising avenues for future re- search. A primary limitation, which our work shares with other large-scale feed-forward models, is the path to industrial-scale deployment. Although Efficient-LVSM makes substantial strides ducing computational costs, translating these large Transformer-ber -bssed architectures into production- level applications with stringent latency and memory constraints remains formidable challenge. In line with the perspective of the original LVSM paper, our model can be viewed powerful proof of concept that further validates potential of geometry-fres Transformer-basee novel view syn- thesis.
Bridging the gap between this academic proof of concept and widespread industrial adoption will likely require significant innovations beyond architectural design. Future work in this direction could explore techniques such as model compression, network quantization, and knowledge distillation to create smaller, faster versions of these models without substantial degradation in quality. We hopeful that our contribution in improving the baseline efficiency the ferw-forward paradigm serves beneficial step the path toward making these models practical for real-world use.
17

---

Published as a conference paper at ICLR 2026
Rendered Images
[vt, 3, H. W]
Unpatzhify (reshape) ...
Rendered
Input Tokens layer [vi,
P Patches
LayerNorm Linear Sigmoid
[vt, ps, p*p"3]
Target Tokens
layer [vt, ps, d]
Self-attention within views
Self-att within views
Self attention within views
Self-attention within views
Self attention within views
Input Tokens at layer k+1 [v),
Target Tokens at layer k+1 [vt. ps, d]
Layer k
Layer k
Split
Spii
Input Tokens [pstvl, d]
KV
Concatenation
Input Tokens [vl,
Updated Target Tokens [ps d]
Concatanation
Self attention within views
Self-attention within views
Input Tokens at ltlayer k [vi, ps,
Self-attention within views
Self attention within views
Self-attention within views
Target Tokens atlayer k ps, View Pachtes ps, p.
Embed by linear layer
View Pachtes p. p]
Embed by
Tinear layer
P Patchify
...
Patchify
Input Views &
P Plücker Rays [vt 9, H, W]
Plücker Rays [vl, 9, H, W]
Figure 9: Detailed Model Structure. ps = HW/p² denotes the number of patches in an image, where is the patch size. vi and vt represent the numbers of input and target views, respectively. When vi or ut appears in the first (batch) dimension, it indicates that the tokens from different views processed jointly batch.
Re-buikd Latent
Duplicatex N
N
Target View Latent
Target View Latent
Concatarate & Re-Encode
Target View Latent
Decoder
Encoder
Decoder
Decoder
Duplicate
Novel Impult
N
Input Views
Target Views
Input Views
Target Views
Concatenate
Target Views
KV Cache
Novel
Novel Imput
Length
Latent
Input
Novel Cost:
0(322) 0(4M2))
Novel Cost (
0(12)
a) Encoder-Decoderr LVSM
b) Decoder-only LVSM
c) Efficient LVSM
Figure 10: Efficient Incremental Inference with KV-Cache. Efficient LVSM saves computation when provided with novel inputs targets by caching the key and value for previous input views.
18


Reviews:
[{"review_id": "1Y02YNsRkm", "reviewer": "Reviewer_fYv1", "summary": "This study proposes Efficient-LVSM, which modifies the neural network architecture of LVSM to make its training and inference more efficient. The key modification is that Efficient-LVSM incorporates a cross-attention block, while stacking different self-attention blocks to extract query and key/value vectors from the target view and input views, respectively. By avoiding full self-attention between the input and target views, Efficient-LVSM reduces computational costs and enables feature caching of input views across different target views. Experimental results demonstrate the effectiveness of the proposed architecture.", "questions": "Q1. Positional embeddings are important for maintaining alignment between views, but there is no description provided in the paper. How are positional embeddings applied in the self-attention and cross-attention layers?\n\n\nQ2. Can the authors discuss the limitations of this study to help readers understand potential directions for future work?\n\nQ3. While keeping a single attention block, could we use different masking strategies instead of using separate modules? For example, a causal mask between input and target views could enable KV-caching of input views. Additionally, attention between input views could incorporate a masking scheme to prevent them from attending to each other for efficiency. We could also optionally use different projection layers while sharing a single attention block, as demonstrated in MMDiT [NewRef-2]. Although I agree that the proposed architecture achieves significant improvements in the experiments, I believe the design could be further simplified.\n\n\nQ4. Figure 9(c) shows that Efficient-LVSM achieves 2× faster training to reach the same performance as LVSM, but the training does not appear to have converged yet. Could the authors provide more GPU hours to ensure full convergence and compare training efficiency more fairly? In addition, was REPA used in this setting? A comparison without REPA could also provide a clearer understanding of how Efficient-LVSM achieves efficiency and effectiveness purely from its architectural design, as analyzed in Table 1.\n\nQ5. I wonder why the authors keep the generated target views independent. In addition, input views do not align their features with other input views. Is there any specific reason to restrict the receptive fields in this way?\n\nQ6. In Table (d), could the authors provide a comparison between the 12-layer self-attention model and the 12+12 Efficient-LVSM? I believe the 12+12 Efficient-LVSM does not actually contain 24 layers but 12, so comparing it with the 12-layer self-attention model would be a fairer setting.\n\nQ7. I initially expected that increasing the number of input views in the previous LVSMs would significantly increase GPU memory usage. However, Figure 9(a) shows that the GPU memory of the LVSM Decoder-Only model does not increase much as the number of input views grows. Could the authors elaborate on this result?\n\nQ8. How much does KV-caching improve inference speed as the number of input views increases?\n\n\n[NewRef-2] Esser, Patrick, et al. \"Scaling rectified flow transformers for high-resolution image synthesis.\" Forty-first international conference on machine learning. 2024.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "S1. Decomposing the full self-attention into different modules with cross-attention makes sense, as has been explored in various domains to design more efficient neural network architectures.\n\nS2. The experiments show significant improvements in novel view synthesis, while also enhancing inference efficiency.\n\nS3. The modified architectures can incorporate REPA to effectively train the hidden representations of input views.", "weaknesses": "W1.\nThe technical contribution is limited. Instead of using full self-attention across input and target views, employing cross-attention is a typical design choice for improving training and inference efficiency [NewRef-1].\n\nW2.\nDespite the performance improvements, Efficient-LVSM cannot address the fundamental limitations of LVSM. For example, its architecture cannot account for the alignments either between generated target views or within the input views. Therefore, the overall impact of this study is limited to being an improved version of LVSM rather than a fundamentally new approach.\n\nW3.\nThe experimental analysis could be strengthened by incorporating more baseline methods. Please refer to my detailed comments below.\n\nW4. [Minor points — not affecting the score] The paper writing should be improved.\n- Line 42: \"ability to to learn\" -> \"ability to learn\"\n- Line 92: input and target inputs share the same subscription $i$ , making the readers confused.\n- Need to use mathbf clearly. For example, $S_i$ is also a set of tokens, but it does not use the bold text type. In Eq. (1), why $R_i$ uses the bold type, while the other parts do not use.\n- Figure 1 uses a check box, but rendering quality or efficiency cannot be described with the check box. For example, we cannot say that the baseline has no efficiency or quality.\n- In Figure 2, what does the asterisk mean? I guess it describes a shared weight between modules, but it is neither a common way to describe it nor explain the details.\n- Line 133 -- $p$ is not defined. $N$ is the number of input views, but $n$ is used for both input/target views.\n- Eq. (2) uses $l$ without any definition.\n\n\n[NewRef-1] Jeong, Yoonwoo, et al. \"NVS-Adapter: Plug-and-play novel view synthesis from a single image.\" European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "n5vB2v2hSm", "reviewer": "Reviewer_tC9E", "summary": "- This paper proposes Efficient-LVSM, a transformer-based large view synthesis model designed to overcome the inefficiencies of the original LVSM architecture. \n\n- The key idea is to decouple the input-view encoding and target-view generation using a dual-stream co-refinement mechanism, combining intra-view self-attention for inputs and self-then-cross attention for targets. \n\n- The approach enables linear complexity in the number of input views, incremental inference with KV-cache.\n\n- Efficient-LVSM outperforms state-of-the-art LVSM by 0.9dB PSNR on the RealEstate10K benchmark with 50% training time and achieves 2−4 times speed acceleration in terms of both training iteration and inference.", "questions": "The paper is well presented; I don't have further questions.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- The paper provides a systematic analysis of LVSM’s inefficiencies and derives a principled redesign via a decoupled encoder-decoder. The KV-cache design enabling incremental inference is a noteworthy contribution for real-time or interactive view synthesis, rarely explored in prior feedforward NVS models.\n\n- Efficient-LVSM achieves state-of-the-art reconstruction quality on both scene-level (RealEstate10K) and object-level (GSO/ABO) benchmarks. The reported 0.9 dB PSNR gain, 4× inference speed-up, and 50 % reduction in training time represent a strong improvement over prior LVSM baselines\n\n- The experiments are thorough: comparisons across datasets and baselines, scaling trends, ablations on architectural variants (self vs cross vs co-refinement), REPA distillation effects, model size, convergence curves, and zero-shot generalization.\n\n- The paper is well written and includes complete training details, REPA settings, and commitments to code release.", "weaknesses": "- The dual-stream co-refinement design is highly similar in spirit to the MM-DiT block in terms of architecture introduced by Stable Diffusion 3 (2024). The authors are encouraged to cite MM-DiT and clarify how Efficient-LVSM extends this pattern to the feedforward NVS setting.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4zUyVZRt1L", "reviewer": "Reviewer_WANz", "summary": "This paper tackles feed-forward novel view synthesis task. It builds upon the framework of LVSM [1], which use a full-attention decoder-only transformer to render novel views conditioned on posed input views and target view plucker rays.  The key architecture change proposed by the authors are to replace the full attention to intra-image self-attention and cross-attention between target views and input views.  This change has two benefits:  1. When rendering new novel views, you don't need to recompute the key-value cache of input views. This also opens up application in incremental inference. 2. with above benefits, the performance (rendering quality) improves on Objecverse and Rel10K dataset, even with reduced training time.   The author also ablated with REPA showing that it can improve the PSNR. \n\n\n\n[1] LVSM: A Large View Synthesis Model with Minimal 3D Inductive Bias", "questions": "Kind of minor, but for task like novel view synthesis, showing some video comparison would be great!", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Clearly justified and very reasonable architectureal change.  The full-attention used by original LVSM restricts its effiency in lot of usecases. And when comparing with the encoder-decoder version of LVSM, the author identified the major problem:  we need to use key-value cache of all layers!      \n2. Strong empirical results. The experiment results on Objverse and Rel10K are quite strong, with much better rendering quality and less training time. \n3. The author shows a very interesting study about repa loss (distill from DINO v3). I original thought such semantic loss would not be useful for novel view synthesis task, but seems that it's quite helpful! This is very interesting.", "weaknesses": "I do have a few comments. I think the authors should list more training details for their methods and their ablation experiments. The batch size, and total number of training iterations. I think it's missed.  \n\nFor training batch sizes, there are some tiny but important details, the original LVSM need to repeat the batch to make sure that each pass of the model only contains one target view, and this is one of the core-reason that the original decoder only LVSM is expensive in training and inference.  So highlighting this training details difference, and show that how many input views and target views totally being used during training is important.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EKcXoVNoJi", "reviewer": "Reviewer_ygh6", "summary": "The work proposes an improvement to LVSM, an ICLR 2025 oral publication. The proposed method, called efficient-LVSM, uses a detached attention mechanism to enhance efficiency and final performance. The architecture separates the input view encoding from the output view encoding, except for layer-wise connections within the same layer. There are additional tricks, such as KV-cache, feature distillation, and alternating self-attention and cross-attention. The results are evaluated against a few other-shot novel-view synthesis methods. The additional detailed efficiency analysis and ablation are performed against LVSM.", "questions": "- Figure 1 is hard to interpret, especially the items in the Table. Some of the attributes need to be detailed either in the caption or in the text. For example, what are \"spatialized pathways\"?\n\nMinor comments\n- line 042: \"to to\" -> \"to\"t.\n- In Figure 2, I believe there are \"M\" decoders instead of \"N\" decoders..?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- Good results\n- Detailed explanation about the components", "weaknesses": "- The work heavily relies on LVSM. In addition to final quality, the efficiency assessment needs to be compared with other methods. \n\n- The figures about the attention mechanism play an important role in understanding the main idea of the paper. However, they are somewhat repetitive and could be condensed into fewer figures—namely, Figures 1, 2, 3, and 6 (and Table 1 as well). I suggest reserving additional space to lay out the results better. The current layout is not very effective. The wrap-around text is hard to follow, and the figures and tables are referenced far from their locations. Some detailed results can be deferred to the appendix to improve the flow and completeness of the descriptions.\n\n- While the introduction describes the main contribution as decoupling the encoder and decoder with an efficient attention mechanism, the work is composed of multiple techniques from existing works, with too many subsections in both Section 2 and the results. While they are effective, the novelty can feel incremental, and the description is dispersed across several succinct sections, making it hard to get the whole picture.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)