UniSplat: Unified Spatio-Temporal Fusion via 3D Latent Scaffolds for Dynamic Driving Scene Reconstruction
Authors: Chen Shi
Shaoshuai Shi
Xiaoyang Lyu
Chunyang Liu
Kehua Sheng
Bo Zhang
Li Jiang
Venue: ICLR 2026 Poster (2026)
Keywords: Autonomous Driving
Feed-Forward Scene Reconstruction
3D Gaussian Splatting
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Feed-forward 3D reconstruction for autonomous driving has advanced rapidly, yet existing methods struggle with the joint challenges of sparse, non-overlapping camera views and complex scene dynamics. 
We present UniSplat, a general feed-forward framework that learns robust dynamic scene reconstruction through unified latent spatio-temporal fusion. UniSplat constructs a 3D latent scaffold, a structured representation that captures geometric and semantic scene context by leveraging pretrained foundation models. To effectively integrate information across spatial views and temporal frames, we introduce an efficient fusion mechanism that operates directly within the 3D scaffold, enabling consistent spatio-temporal alignment. To ensure complete and detailed reconstructions, we design a dual-branch decoder that generates dynamic-aware Gaussians from the fused scaffold by combining point-anchored refinement with voxel-based generation, and maintain a persistent memory of static Gaussians to enable streaming scene completion beyond current camera coverage. Extensive experiments on real-world datasets demonstrate that UniSplat achieves state-of-the-art performance in novel view synthesis, while providing robust and high-quality renderings even for viewpoints outside the original camera coverage.

Published as a conference paper at ICLR 2026
UNISPLAT: UNIFIED SPATIO-TEMPORAL FUSION VIA 3D LATENT SCAFFOLDS FOR DYNAMIC DRIVING SCENE RECONSTRUCTION
Chen Shi
Shaoshuai Shi
Xiaoyang Lyu Bo Zhang*
Chunyang Liu
Kehua Sheng*
Li Jiang"!
TTee Chinese University of Hong Kong, Shenzhen
"The University of Hong Kong
"Voyager Rescarch, Didi Chuxing
Project Page:
ABSTRACT
Feed-forward 3D reconstruction for autonomous driving has advanced rapidly, yet existing methods struggle with the joint challenges of sparse, non-overlapping camera views and complex scene dynamics. We present UniSplat, general feed-forward framework that learns robust dynamic scene reconstruction through unified latent spatio-temporal fusion. UniSplat constructs 3D latent scaffold, a structured representation that captures geometric and semantic scene context by leveraging pretrained foundation models. To effectively integrate information across spatial views and temporal frames, we introduce an efficient fusion mech- anism that operates directly within the 3D scaffold, enabling consistent spatio- temporal alignment. To ensure complete and detailed reconstructions, we design dual-branch decoder that generates dynamic-aware Gaussians from the fused scaffold by combining point-anchored refinement with voxel-based generation, and maintain persistent memory of static Gaussians enable streaming scene completion beyond current camera coverage. Extensive experiments on real-world datasets demonstrate that UniSplat achieves state-of-the-art performance novel view synthesis, while providing robust and high-quality renderings even for view- points outside the original camera coverage.
INTRODUCTION
Replicating 3D scenes from urban driving sequences has emerged a core capability for au- tonomous systems, supporting simulation (Cao et al., 2025; Yang et al., 2023; Tonderski et al., 2024), scene understanding (Huang et al., 2024b; 2025; Yan et al., 2025a), and long-horizon plan- ning (Murai et al., 2025). Recent advances in 3D Gaussian Splatting (Yan al., 2024; Jiawei al., 2025; Kerbl et al., 2023) have demonstrated impressive rendering efficiency and fidelity. However, these methods typically assume substantial viewpoint overlap among input images and rely per- scene optimization, which limits their upplicability in real-time sal-time driving scenarios.
To enable faster inference, feed-forward reconstruction methods have emerged to synthesize novel views in single forward pass (Xu et al., 2025; Chen et al., 2024: Zhang et al., 2025; Iu et al., 2024). These methods typically encode inter-view correlations within the image domain via cross-attention or by constructing a multi-view stereo (MVS) cost volume, and subsequently decode the Gaussian primitives from the resulting fused features. Notably, the choice of fusion strategy is crucial, as significantly impacts the final rendering quality. EvolSplat (Miao et al., 2025) integrates multi-frame geometric information from front-view monocular sequences using 3D-CNN, but ignores semantic fusion and lacks mechanisms for dynamic handling. Meanwhile, Omni-Scen (Wei et al.. 2025) leverages Triplane Transformer for strong multi-view fusion but does not incorporate temporal
Corresponding author.

---

Published as a conference paper at ICLR 2026
aggregation and is constrained by coarse- srans-grriied 3D details. Despite these advances, robust construction in urban driving scenarios remains challenging, particularly in maintaining a unified latent representation that evolves smoothly over time, handling partial observations, occlusions, and dynamic motion, and efficiently generating high-fidelity Gaussians from sparse inputs.
To address these challenges, we propose UniSplat, general feed-forward framework for dynamic scene modeling from multi-camera videos. The core insight of UniSplat is to construct a unified 3D scaffold that fuses both multi-view spatial information and multi-frame temporal information. This scaffold facilitates geometric and semantic contextual interaction in 3D space, supports efficient long-term terg-term information integration and dynamic modeling, and enables effective decoding of Gaus- sian primitives. By preserving and fusing essential information, it ensures coherent and consistent scene reconstruction over time.
Specifically, the UniSplat framework follows a three-stage pipeline. First, we construct an ego- centric 3D scaffold by feeding multi-view images pretrained geometry foundation model and a visual foundation model, encoding both geometry structure and semantics cues into sprase 3D fea- ture volume. Second, we perform spatio-temporal fusion by integrating multi-view spatial context within the current frame's scaffolds and fusing historical scaffolds into current scaffolds via ego- motion compensation, yielding a temporal-enhaneee scene representation. Third, we decode the fused scaffold into Gaussians via baal-branchhhe strategy: one branch predicts Gaussians at sparse point locations for fine-grained detail while the other directly generates new Gaussians from voxel centers complement point anchor predictions. Each Gaussian is assigned dynamic probability score to identify static content, allowing us to maintain a memory bank of persistent static Gaussians across frames for long-term scene completion.
We evaluate our method on the Waymo Open dataset (Sun et al., 2020) and NuScenes (Caesar al., 2020) dataset, which present dynamic street scenes with complex environmental conditions and limited overlap for multi-camera images. Experimental results demonstrate that our approach achieves state-of-the-arr performance across both datasets in input-view reconstruction and novel- view synthesis. Notably, with the help of temporal memory, our model exhibits strong robustness and superior rendering quality when synthesizing views outside the original camera coverage.
In summary, our main contributions are follows:
We introduce UniSplat, a novel feed-forward framework for dynamic scene reconstruction from multi-camera videos via unified 3D latent scaffold.
We design a novel scaffold-based fusion mechanism that supports unified spatio-temponn aligmment and progressive scene memory integration.
We propose a dual-branch Gaussian generation mechanism with dynamic-ave aware filtering, enabling fine-grained and complete rendering and memory-based scene completion.
Comprehensive experiments on two large-scale driving datasets demonstrate that UniSplat significantly outperforms state-of-the-an arrt feed-forward reconstruction methods, with gen- eralization capability for challenging views outside the observed camera frustums.
2 RELATED WORK
Neural 3D Reconstruction. The field of neural 3D reconstruction has witnessed remarkable progress, largely driven by Neural Radiance Fields (NeRF) (Mildenhall al., 2021) and, more recently, 3D Gaussian Splatting (3DGS) (Kerbl et al., 2023). NeRF represents scenes as continu- ous volumetric functions, achieving high-fidelity renderings but incurring substantial computational costs. Subsequently, 3DGS introduced explicit paint-based representations with highly efficient rasterization, enabling real-time rendering. Despite the impressive performance of NeRF, 3DGS, and their extensive variants (Hu et al., 2023; Xu et al., 2022; Müller et al., 2022: Hu et al., 2023; Yu et al., 2024; Yang et al., 2025a), these methods are usually limited by the reliance on dense input views and costly per-scene optimization, thereby restricting their scalability. Alternatively, feed-forward methods tackle this challenge by learning generalizable scene priors from large-scale datasets during training, allowing for immediate reconstruction from sparse observations at inference time. MuRF (Xu et al., 2024) employs target view frustum volumes for radiance field reconstruction. PixeISplat (Charatan al., 2024) and Splatter Image (Szymanowicz al., 2024) predict per-pixel
2

---

Published as a conference paper at ICLR 2026
3D Gaussians from image features, while MVSplat (Chen et al., 2024) leverages cost volumes for geometric consistency and DepthSplat (Xu et al., 2025) integrates features from pre-trained monoc- ular depth models to improve robustness. However, these approaches still face significant challenges in complex urban driving scenarios, where minimal overlap among surround-view cameras compro- mises multi-view correspondence and the presence of highly dynamic objects complicates temporal aggregation. Beyond these explicit geometric methods, token-based transformers (Jin et al., 2025) and diffusion-based models (Gao et al., 2025; 2024) have also been explored for generalizable view synthesis without explicit reconstruction, but they typically suffer from high rendering costs or hal- lucinate content that inconsistent with the input context. In this work, we develop a ferwafo framework to reconstruct complete driving scenes from sparse views while effectively leveraging multi-frame information.
Driving Scene Reconstruction with 3D Gaussians. Leveraging advances in 3D Gaussian Splat- ting, several works (Chen et al., 2023; Huang et al., 2024a; Zhou al., 2024b; Yan et al., 2024; Zhao et al., 2025; Yan et al., 2025b; Fan et al., 2025; Jiawei et al., 2025) specialize in driving scenes (Pei et al., 2025a;b), focusing on 3D or 4D reconstruction within individual scenes through offline op- timization. In parallel, generalizable methods have emerged. These approaches (Tian et al., 2025; Lu et al., 2024) typically employ depth networks to determine Gaussian primitive positions a feed-forward manner and predict per-pixel Gaussians along camera rays. To enhance global consis- tency and completeness, several techniques incorporate 3D spatial representations. EVolSplat (Miao al.. 2025) directly accumulates depth across multiple frames and leverages 3D-CNNs to refine Gaussian geometry. Omni-Scene (Wei et al., 2025) transforms multi-view image features into Tri- Plane representations and decodes voxel-anchored Gaussians to complement pixel-based estimates. SCube (Ren et al., 2024) constructs detailed sparse-voxel scaffold via hierarchical voxel latent diffusion model. However, these methods often focus on static or fingle-frame reconstruction and struggle to simultaneously handle multi-view fusion and dynamic multi-frame aggregation. More recently, unsupervised 4D reconstruction approaches have been proposed, but they either lack effec- tive 3D alignment for complex scene flow estimation (Yang et al., 2025c) require LiDAR initial- ization (Wang et al., 2025b). To counter these challenges, we propose UniSplat, a novel framework that unifies multi-view fusion and dynamic multi-frame aggregation within 3D latent scaffold.
3D Geometry Reconstruction. End-to-end, data-driven pipelines that reconstruct scene geom- etry directly from images have progressed rapidly. DUSt3R (Wang et al., 2024) pioneers transformer- based framework that predicts 3D point maps from uncalibrated image pairs. Subse- quent works (Wang et al., 2025f;a; Yang et al., 2025b; Wang al., 2025c; Chen et al., 2025; Xiao et al., 2025) extend this paradigm to arbitrary multi-view inputs and scale up both training data and model capacity, achieving state-of-the-art reconstruction accuracy with strong generalization across diverse scenes. However, these methods generally struggle with poor texture representation and en- counter multi-view misalignment under minimal overlap, limiting novel view rendering quality. In this work, we employ these 3D foundation models to obtain geometry initialization from images, and then perform 3D alignment and fusion in the learned latent scaffold.
3 UNISPLAT
UniSplat operates on a continuous stream of multi-camera frames, maintaining a unified 3D latent representation of the scene that evolves over time. As shown in Fig. 1, each time step begins with 3D scaffold construction from multi-view images (Sec. 3.2), producing a set of 3D voxels (the latent scaffold) that encodes the scene's geometry and semantics in an ego-centric coordinate frame. We then perform a unified spatio-temporal fusion, integrating information across views within the current scaffold and aggregating it with the latent scaffold from the previous time step (Sec. 3.3). Finally, we achieve dynamic-aware Gaussian generation (Sec. 3.4) through a dual-branch decoder that estimates dynamic-aware Gaussian primitives from both points and voxels, while maintaining temporal memory bank that accumulates static Gaussians over time to address incomplete scene coverage caused by sparse camera inputs and limited fields of view.
3.1 PRELIMINARY
3D Gaussian Splatting (Kerbl et al., 2023) represents a scene as a collection of 3D Gaussian primitives g
{Gi) N. Each primitive G, is defined by tuple of learnable parameters
3

---

Published as a conference paper at ICLR 2026
8, {Hs C, Σ. cr). representing its 3D center position, opacity, covariance matrix, and color coefficients, respectively. To render an image from a target viewpoint, these 3D Gaussians are pro- jected onto the 2D image plane and blended using diffcrentiable alpha compositing. Specifically, for a particular pixel, the color contribution C from all Gaussians whose projections cover that pixel is:
C= Σ C II (1-aa)) az), SEN
where N is the set of Gaussians overlapping the pixel, sorted by depth. Beyond simple color render- ing, several works (Zhou et al., 2024a: Zuo et al., 2025) augment Gaussians with additional param- eters, which can be rendered into 2D feature map using the same alpha compositing mechanism, enabling the distillation of knowledge from 2D foundation models. Inspired by this extensibility, we introduce learnable dynamic attribute for each Gaussian to explicitly disentangle scene dynamies.
(a) 3D Scaffold Construction
(b) Unifled Spatio-Temporal Scaffold Fusion
(c) Dynamic-aware Goussian Generation
Dand Beo--Beeschh Gasovaan Deceska
Dynamic Ronderisg
Geometery
Tengeral Fasien
Model
10.0 P.
Pitsi
Vi
Deuader
Uy Hiyeine
Vistad Fouedaaiin Model
[1V. P.O))
Specia Edhaxed
Spuzio lemperal Frand Scafiste
Sceffill 8,
Voudli)
Becade
St
Currenr Jipar
Streassing Monury
Stree
Mi-1
Dysentiaavare Gamila Completion
Uptaate
Mrli-Carum Semorddiiio Images
Terpperl Pasell Seufishl
State Merocy Gunrin
Figure 1: Overview of UniSplat. Given multi-camert images from vchicle-mounted cameras, UniSplat leverages foundation models to construct geometry-semanee aware 3D latent scaffolds, where unified spatio-temporal fusion is performed. From this scaffold, dual-branch decoder gen- erates dynamic-awan Gaussian primitives using both point anchors and voxel centers, with dynamic filtering maintaining persistent memory of static scene content. The red boxes highlight dynamic car that is filtered out in our memory module (best viewed when zoomed in).
3.2 3D SCAFFOLD CONSTRUCTION
Constructing an accurate 3D scaffold from sparse, minimally overlapping camera views is a primary challenge in multi-view reconstruction for driving scenes. To address this, we harness the power of geometry foundation models to infer coherent 3D structure from multi-view images in one forward pass. We then enrich this 3D geometric scaffold with semantic information from a visual foundation model. This process yields latent scaffold representation in the ego-centric coordinate frame of the vehicle, which provides a strong basis for subsequent spatio-temporas fusion.
Metric-Scale 3D Geometry Generation. Given synchronized multi-view images I, = } Nan from a multi-camera rig, we apply state-of-the-art feed-forward multi-view geometry foun- dation models (e.g., Wang et al. (2025a;D) to directly predict a dense 3D point map Piui e Noa Hang Wi., where each pixel is associated with 3D coordinate inferred jointly from all views. Unlike view depth estimation and late fusion, this unified approach leverages learned multi-view correspondences to generate coherent scene-level point cloud. However, such predic- tions often suffer from scale ambiguity, which problematic autonomous driving. Thus, we introduce an auxiliary scale alignment branch: a small MLP predicts per-camera scale factors from the pooled geometry features:
jk-l
7 MLP(AAgPool (H, W3)) R Na..
where F8tD denotes the hidden feature maps from the geometry model and AvgPool(-. {H, W)) represents the average pooling over the height and width dimensions for each view. The seale prediction is supervised by minimizing the error between 7 and the optimal seale vector computed using the ROE solver (Wang et al., 2025d) with LiDAR point references. Applying 7 to pinit yields metric-consistent point cloud P, as the geometric foundation of our scaffold.
Scaffold Construction with Geometric-Semantie Context. As the generated P, is an unstruc- tured point set, we organize these points into sparse voxel grid and fuse geometric and semantic
4

---

Published as a conference paper at ICLR 2026
features to create the 3D latent scaffold. To achieve this, we first extract semantic-awar 2D features Fi from the imput views using a visual foundation model (Oquab et al., 2023), and fuse them with geometric features F to obtain unified multi-view feature map F,, We then voxelize the point cloud into N, voxels within an ego-centric cuboid [Pmin E R Pmax E R$] covering the surround- ing scene. The volume is partitioned into voxels of size e, and only voxels containing points considered valid. Specifically, for each voxel i, we compute its coarse geometric voxel feature v as the average of the coordinates of points j E In that lie in that voxel:
iiii ΣJEI, Pij
i (L, Nob.
(3)
EjEr, 1
where I; is the index set of points within i-th voxel. Next, enrich the voxel with geometric- semantic context, we project each voxel center into the input views and sample the corresponding features from Fr, which then concatenated with the initial voxel feature vinit The resulting 3D scaffold S of the scene is formally defined as a set of these voxels:
S. {(v: C. Pj )
where C, is the feature dimension, represents the voxel feature encoding both geometric and semantic context, and p, denotes the corresponding voxel center that preserves explicit 3D structure.
3.3 UNIFTED SPATIO-TEMPORAA SCAFFOLD FUSION
A key advantage of our scaffold representation lies in its inherent structure, which encodes explicit 3D geometry within unified ego-centric space. This design enables contextual interaction in the unified 3D space, supporting direct and efficient spatio-temporal fusion across multiple views and temporal frames within a single scaffold representation.
Spatial Scaffold Fusion. Unlike traditional approaches Chen et al. (2024); Xu al. (2025) that fuse spatial information across views in 2D space using image-level cross-attention, which is often hindered by limited overlap between views, we perform spatial fusion directly in the 3D scaffold space. In this representation, spatially corresponding information from different views is naturally aligned in 3D space. Specifically, we employ a sparse 3D U-Net & to integrate multi-view features, producing a spatially-enhaneee scaffold representation s
8:00 ((S.).
(5)
Temporal Scaffold Fusion. Instead of processing historical raw images as in existing works (Lu al., 2024; Tian et al., 2025), we integrate temporal cues directly within the scaffold representation in a streaming manner. Given the previous fused latent scaffold features Sfuned from a streaming memory, we first warp its voxel centers into the current frame's 's coordinate system using the known ego-pose TÉ and their features are tagged with time-step embedding to distinguish them from current observations. We then merge the transformed previous scaffold Stmel with the current scaf- fold S via element- weseentwwise addition any overlapping voxels, and simply union the features for non-overlapping regions. We denote this operation as a sparse tensor addition:
Sfused = 0000 + Warp(Sffud fused Th-1)
(6)
where @ denotes sparse tensor addition that aggregates features at overlapping voxel locations while preserving non-overlap features from both sparse tensors. The resulting tensor Sfused is further refined by lightweight sparse convolutional network to capture complex temporal dependencies and is cached back into the streaming memory maintain long-term temporal information.
3.4 DYNAMIC-AWARE GAUSSIAN GENERATION
Building upon the spatio-temporally fused scaffold S(used generate set of 3D Gaussian prim- itives via a dual-branch decoding strategy, yielding primitives that explicitly disentangle static and dynamic scene components, which enables progressive scene completion over time.
Qual-Branch Gaussian Decoder. Our Gaussian decoder comprises two complementary branches that jointly enhance reconstruction fidelity and completeness. The point decoder branch focuses
5

---

Published as a conference paper at ICLR 2026
on preserving fine- fineggrained geometric details by leveraging the point-level anchors from the recon- structed metric-scale point map Pt. For each point Pt, E Pt, we locate its voxel coordinate in the scaffold and retrieve the corresponding latent feature from S as:
Pt. - Paain
Retrieve
Lused
where [.] denotes the voxel indexing operation. I point falls outside the scaffold's spatial extent, zero-padding is applied. Since each point Pt, maintains one-to-one correspondence with its source pixel, we additionally sample 2D image feature f2. for each point from the multi-view viet feature maps Fr. These features are concatenated to predict the Gaussian primitives via an MLP:
{AAtii OG, E,, Cj d = MLP((EE. JF:l),
(8)
where Ays; denotes the Gaussian's offset from the point anchor, and de E is a learned dynamic score indicating motion likelihood. This branch yields detailed set of Gaussians denoted as g.
The voxel decoder branch complements the point-based decoding by directly predicting new Gaus- sian primitives from voxel-level scaffold features, effectively filling in sparsely covered regions and enhancing the scene completeness. For each voxel Stused, we adopt compact MLP produce sets of Gaussian parameters (as in Eq. 8) per voxel. The center of each Gaussian is derived by adding the predicted displacement to the voxel center, forming the set Groxel The complete reconstruction at time / is then given by G3 = gpsint U grosel
Dynamic-aware Gaussian Completion. To enhance temporal consistency and alleviate occlusion- induced sparsity, we introduce a memory mechanism that maintains accumulated static Gaussians over time. Each Gaussian primitive is associated with a dynamic attribute d,, enabling motion-awe aware filtering. Given the static memory M,-1 from the previous frame, we transform it into the current ego-centric coordinate system and perform view filtering to remove Gaussians visible in the current field of view. The resulting filtered memory Mi is then fused with the current reconstruction:
g gromplete G. M
where g gromplete provides comprehensive scene representation that fills in the blind spots of the current frame's reconstruction. Finally, the memory is updated by retaining static Gaussians from the current frame:
M, M, u (G, e Ge d ra), ie Ng.
(10)
where тa score threshold, and Ng. is the total number of current Gaussians. This streaming mechanism enables temporally persistent reconstruction while suppressing dynamic artifacts.
3.5 TRAINING OBJECTIVE
The model is optimized via a composite loss function defined over the rendered outputs from G;:
L= Σ (Ai + >ippippss + >z + >4
Σ
A O B
(11)
VEVape
where Linee and Cipps are the MSE reconstruction and LPIPS perceptual losses (Zhang et al., 2018) between rendered and ground-truth images for view v, L'ym is the cross-entropy loss between dered dynamic scores and ground-truth dynamic segmentation masks, and L is smooth-L1 loss for scale supervision. Vinpui refers to the set of input camera views at time and Vnovel denotes novel viewpoints at time ( 1. The operator background mask B" excludes dynamic regions to prevent optimization instability. Further details regarding dynamic rendering are provided in Appendix A.1.
denotes element-wise multiplication, where the
EXPERIMENTS
4.1 EXPERIMENTAL. SETTINGS
Datasets and Metrics. We conduct experiments on two large-scale autonomous driving bench- marks: Waymo Open (Sun et al., 2020) and nuScenes (Caesar al., 2020) datasets. The Waymo

---

Published as a conference paper at ICLR 2026
Table 1: Quantitative results the Waymo Dataset. The best results marked in bold and
underlined entries indicate second-place performance. * Evaluation conducted on front 3 views
only. t: Results obtained using optimal scale alignment.
Reconstruction
Novel View Synthesis
Method
Views
PSNR
SSIM
I.PPSS
PSNR
SSIM
LPIPS.]
EvolSplat (Miao al., 2025)
Front
23.35
0.70
0.29
UniSplat
Front
28.93
0.86
0.18
27.34
0.80
0.20
DriveRecon (Lu et al., 2024)
Multi
23,86
0.72
0.33
17.32
0.58
0.53
MVSplat (Chen et al., 2024)
Multi
24.94
0.80
0.23
22.04
0.68
0.34
DepthSplat (Xu et al., 2025)
Multi
25.38
0.76
0.26
23.86
0.70
0.31
UniSplat
Multi 28.56
0.83
0.20
25.12
0.74
0.27
UniSplati
Multi 29.58
0.86
0.17
25.98
0.76
0.24
Table 2: Quantitative results on the nuScenes Dataset. We highlight best results in bold and second- place results with underlines.
*: reported by Wei et al. (2025).
Method
PSNRT
LPIPS SSIM
PixelSplat" (Charatan et al., 2024)
21.51
0.616
0.372
MVSplat" (Chen et al., 2024)
21.61
0.658
0.295
Omin- Scene (Wei et al., 2025)
24.27
0.736
0.237
UniSplat
25.37
0.765
0,246
Open dataset includes 798 training and 202 validation sequences, with all sequences approximately 20 seconds long and captured at 10Hz using five cameras. For nuScenes, which provides six view images per frame, we adopt the strategy of Wei et al. (2025) and partition scenes into equally spaced bins along the vehicle trajectory, yielding 135,941 training and 30,080 valida- tion bins. Each bin consists of multiple sequential frames, and the central frame serves as the input.
To measure visual quality, we adopt standard image quality metrics including PSNR, SSIM (Wang et al., 2004), and LPIPS (Zhang et al., 2018). Following Yang et al. (2024); Lu al. (2024), the Waymo benchmark evaluates two tasks: reconstruction, for which images current timestep serve as targets, and novel view synthesis, which synthesizes images at the subsequent timestep + 1. For nuScenes, consistent with Wei et al. (2025), evaluate target views consisting of the first, last, and central frames of each bin.
Implementation Details. For our 3D scaffold reconstruction, employ frozen pretrained ge- ometry transformer T² (Wang et al., 2025f) for initial geometry generation and a pretrained DI- NOv2 ViT-small backbone (Oquab et al., 2023) for semantic feature extraction. The scaffold is built within real-world volume of [-72m. -72m, -4m, 72m, 72m, 12m], using an initial voxel size of 0. (0.1m, 0.2m). Scaffold spatial fusion is performed using sparse 3D U-Net with a maximum
downsampling factor of 8x, while the temporal fusion employs separate sparse 3D U-Net with a maximum downsampling factor of xx. the Gaussian decoding stage, the second branch generates g 4 primitives per voxel, and the dynamic attribute threshold for streaming scene completion is set to та = 0.7. We adopt image resolutions of 350 518 for the Waymo dataset and 224 406 for the nuScenes dataset. All models are trained using the Adam W optimizer (Loshchilov & Hutter, 2019) on 16 NVIDIA H20 GPUs with a total batch size of 32. For the training objective, we set A1=1.0, >==0.01, A0=0.11, and =000.2 Additional implementation details are provided in Appendix A.I.
4.2 MAIN RESULTS
Waymo. We compare UniSplat against ofate-of-the-art sparse-view reconstruction methods, includ- ing MVSplat (Chen et al., 2024), DepthSplat (Xu et al., 2025), EvolSplat (Miao et al., 2025), and DriveRecon (Lu et al., 2024). For the general methods MVSplat and DepthSplat, we retrain them on the Waymo Open Dataset using their official codebases. For driving-specifs methods EvolSplat and DriveRecon, we conduct evaluation on our validation scenes and resize their outputs to match the resolution for fair comparison. The quantitative results are summarized in Table 1. UniSplat consistently outperforms all baselines across every metric for both input view reconstruction and novel view synthesis. The qualitative comparisons DepthSplat struggle to reconstruct fine geometric details and exhibit noticeable artifacts, especially in overlapping regions between adjacent cameras. In contrast, method produces visually coher-
shown in Figure 2. Notably, MV and

---

Published as a conference paper at ICLR 2026
ent and high-quality results. We also report variant (denoted by t), in which per-camera scales are set to optimal values derived from LiDAR pointmap, leading to additional improvements.
NuScenes. Following Wei et al. (2025), evaluate UniSplat the nuScenes benchmark under the same protocol. As shown Table 2, UniSplat surpasses the previous state of the art, Omni- Senn-Seenn, achieving 25.37 PSNR (+1.10dB). The qualitative comparisons provided in Appendix A.3.
Dynamic-aware Gaussian Completion. UniSplat predicts per-Gaussian dynamic attributes, en- abling the progressive construction of the in Figure 3, the top section presents a baseline without dynamic filtering, where ghosting artifacts arise from accumulated dynamic objects. In contrast, our approach effectively completes missing regions while suppressing such artifacts, As illustrated in the bottom section, UniSplat successfully completes unobserved areas arising from two typical cases: limited 360 coverage Waymo's 's five- camera setup and cross-camest blind spots. Moreover, we can observe our model clearly separates dynamic vehicles from parked ones, demonstrating its effective use of temporal context.
during inference without manual labels. As shown
STOP
<TOP
STOP
STOP
Ground Truth
Ours
DepthSplat
MVSplat
Figure/ 2: Qualitative comparisons on the Waymo dataset. Our method yields more detailed and consistent geometry than existing works. Red boxes indicate artifacts. Best viewed zoomed in.
4.3 ABLATION STUDY
In this section, conduct ablation studies the Waymo Open Dataset (Sun et al., 2020) to investi- gate the individual components of our framework, with focus on novel view synthesis performance. For efficiency,
subsample the first 20% of frames from each sequence and apply optimal scale alignment to the point map to accelerate model convergence. All models are trained for 20 epochs with batch size of 32 on 16 GPUs.
Ablation on Geometric and Semantic Features in ScafFold. Table investigates the contribu- tion of geometric and semantic features from foundation models to the scaffold representation. The absence of semantic features causes severe decline in LPIPS, increasing the error by 0.05, which can be attributed the fact that LPIPS measures perceptual similarity using deep semantic representations. In contrast, the 2nd and 3rd rows show that performance gap is less pronounced when only DINO features are used, suggesting that current large-scale pretrained 2D foundation model (Siméoni et al., 2025) may implicitly capture certain geometric priors.
Analysis of Temtiol Fusion. We ablate the effects of our spatial and temporal scaffold fusion, with results summarized in Table 4. As shown in 1s and 2nd rows, the incorporation of spatial scaffold fusion, which aggregates spatial information in 3D space, improves performance by +0.36dB in PSNR and +0.02 in SSIM compared to the baseline that only relies image- domain fu- sion. Further integration of temporal scaffold fusion, which incorporates historical context through ego-motion warping and fusion in the latent scaffold domain, brings an additional gain of +0.58dB
8

---

Published as a conference paper at ICLR 2026
Back Left
Rotate
Back
Rotate
Tont Left
Rotate
Input View
Dynamics
W/O Memory
Memory
Figure 3: Qualitative results of scene completion on the Waymo dataset. Top: Aggregated scene without dynamic filtering, where red boxes indicate ghosting artifacts caused by accumulating the dynamic car. Bottom: Our method, equipped with dynamic-aware Gaussians, completes unob- served regions due limited sensor coverage and bridges cross-camera gaps while avoiding dy- namic artifacts. The predicted dynamic masks used for filtering are shown for reference.
Table 3: Impact of feature composition of F,. Table 4: Analysis of spatio-temporal fusion. "Geo" and "Sem" denote geometric and semantic "Spa" and "Tem" denote spatial and temporal fu- features, respectively.
sion, respectively.
Geo Sem
PSNR1
SSIMf
LPIPS,
Spa
Tem
PSNR1
SSIM
LPIPS]
24.78
0.73
0.35
24.14
0.68
0.32
24.85
0.72
0.31
24.50
0.70
0.32
25.08
0.74
0.30
25.08
0.74
0.30
in PSNR and +0.04 in SSIM. We also compare against a variant that explicitly uses two consecu- tive frames without latent-space temporal propagation. This approach achieves lower PSNR of 24.72dB, likely due to its limited ability to model dynamic elements and restricted temporal context. These results demonstrate the effectiveness of our unified spatio-temporal modeling approach that operates directly within the 3D scaffold representation for handling sparse, oivelapl camera views and complex dynamic driving scenes.
Dual-Branch Gaussian Decoder. We validate our dual-branch decoder design in Table 5. Using only point-anchored Gaussians results in a performance degradation 0.46 in PSNR, 0.02 in SSIM, and increase of 0.08 in LPIPS error, underscoring the critical role of voxel-generated Gaussians in improving scene completeness by effectively filling sparsely covered regions. The voxel-only variant is excluded from comparison it fails catastrophically at long-range rendering (Wei al., 2025), yielding consistently poor performance across all metrics.
Geometry Foundation Model. In Table 6, We ablate the impact of the geometry foundation model on our framework's performance. Specifically, replacing the default model with MoGe-2 (Wang al., 2025e), recently introduced open-domain geometry estimation method, yields consistent performance, which indicates that our approach is robust to the choice of the underlying geometry foundation model. Notably, we exclude the representative VGGT (Wang et al., 2025a), as our em- pirical observations indicate that it generalizes less effectively than z in outdoor driving scenarios.
CONCLUSION
We presented UniSplat, a unified feed-forward framework for dynamic driving scene reconstruc- tion and novel view synthesis. Our core contribution is the introduction of 3D latent scaffold

---

Published as a conference paper at ICLR 2026
Table 5: Ablation study on the two branches our Table 6: Performance comparison of different geometry foundation models.
Gaussian decoder.
Point Voxel
PSNRT
SSIMI
LPIPSI
Models
PSNR
SSIM
LPIPS.
24.62
0.72
0.38
MoGe-2
24.98
0.74
0.29
25.08
0.74
0.30
n²
25.08
0.74
0.30
that seamlessly unifies spatio-temporal fusion from multi-camera videos. By leveraging foundation models, this scaffold encodes robust geometric and semantic priors, enabling efficient fusion directly in 3D space. We further proposed dual-branch Gaussian decoder that generates dynamic-aware primitives from the scaffold, coupled with a streaming memory mechanism to accumulate static scene content time for long-term completion. Extensive experiments Waymo and nuScenes demonstrate that UniSplat not only achieves state-of-the-ar performance under standard settings but also exhibits remarkable generalization to challenging viewpoints outside the original camera coverage. We believe that our framework provides a promising foundation for future research on dynamic scene understanding, interactive 4D content creation, and lifelong world modeling.
ACKNOWLEDGMENTS
This work is supported by Guangdong Basic and Applied Basic Research Foundation (2025A151501 1434), Shenzhen Science and Technology Program (ZDCY202509011 133000011, and CCF-DiDi GAIA Collaborative Research Funds.
ETHICS STATEMENT
We confirm adherence to the ICLR Code of Ethics and have carefully evaluated the ethical impli- cations of our research. Our work advances 3D reconstruction for autonomous driving, aiming to improve safety while encouraging responsible deployment. We utilize established public datasets (Waymo Open Dataset and nuScenes) in strict compliance with their licenses, without processing sensitive personal identifiers.
REPRODUCIBILITY STATEMENT
To ensure reproducibility of our results, we have provided comprehensive details necessary to repli- cate our experiments and publicly released our source code. The main text outlines our experimental settings in Section 4.1, including dataset usage, evaluation metrics, and training configurations. Fur- ther implementation specifics are documented in Appendix A.I, which covers network architecture details, hyperparameter settings, and the of software libraries such SpConv for sparse volutions. All experiments are based on publicly available datasets, including the Waymo Open Dataset and the nuScenes dataset, and use clearly defined data splits and evaluation protocols sistent with prior work. For a fair comparison with baseline methods, we describe the retraining procedures and adaptations Appendix A.I.
REFERENCES
Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: multimodal dataset for autonomous driving. In Proceedings the IEEE/CVF conference on computer vision and pattern recognition, 2020.
Wei Cao, Marcel Hallgarten, Tianyu Li. Daniel Dauner, Xunjiang Gu, Caojun Wang, Yakov Miron, Marco Aiello, Hongyang Li, Igor Gilitschenski, Boris Ivanovic, Marco Pavone, Andreas Geiger, and Kashyap Chitta. Pseudo-simulation for autonomous driving. Conference on Robot Learn- ing (CoRL), 2025.
David Charatan, Sizhe Lester Li, Andrea Tagliasacchi, and Vincent Sitzmann. pixelsplat: 3d gaus- sian splats from image pairs for scalable generalizable 3d reconstruction. In Proceedings of the IEFF/CVF conference computer vision and pattern recognition, 2024.
10

---

Published as a conference paper at ICLR 2026
Yuedong Chen, Haofei Xu, Chuanxia Zheng, Bohan Zhuang, Mare Pollefeys, Andreas Geiger, Tat- Jen Cham, and Jianfei Cai. Mvsplat: Efficient 3d gaussian splatting from sparse multi-view images. In European Conference Computer Vision. Springer, 2024.
Yurui Chen, Chun Gu, Junzhe Jiang, Xiatian Zhu, and Li Zhang. Periodic vibration gaussian: Dynamic urban scene reconstruction and real-time rendering. arXiv preprint arXhv:2311.1851,, 2023.
Zhuoguang Chen, Minghui Qin, Tianyuan Yuan, Zhe Liu, and Hang Zhao. Long3r: Long sequence streaming 3d reconstruction. arXiv preprint arXiv:2507.18255, 2025.
Spconv Contributors. Spcomv: Spatially sparse convolution library. https: com/ Eraveller5$/speont 2022.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2021.
Lue Fan, Hao Zhang, Qitai Wang, Hongsheng Li, and Zhaoxiang Zhang, Freesim: Toward free- viewpoint camera simulation in driving scenes. Proceedings of the Computer Vision and Pat- tern Recognition Conference, 2025.
Ruiyuan Gao, Kai Chen, Enze Xie, Lanqing HONG, Zhenguo Li, Dit- Yan Yeung, and Qiang Xu. Magicdrive: Street view generation with diverse 3d geometry control. In The Twelfth Interna- tional Conference on Learning Representations, 2024.
Ruiyuan Gao, Kai Chen, Bo Xiao, Lanqing Hong, Zhenguo Li, and Qiang Xu. Magicdrive-v22 High- resolution long video generation for autonomous driving with adaptive control. In Proceedings of the IEEE/CVF international conference computer vision, 2025.
Mu Hu, Wei Yin, Chi Zhang, Zhipeng Cai, Xiaoxiao Long, Hao Chen, Kaixuan Wang. Gang Yu, Chunhua Shen, and Shaojie Shen. Metric3d v2: A versatile monocular geometric foundation model for zero-shot metric depth and surface normal estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
Wenbo Hu, Yuling Wang, Lin Ma, Bangbang Yang, Lin Gao, Xiao Liu, and Yuewen Ma. Tri-miprf: Tri-mip representation for efficient anti-aliasing neural radiance fields. In Proceedings of the JEEE/CVF International Conference on Computer Vision, 2023.
Nan Huang, Xiaobao Wei, Wenzhao Zheng, Pengju An, Ming Lu, Wei Zhan, Masayoshi Tomizuka, Kurt Keutzer, and Shanghang Zhang. S3gaussian: Self-supervised street gaussians for au- tonomous driving. arXiv preprini urXiv:2405.2032. 2024a.
Yuanhui Huang, Wenzhao Zheng, Yunpeng Zhang, Jie Zhou, and Jiwen Lu. Gaussianformer: Scene as gaussians for dision-based 3d semantic occupancy prediction. In European Conference Computer Vision, 2024b.
Yuanhui Huang, Amonnut Thammatadatrakoon. Wenzhao Zheng, Yunpeng Zhang, Dalong Du, and Jiwen Lu. Gaussianformer-2: Probabilistic gaussian superposition for efficient 3d occupancy prediction. In Proceedings of the Computer Vision and Pattern Recognition Conference, 2025.
Xu Jiawei, Deng Kai, Fan Zexin, Wang Shenlong, Xie Jin, and Yang Jian. AD-GS: Object- B-Spline Gaussian splatting for self-supervised autonomous driving. International Conference on Computer Vision, 2025.
Haian Jin, Hanwen Jiang. Hao Tan, Kai Zhang, Sai Bi, Tianyuan Zhang, Fujun Luan, Noah Snavely, and Zexiang Xu. LVSM: A large view synthesis model with minimal 3d inductive bias. In The Thirteenth International Conference on Learning Representations, 2025.
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splat- ting for real-time radiance field rendering. ACM Trans. Graph., 2023.
11

---

Published as a conference paper at ICLR 2026
Lu Ling, Yichen Sheng, Zhi Tu, Wentian Zhao, Cheng Xin, Kun Wan, Lantao Yu, Qianyu Guo, Zixum Yu, Yawen Lu, et al. DI3dv-10k: A large-scale scene dataset for deep learning-base basadd 3d vision. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2024.
lyy Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Confer- ence on Learning Representations, 2019.
Hao Lu, Tianshuo Xu, Wenzhao Zheng, Yunpeng Zhang, Wei Zhan, Dalong Du, Masayoshi
Tomizuka, Kurt Keutzer, and Yingcong Chen. Drivingrecon: Large 4d gaussian reconstruction model for autonomous driving, arXiv preprini arXiv:2412.0900 2024.
Sheng Miao, Jiaxin Huang, Dongfeng Bai, Xu Yan, Hongyu Zhou, Yue Wang, Bingbing Liu, An- dreas Geiger, and Yiyi Liao. Evolsplat: Efficient polume-basee gaussian splatting for urban view synthesis. In Proceedings of the Computer Vision and Pultern Recognition Conference, 2025.
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 2021.
Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics prim- itives with multiresolution hash encoding. ACM transactions on graphics (TOG), 2022.
Riku Murai, Eric Dexheimer, and Andrew J Davison. Mast3r-slam: Real-time dense slam with 3d reconstruction priors. In Proceedings of the Computer Vision and Pattern Recognition Confer- ence, 2025.
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023.
Muleilan Pei, Shaoshuai Shi, Xuesong Chen, Xu Liu, and Shaojie Shen. Foresight in motion: Re- inforcing trajectory prediction with reward heuristics. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, 2025a.
Muleilan Pei, Shaoshuai Shi, Lu Zhang, Peiliang Li, and Shaojie Shen. GoIRL: Graph-oriented inverse reinforcement learning for multimodal trajectory prediction. International Conference on Machine Learning. 2025b.
Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.000 00714. 2024.
Xuanchi Ren, Yifan Lu, Hanxue Liang, Zhangjie Wu, Huan Ling, Mike Chen, Sanja Fidler, Francis Williams, and Jiahui Huang. Scube: Instant large-scale scene reconstruction using voxsplats. Advances in Neural Information Processing Systems, 2024.
Oriane Siméoni, Huy V Vo, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Mare Szafraniec, Seungeun Yi, Michaël Ramamonjisoa, et al. Dinov3. arXiv preprint arXiv:2508. 2000::2000..0004,,, 2025.
Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, et al. Scalability in perception for au- tonomous driving: Wayme open dataset. In Proceedings of the IEEE/CVF conference com- puter vision and pattern recognition, 2020.
Stanislaw Szymanowicz. Chrisitian Rupprecht, and Andrea Vedaldi. Splatter image: Ultra-fast single-view 3d reconstruction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2024.
Qijian Tian, Xin Tan, Yuan Xie, and Lizhuang Ma. Drivingforward: Feed-forward 3d gaussian splatting for driving scene reconstruction from flexible surround-view input. In Proceedings of the AAAI Conference Artificial Intelligence, 2025.
12

---

Published as a conference paper at ICLR 2026
Adam Tonderski, Carl Lindström, Georg Hess, William Ljungbergh, Lennart Svensson, and Christoffer Petersson. Neurad: Neural rendering for autonomous driving. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2024.
Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. In Proceedings of the Computer Viston and Puttern Recognition Conference, 2025a
Jingkang Wang, Henry Che, Yun Chen, Ze Yang, Lily Goli, Sivabalan Manivasagam, and Raquel Urtasun. Flux4d: Flow-based unsupervised 4d reconstruction. Advances in Neural Information Processing Systems, 2025b.
Qianqian Wang, Yifei Zhang, Aleksander Holynski, Alexci Efros, and Angjoo Kanazawa. Con- tinuous 3d perception model with persistent state. In Proceedings of the Computer Vision and Pattern Recognition Conference, 2025c.
Ruicheng Wang, Sicheng Xu, Cassie Dai, Jianfeng Xiang, Yu Deng, Xin Tong, and Jiaolong Yang. Moge: Unlocking accurate monocular geometry estimation for open- images with optimal training supervision. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp- 5261-5271, 2025d.
Ruicheng Wang, Sicheng Xu, Yue Dong, Yu Deng, Jianfeng Xiang, Zelong Lv, Guangzhong Sun, Xin Tong, and Jiaolong Yang. Moge-2: Accurate monocular geometry with metric scale and sharp details. arXiv preprint 2xii:22507.02546, 2025e.
Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Ge- ometric 3d vision made easy. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, 2024.
Yifan Wang, Jianjun Zhou, Haoyi Zhu, Wenzheng Chang, Yang Zhou, Zizun Li, Junyi Chen, Jiang- miao Pang, Chunhua Shen, and Tong He. \ Scalable permutation-equivante visual geometry learning. arXiv preprint urXiv:2507.13344 2025f.
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero Simoncelli. Image quality assessment: from error visibility structural similarity. IEEE transactions image processing, 2004.
Dongxu Wei, Zhiqi Li, and Peidong Liu. Omni-scene: Omni-gaussian representation for ego-centric sparse-view scene reconstruction. In Proceedings of the Computer Vision and Pattern Recognition Conference, 2025.
Yuxi Xiao, Jianyuan Wang. Nan Xue, Nikita Karaev, Yuri Makarov, Bingyi Kang, Xing Zhu, Hu- jun Bao, Yujun Shen, and Xiaowei Zhou. Spatialtrackerv2: 3d point tracking made easy. In Proceedings of the IEEE/CVF international conference on computer vision, 2025.
Haofei Xu, Anpei Chen, Yuedong Chen, Christos Sakaridis, Yulun Zhang, Marc Pollefeys, Andreas Geiger, and Fisher Yu. Murf: multi balti-baseline radiance fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024.
Haofei Xu, Songyou Peng, Fangjinhua Wang, Hermann Blum, Daniel Barath, Andreas Geiger, and Mare Pollefeys. Depthsplat: Connecting gaussian splatting and depth. In Proceedings of the Computer Vision and Pattern Recognition Conference, 2025.
Qiangeng Xu, Zexiang Xu, Julien Philip,
Sai Bi, Zhixin Shu, Kalyan Sunkavalli, and Ulrich Neu- mann. Point-nerf: Point-based neural radiance fields. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022.
Xiaoyang Yan, Muleilan Pei, and Shaojie Shen. St-gs: Vision- based 3d semantic occupancy predic- tion with spatial-temporal gaussian splatting. arXiv preprint arXiv:2509. 16552, 2025a.
Yunzhi Yan, Haotong Lin. Chenxu Zhou, Weijie Wang, Haiyang Sun, Kun Zhan, Xianpeng Lang, Xiaowei Zhou, and Sida Peng. Street gaussians: Modeling dynamic urban scenes with gaussian splatting. In European Conference on Computer Vision, 2024.
13

---

Published as a conference paper at ICLR 2026
Yunzhi Yan, Zhen Xu, Haotong Lin, Haian Jin, Haoyu Guo, Yida Wang, Kun Zhan, Xianpeng Lang, Hujun Bao, Xiaowei Zhou, et al. Streetcrafter: Street view synthesis with controllable video diffusion models. In Proceedings of the Computer Vision and Pattern Recognition Conference, 2025b.
Haosen Yang, Chenhao Zhang, Wenqing Wang, Marco Volino, Adrian Hilton, Li Zhang, and Xiatian Zhu. Improving gaussian splatting with localized points management. In Proceedings of the Computer Vision and Pattern Recognition Conference, 2025a.
Jianing Yang, Alexander Sax, Kevin J Liang. Mikael Henaff, Hao Tang, Ang Cao. Joyce Chai, Franziska Meier, and Matt Feiszli. Fast3r: Towards 3d reconstruction of 1000+ images one forward pass. In Proceedings of the Computer Vision and Pattern Recognition Conference, 2025b.
Jiawei Yang, Boris Ivanovic, Or Litany, Xinshuo Weng, Seung Wook Kim, Boyi Li, Tong Che, Danfei Xu, Sanja Fidler, Marco Pavone, and Yue Wang. EmerneRF: Emergent spatial-temporal scene decomposition via self-supervision. In The Twelfth International Conference on Learning Representations, 2024.
Jiawei Yang. Jiahui Huang, Boris Ivanovic, Yuxiao Chen, Yan Wang, Boyi Li, Yurong You, Apoorva Sharma, Maximilian Igl, Peter Karkus, Danfei Xu, Yue Wang, and Marco Pavone. STORM: Spatio-temporal reconstruction model for large-scale outdoor scenes. In The Thirleenth Interna- tional Conference on Learning Representations, 2025c.
Ze Yang, Yun Chen, Jingkang Wang, Sivabalan Manivasagam, Wei-Chiu Ma, Anqi Joyce Yang, and Raquel Urtasun. Unisim: A neural closed-loop sensor simulator. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, 2023.
Zehao Yu, Anpei Chen, Binbin Huang, Torsten Sattler, and Andreas Geiger. Mip-splatting: Alias- free 3d gaussian splatting. Proceedings of the IEEE/CVF conference on computer vision and paltern recognition, 2024.
Chuanrui Zhang, Yingshuang Zou, Zhuoling Li, Minmin Yi, and Haoqian Wang. Transplat: Gener- alizable 3d gaussian splatting from sparse multi-view images with transformers. In Proceedings of the AAAI Conference Artificial Intelligence, 2025.
Richard Zhang, Phillip Isola, Alexci A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2018
Guosheng Zhao, Chaojun Ni, Xiaofeng Wang. Zheng Zhu, Xueyang Zhang, Yida Wang, Guan Huang. Xinze Chen, Boyuan Wang, Youyi Zhang, et al. Drivedreamer44: World models are effective data machines for 4d driving scene representation. In Proceedings of the Computer Vision and Pattern Recognition Conference, 2025.
Shijie Zhou, Haoran Chang, Sicheng Jiang, Zhiwen Fan, Zehao Zhu, Dejia Xu, Pradyumna Chari, Suya You, Zhangyang Wang, and Achuta Kadambi. Feature 3dgs: Supercharging 3d gaussian splatting to enable distilled feature fields. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, 2024a.
Tinghui Zhou, Richard Tucker, John Flynn, Graham Fyffe, and Noah Snavely. Stereo magnification: Learning view synthesis using multiplane images. arXiv preprint arXiv: vrxii:/805. /8055098/7 2018.
Xiaoyu Zhou, Zhiwei Lin, Xiaojun Shan, Yongtao Wang, Deqing Sun, and Ming- Hsuan Yang. Driv- inggaussian: Composite gaussian splatting for surrounding dynamic autonomous driving scenes. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2024b.
Xingxing Zuo, Pouya Samangouei, Yunwen Zhou, Yan Di, and Mingyang Li. Fmgs: Foundation model embedded 3d gaussian splatting for holistic scene understanding. International Journal of Computer Vision, 2025.
14

---

Published as a conference paper at ICLR 2026
A
APPENDIX
Input View
Novel View & GT
Ours
Omni-Scene
Figure 4: Qualitative comparisons on the nuScenes dataset. The red boxes highlight undesirable artifacts
Front Left
Rotate
Back Right
Back Right
Rotate
Input View
Dynamics
W/O Memory
Memory
Figure 5: Additional qualitative results of streaming scene completion on the Waymo dataset
A.1 IMPLEMENTATION DETAILS
Implementation details of UniSplat. We employ the SpConv (Contributors, 2022) library im- plement the sparse 3D U-Net, which comprises convolutional and transposed convolutional layers and achieves maximum downsampling factor 8x. The model trained streaming manner using clips of 20 frames for 20 epochs, with an initial learning rate of 1.5 10-1 following cosine decay schedule. For the semantic backbone within the scaffold reconstruction, we uses learn- ing rate scaled by factor of 0.1. To address the severe class imbalance in the dynamic segmentation loss, we incorporate a negative sampling strategy that randomly selects 50,000 negative pixels per sample for loss computation. For Gaussian rasterization, (2023) and, following StreetGaussian (Yan et al., 2024), set the spherical harmonics (SH) degree to 1 for efficiency.
adopt the framework of Kerbl et al.
Dynamic Rendering. To supervise the dynamic attributes of the Gaussians in G,, we introduce dynamics rendering mechanism that renders dynamic masks using the standard differentiable
15

---

Published as a conference paper at ICLR 2026
Table 7: Efficiency comparison the nuScenes dataset.
Method
FPSt
Mem.(GB),
Pa)am(M)
PSNRT
SSIMI
LPIPS].
Omin-Scene (Wei et al., 2025)
2.5
8.22
81.7
24.27
0.736
0.237
UniSplat
4.0
8.30
91.0
25.37
0.765
0.246
Gaussian-splatting pipeline, with dynamic logits as inputs instead of colors:
i-1 (1(1as) 3=1
D Σ d, iEN
(12)
Where D denotes the per-pixel dynamic probability. For ground-truth mask generation, we identify moving objects via 3D bounding box tracking, project them onto the image plane to create prompts for SAM2 (Ravi al., 2024), and subsequently use the model generate the final masks.
Implementation details of UniSplat counterparts. To adapt general feedforward reconstruction baselines to the autonomous driving setting, we retrain MpSplat (Chen al., 2024) and Depth- Splat (Xu et al., 2025) on the Waymo Open Dataset Sun et al. (2020). For MVSplat, we initialize the model using its official weights pre-trained on RealEstate 10K (Zhou al., 2018). Context views are from the current timestep, while target viewpoints include both the current and next timesteps. Training is conducted with a batch size of 16 on H20 GPUs for 40,000 iterations, as further training empirically degrades performance. For SepthSplat, 2021). All other training settings remain consistent with those used for MVSplat.
initialize from its official weights tre-trained on dI3dV (Ling et al., 2024) and use the variant equipped with a VIT-B backbone (Dosovitskiy et al.,
Evaluation Protocol on NuScenes. Due to the patch size constraint of our geometry foundation model, which requires image dimensions to be divisible by 14, train model at resolution of 224 x 406, differing from the 224 x 400 resolution used by Omni-Scene (Wei et al., 2025). For fair comparison, evaluation is performed by resizing our model's outputs to 224 x 400, aligning with the baseline's resolution before metric computation.
A.2 EFFICIENCY ANALYSIS
We benchmark the efficiency of our method against driving-specific reconstruction model, on the nuScenes dataset (Caesar et al., 2020). Note that Onni-Scene initializes its pixel-aligned Gaussians using predictions from a pretrained monocular depth estimation model (Hu et al., 2024), which are precomputed and not included in its computa- tional cost. To ensure fair comparison, we also exclude the cost of our geometry foundation model during inference. The results summarized in Table 7. UniSplat attains higher runtime efficiency FP. vs. 2.5 FPS) while surpassing Omni-Scene by large margin in reconstruction quality. We attribute this our fine-grained spatial fusion and streaming temporal aggregation in latent scaffold space. We also observe that Omni- Semii-Seee'ss ' rendering stage is the primary bottleneck (60% of in- ference time), it generates roughly 2 million voxel-based Gaussians per scene. All experiments were conducted on a single H20 GPU. The reported inference time represents the end-to-end recon- struction and rendering of all 18 target frames per sample, averaged over 2,048 samples, with data loading time excluded.
a state-of-the-an open-source
A.3 MORE QUALITATIVE RESULTS.
Qualitative Comparisons on the nuScenes dataset. Qualitative comparisons for novel view syn- thesis against Omni-Scene are presented in Figure 4. Our method demonstrates superior spatial coherence, as evidenced in challenging cases such as the thin pole (first row), and produces fewer artifacts like the buildings shown in the second and third rows.
Streaming Scene Completion. Figure provides additional qualitative results for our streaming scene completion capability. As shown in the first and second rows, when the viewpoint rotates or shifts to the regions outside the camera frustums, our method robustly reconstructs these newly visible areas, maintaining high fidelity and spatial coherence. The third row illustrates failure case in which a moving pedestrian is misclassified as static. As a result, the dynamic object is improperly retained in the memory, leading noticeable ghosting artifacts in the rendered sequence.
16

---

Published as a conference paper at ICLR 2026
T Os
T 1.3g
Input View
Voxel-rendered
Input View
VoxeI-rendered
Figure 6: Visualization of a moving vehicle at two timestamps (T = 0 s and T= 1.3 s). For each time step, we show the input view and the corresponding voxel-rendered result from our scaffold. Despite the vehicle's s motion, the renderings exhibit no ghosting artifacts or temporal inconsistencies.
Rotate
Input View
Point Branch
Voxel Branch
Dual Branch
Figure 7: Reconstructions from the point-only, voxel-only, and dual-branch decoders under camera
rotation. The red boxes highlight artifacts that appear when using only point branch.
Dynamic Handling the Latent Scaffold. Beyond the explicit dynamic filtering employed in our streaming Gaussian memory, observe that the latent scaffold itself exhibits an inherent ability to handle scene dynamics. Through unified spatio-temporal fusion, the model implicitly learns to aggregate multi-frame features according geometric consistency. In Figure 6, we visualize the reconstruction of a scene containing a moving vehicle. Although features from the moving vehicle are repeatedly integrated into nearby scaffold voxels over time, the voxel-rendered results remain free of trailing artifacts or temporal inconsistencies. This suggests that the learned fusion in the latent space effectively integrates temporal information and suppresses outdated evidence from dynamic objects.
Visual Analysis of the Dual-Branch Decoder. better illustrate the behavior of the point and voxel branches, we visualize the rendering outputs from individual branches under large view- point change in Figure The point branch preserves high-frequency details leads to overfitting to the input view, resulting in holes and distortions in the novel view (highlighted in red). Con- versely, the voxel branch serves as a continuous volumetric backbone, although it tends to produce smoother reconstructions with limited fine-graine sharpness. The final dual-brance decoder effee- tively combines these complementary strengths, recovering sharp details while maintaining robust structural integrity in novel views.
A.4 DYNAMIC ACTOR EDITING
Leveraging the explicit disentanglement of static and dynamic Gaussians, UniSplat supports flexible scene manipulation tasks, including the removal, relocation, and insertion of dynamic actors.
Extracting dynamic actors. Using the learned dynamic scores, first render 2D dynamic mask and extract connected components. 3D Gaussians corresponding to the selected region with high dynamic probabilities are then grouped as independent actor assets.
Background restoration after editing. A primary challenge in object removal relocation is the "disocclusion" problem, where the background region behind moving object unobserved in the current frame. Our framework addresses this by leveraging the streaming memory. In the standard pipeline, we employ a view-filtered memory M (Eqs. 9 and 10) to avoid redundancy with the current observations. For editing tasks, however, explicitly query the full memory state M which allows us to recover static Gaussians captured at earlier timestamps but occluded by the dynamic actor in the current view.
17

---

Published as a conference paper at ICLR 2026
Input View
Moving
Adding
Removing
Figure 8: Dynamic actor editing. Starting from the Input View (left), we demonstrate three editing operations: Moving the vehicle, Adding vehicle instance to the scene, and Removing the vehicle entirely.
Input View
W/O Memory
Memory
Figure 9: More qualitative results of actor removing.
Unified re-rendering with edited actors. Finally, the edited dynamic Gaussians (c.g., moved to a new position or newly inserted) are combined with the restored static scene. This unified set is rendered to produce the manipulated scene. As illustrated in Figure 8 and Figure 9, we demonstrate successful manipulation operations, including moving, adding, and removing an actor, highlighting the geometric consistency of both the edited actor and the recovered background.
A.5 VIDEO VISUALIZATION
We present a supplementary video demonstrating our reconstruction results on two scenes, showcas- ing novel view synthesis under camera shifts, along with dynamic prediction capabilities. We note that in certain frames (e.g., at 1:03), the low Gaussian opacity of misalignment between dynamic masks and RGB content. Please refer to the video file included in the supplementary material.
glass surfaces results in perceived
B DECLARATION OF LLM USAGE
Large Language Models (LLMs) utilized to assist with language refinement and manuscript preparation, including grammar checking and enhancing textual clarity. All scientific concepts, methodological innovations, experimental frameworks, data analysis, and conclusions presented in this work are independently developed by the authors. We have thoroughly reviewed and validated all content, and assume complete responsibility for the accuracy and integrity of this manuscript.
18


Reviews:
[{"review_id": "Mrhtjson5x", "reviewer": "Reviewer_uYox", "summary": "This paper presents UniSplat, a feedforward framework for dynamic driving scene reconstruction and novel view synthesis. The core idea is to construct a 3D latent scaffold that unifies multi-view spatial and multi-frame temporal fusion. The scaffold is built by leveraging geometry and visual foundation models to encode geometry and semantics in 3D space. UniSplat then introduces a Gaussian decoder to generate the 3DGS reconstruction from the scaffold, and a memory mechanism to maintain the accumulated static Gaussians over time to improve completion. The proposed UniSplat achieves state-of-the-art performance on Waymo and nuScenes datasets and demonstrates SoTA performance for novel view synthesis compared to existing feed-forward reconstruction methods.", "questions": "* How does UniSplat perform when extended beyond driving scenes to other dynamic 4D environments (e.g., human motion, indoor scenes)?\n* Could the authors clarify the scalability of the scaffold fusion to longer sequences (e.g., 1–2 minutes) and higher-resolution scenes?\n* Can the streaming memory lead to error accumulation or stale Gaussians over time? How is this mitigated?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The paper is well written and clearly structured, making the technical ideas easy to follow despite the method’s complexity.\n* The proposed scaffold-based fusion mechanism is both intuitive and practical, effectively addressing the challenge of modelling dynamic scenes while supporting progressive memory updates.\n* Experimental evaluation is thorough, including comparisons to multiple strong baselines on two large-scale driving datasets, with consistent quantitative and qualitative improvements. Ablation studies also provide convincing evidence of the contribution of each component.", "weaknesses": "* It is not entirely clear how the scaffold-based fusion mechanism handles dynamic content. For example, when a vehicle moves across frames, how are its features aligned or updated consistently during fusion?\n* The paper shows novel view rendering by rotating the camera, but it remains unclear whether dynamic actors can be moved or manipulated. Supporting such editing would enable full camera simulation.\n* Maintaining and updating a dense 3D scaffold could be computationally expensive for long sequences or large-scale scenes. The paper does not analyze memory usage or runtime trade-offs beyond short sequences.\n* The evaluation focuses mainly on reconstruction metrics. Additional experiments on temporal consistency, and robustness under occlusions would make the claims more convincing.\n* The predicted dynamic masks appear misaligned with RGB images in the supplementary video (e.g., at 1:03). Clarifying the source of this misalignment would strengthen the work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AFJpTMkm7K", "reviewer": "Reviewer_cKtV", "summary": "The paper proposes UniSplat, a feed-forward framework for dynamic driving-scene reconstruction that (i) builds an ego-centric 3D latent scaffold by fusing features from a geometry foundation model and a visual foundation model, (ii) performs spatio-temporal fusion directly in the scaffold via sparse 3D UNets with pose-warped accumulation, and (iii) decodes dynamic-aware Gaussians using a dual branch (point-anchored + voxel-anchored) with a streaming static-memory to complete unseen regions. \n\nThe dynamic prediction is supervised by pseudo-labels. These labels for moving objects are identified via 3D bounding-box tracking; the boxes are projected to the image to prompt SAM2, which returns dynamic segmentation masks. \n\nOn Waymo and nuScenes, UniSplat reports leading image metrics for reconstruction and novel view synthesis, and claims robustness for viewpoints outside camera coverage.", "questions": "1. Positioning / novelty. Is there any unique capability or perspective/insight offered by the proposed method?\n\n2. Baseline Strength vs. Main Results. I understand that numbers in ablation studies are not directly comparable to those in main results, but it seems like a minimal UniSplat variant (without several add-ons) already performs strongly, possibly already exceeding the competitor results in the main table?\n\n3. End-to-end latency. Please report full inference latency that includes the geometry/vision foundation stages.\n\n4. External-Free and Voxel-Only Variants. Are these experiments feasible in the current setting? \n  - Table 3: Add a no-Geo / no-Sem point (no external models) to show the fully end-to-end trained performance.\n  - Table 5: Add a voxel-only prediction variant (no point-anchored branch) to isolate the value of the dual-branch decoder.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- An engineering-first pipeline. This paper proposes three-stage pipeline with a good motivation for 3D scaffold–space fusion, using sparse 3D UNets for spatial aggregation and pose-conditioned temporal accumulation. Overall it's easy to follow.\n- Dynamic-handling. The proposed method can handle dynamic scenes.\n- Experimental coverage. Evaluations on Waymo and nuScenes with both quantitative tables and qualitative figures; ablations cover feature composition, spatial vs temporal fusion, and decoder branches.", "weaknesses": "- **A System with Limited Conceptual Novelty**: The primary weakness is the paper's contribution, which appears to be more of a strong engineering effort than a conceptual breakthrough.\n  - The “3D latent scaffold” (spatial-fusion) is essentially a sparse voxel grid with fused geometry+semantic features, a very standard way (fused in 3D grids) for fusing 3D features in many domains (like SLAM, or 3D understanding) and recent generalizable 3DGS/voxel/triplane works. \n  - Warping and fusing features/primitives from previous frames by a pose-warped addition is another straightforward and known method used by the community for years (e.g., BEVFormer, BEVFusion and earlier works). \n  - Using powerful, frozen geometry and visual foundation models to provide strong priors is a common practice. \n  - The dual point/voxel decoder branches are a sensible design but I feel this is a compromise to sparse-voxel representations.\n  - While the final system is effective, the core ideas themselves are largely incremental, making the work feel less insightful and more engineering-centric.\n\n- Missing Related Work & Supervision Dependency: The paper misses discussion of several highly relevant works on feed-forward dynamic scene reconstruction, such as Flux4D (Want et al., 2025) and STORM (Yang et al., 2025). These methods also tackle large-scale, dynamic outdoor scenes, but they explore unsupervised 4D reconstruction, i.e., they do not require dynamic segmentation masks from external sources like object bounding boxes and SAM2.\n\n\n- (Not a weakness) Setting novelty aside, this is a well-executed project with sufficient engineering efforts and clear implementation details.\n\n\n[1] Want et al. \"Flux4D: Flow-based Unsupervised 4D Reconstruction.\" In NeurIPS 2025\n\n[2] Yang et al. \"STORM: Spatio-Temporal Reconstruction Model for Large-scale Outdoor Scenes.\" In ICLR 2025", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ac95v2QRgx", "reviewer": "Reviewer_rymU", "summary": "This paper introduces UniSplat, a unified framework for spatio-temporal fusion and dynamic scene reconstruction.\nThe key idea is to construct a latent 3D scaffold where both spatial fusion (within a frame) and temporal fusion (across frames) are performed in the same structured voxel domain.\nThe previous fused scaffold is warped to the current frame using ego-pose and merged directly, avoiding redundant computation and maintaining geometric consistency.\nA dual-branch Gaussian decoder combines point-based details with voxel-based coverage, while a dynamic filtering mechanism accumulates only static components in a memory buffer to enable out-of-FOV reconstruction.\nExperiments on the Waymo and nuScenes datasets show that UniSplat significantly improves reconstruction accuracy and efficiency over prior works such as MVSplat, Omni-Scene, and DriveDreamer4D.", "questions": "Why did you choose to perform spatio-temporal fusion entirely in the latent 3D scaffold domain rather than using separate spatial and temporal modules (as in Omni-Scene)? What specific benefits (efficiency, stability, or accuracy) did you observe?\n\nCould you elaborate on why the dual-branch decoder was designed in its current form? Were other fusion strategies (e.g., adaptive weighting between point and voxel) explored?\n\nHow robust is the dynamic filtering threshold under varying dynamic motion or sensor noise? Could adaptive strategies improve generalization?\n\nWhy not adopt token-based or diffusion-based representations for the scaffold? Is the explicit voxel representation crucial for efficiency or accuracy?\n\nHow does the method behave when the camera view coverage becomes extremely sparse (e.g., two front-facing cameras only)?\n\nAre there scenarios where the streaming memory might accumulate false positives from misclassified dynamic objects, and how would the method handle this?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Proposes a unified spatio-temporal fusion paradigm in a single latent 3D scaffold, which is a conceptually cleaner and more efficient design than previous separate spatial and temporal modules.\nThe dual-branch Gaussian decoder effectively balances fine-grained details (point) and global completeness (voxel).\nThe dynamic filtering + memory streaming mechanism elegantly addresses out-of-FOV reconstruction and ghosting issues caused by dynamic objects.\nStrong experimental results and ablations demonstrate the necessity and contribution of each module.\nThe framework is feed-forward and efficient, making it more practical than diffusion- or transformer-based alternatives.", "weaknesses": "While the framework is conceptually unified, many of its components (voxel scaffold, temporal warping, Gaussian splatting) are adapted from prior work.\nThe dynamic filtering relies on threshold-based heuristics; it is not clear how robust this is under complex motion or sensor noise.\nThe paper lacks a deeper comparison with recent diffusion-based or token-based reconstruction frameworks, which could strengthen the positioning of this method.\nSome design choices (e.g., the specific form of the dual-branch decoder, scaffold resolution) are under-explained, and might be perceived as empirical rather than principled.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wVBtOtRkzc", "reviewer": "Reviewer_2VZv", "summary": "This paper proposes UniSplat, a feed-forward framework for dynamic driving scene reconstruction based on 3D Gaussian Splatting, introducing a 3D latent scaffold that enables unified spatio-temporal feature fusion across multi-view camera inputs and sequential frames. A dual-branch Gaussian decoder jointly handles fine-grained reconstruction and spatial completion, while a streaming Gaussian memory with dynamic-aware filtering progressively accumulates static scene content to address coverage gaps beyond the current field of view.", "questions": "1. Regarding the streaming memory, can the authors provide more insight into how the memory size evolves over long sequences? Does the model include any mechanism for memory pruning or confidence-based discarding, and could memory saturation impact performance?\n2. For the dual-branch decoder, have the authors observed complementary behaviors between point-anchored and voxel-generated Gaussians across different scene structures? A visual or statistic showing their relative contributions would help clarify the motivation.\n3. How sensitive is the overall system to the accuracy of the geometry foundation model used for initializing the scaffold? For example, if the model produces slight drift or inconsistent scale, could this propagate and affect memory fusion stability?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The 3D latent scaffold and spatio-temporal fusion mechanism introduce a more structured way to aggregate multi-view and temporal cues, beyond 2D feature aggregation used in previous feed-forward approaches.\n2. Dynamic-aware Gaussian memory is conceptually appealing, enabling persistent scene completion and reducing ghosting artifacts from moving objects.", "weaknesses": "1. Readers may find it difficult to understand why the scaffold representation is fundamentally more suitable for handling sparse multi-view driving scenarios.\n2. The streaming Gaussian memory mechanism is effective in mitigating ghosting and compensating camera blind spots. However, it is introduced as a component-level implementation rather than a more formalized recurrent world-state update.\n3. The dual-branch decoder (point-based detail refinement and voxel-based completion) shows quantitative gains, but the ablations do not analyze where each branch contributes most (e.g., near-field vs.\\ far-field geometry, static vs.\\ dynamic regions).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)