ODE-GS: Latent ODEs for Dynamic Scene Extrapolation with 3D Gaussian Splatting
Authors: Daniel Wang
Patrick Rim
Tian Tian
Dong Lao
Alex Wong
Ganesh Sundaramoorthi
Venue: ICLR 2026 Poster (2026)
Keywords: gaussian splatting
latent ODE
extrapolation
reconstruction
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: We introduce ODE-GS, a novel approach that integrates 3D Gaussian Splatting with latent neural ordinary differential equations (ODEs) to enable future extrapolation of dynamic 3D scenes. Unlike existing dynamic scene reconstruction methods, which rely on time-conditioned deformation networks and are limited to interpolation within a fixed time window, ODE-GS eliminates timestamp dependency by modeling Gaussian parameter trajectories as continuous-time latent dynamics. Our approach first learns an interpolation model to generate accurate Gaussian trajectories within the observed window, then trains a Transformer encoder to aggregate past trajectories into a latent state evolved via a neural ODE. Finally, numerical integration produces smooth, physically plausible future Gaussian trajectories, enabling rendering at arbitrary future timestamps. On the D-NeRF, NVFi, and HyperNeRF benchmarks, ODE-GS achieves state-of-the-art extrapolation performance, improving metrics by 19.8% compared to leading baselines, demonstrating its ability to accurately represent and predict 3D scene dynamics.

Published as a conference paper at ICLR 2026
ODE-GS: LATENT ODES FOR DYNAMIC SCENE EXTRAPOLATION WITH 3D GAUSSIAN SPLATTING
Daniel Wang' Patrick Rim'
Tian Tian
Alex Wong'
Ganesh Sundaramoorthi
Yale University 'TU Delft 'RIX
1 "fdaniel. . wang. . chw33, patrick. alex. wong) @yaley edu Tr.Tiin@studen tudelft.nt $ganesh. wuw.daaaamoaaorrtrrttttt.....
ABSTRACT
We introduce ODE-GS, a novel approach that integrates 3D Gaussian Splatting with latent neural ordinary differential equations (ODEs) to enable future extrapolation of dynamic 3D scenes. Unlike existing dynamic scene reconstruction methods, which rely on conditondt deformation networks and are limited to interpola- tion within a fixed time window, ODE-GS eliminates timestamp dependency by modeling Gaussian parameter trajectories continuous-time latent dynamics. Our approach first learns an interpolation model to generate accurate Gaussian trajecto- ries within the observed window, then trains a Transformer encoder to aggregate past trajectories into a latent state evolved via a neural ODE. Finally, numerical integration produces smooth, physically plausible future Gaussian trajectories, enabling rendering at arbitrary future timestamps. On the D-NeRF, NVFi, and HyperNeRF benchmarks, ODE-GS achieves state-of-the-art extrapolation perfor- mance, improving metrics by 19.8% compared to leading baselines, demonstrating its ability to accurately represent and predict 3D scene dynamics.
INTRODUCTION
Recently, 3D Gaussian Splatting (3DGS) methods have emerged as an effective approach for dynamic scene reconstruction. By training on images taken from a time-dependent 3-dimensional (3D) scene, such methods enable photorealistic novel view synthesis (NVS) for any time within the observed window. However, the task of enneiitionnextrpplaaain future scene dynamics from past observations--rennn remains largely underexplored. Performing such task is a well-studied capability that humans possess (Rao & Ballard 1999; Mrotek & Soechting 2007 Battaglia et al. 2013 Khoei al. 2017), while mirroring this capability in inteliggent systems is of great interest for applications such as self-driving, robotics, and augmented reality. Our focus is then to bridge this gap and enable the ability to forecast future 3D states in the context of dynamic scene reconstruction, which will refer to as dynamic scene extrapolation.
Performing dynamic scene extrapolation fundamentally more challenging than reconstruction. Existing methods are primarily designed for temporal interpolation within the observed window, where arbitrary view points at given time can be reconstructed by conditioning models on timestamps. In contrast, temporal extrapolation is inherently under-constrained, as there exist infinitely many pos- sible future dynamics given our past observations. Therefore, popular dynamic scene reconstruction methods, such as TiNeu Vox (Fang et al. 2022). Deformable 3D Gaussians (Yang et al. (2024), and 4D Gaussian Splatting (Wu et al.) 2024). excel "filling the gaps" between observed timestamps, but degrade when extended beyond them. In such cases, future timestamps fall outside the training distribution, leading to distrr-iittiittionn (OOD) failures.
As with many under-constrained problems, our task then becomes estimating the most likely future dynamics given existing observations. Incorporating physical assumptions can substantially narrow this solution space. An example of such assumption is the spatio-temporal smoothness of motion, which provides strong constraints on the evolution of scene dynamics. Differential equations have long served principled tools for deseribing the evolution of physical systems (Chen et al. 2018)), and ordinary differential equations (ODEs) in particular offer a natural formalism for representing continuous and physically plausible motion trajectories.

---

Published as a conference paper at ICLR 2026
Observed Time
Unobserved Time
Time
Sequance Model
Interpolation
Extrapolation
Trajectory
ODE Latent Dynamics
Figure I: Unlike existing methods that focus interpolation, i.e., reconstructing novel scene views at unseen timestamps within the observed window, we focus on extrapolation, i.e., extending scene dynamics beyond the observed times, by first training representation of the observed scene and then using a seguuecce-t-seeqenee model to reconstruct future novel views via latent ODE dynamics.
In light of these considerations, we propose to represent the temporal evolution of a dynamic scene in continuous-time latent space, where the evolution is governed by ODE. In this way, impose physical assumption of smooth motion into the predictive model (Chen et al. 2018 Rubanova et al. 2019), while retaining 3DGS's capability of figh-fidelity rendering. Unlike prior approaches that condition directly on explicit timestamps-lcading to OOD failures-We reformulate dynamic scene extrapolation as sequence-tr -oe-nnce-----ueeeeeeee forecasting problem, which naturally aligns with 3DGS's explicit representation of the 3D scene. Specifically, we first encode temporal sequence of Gaussian parameters using Transformer (Vaswani et al., (017) into latent state that embeds past motion. We then model the temporal evolution of this latent state via neural ODE, where neural network parameterizes its velocity field. Extrapolation is thus realized by numerically integrating the latent dynamics beyond the observed window, yielding future latent states. These states subsequently decoded back into Gaussian parameters, which can be rendered into novel future views. To further constrain extrapolation,
incorporate additional physical priors lightweight regularizers during training. These include penalties that encourage smoothness both the latent trajectories and the decoded Gaussian parameters, reinforcing the assumption of continuous motion.
The resulting method, ODE-GS (Ordinary Differential Equation- Gaussian Splatting), decou- ples scene reconstruction from temporal forecasting. We first optimize Gaussian interpolation model within the observed window, by training a set of canonical Gaussian parameters and defor- mation network. This interpolation model is then frozen and used as a data generator, producing temporal trajectories of 3D Gaussian parameters. We train the Transformer-Laten ODE on these trajectories, but crucially, the model is conditioned only on partial prefix of each sequence and learns to extrapolate the remainder. This training setup equips ODE-GS to extrapolate beyond the observed time horizon at inference, by integrating forward time via our latent ODE and decoding back into future Gaussian parameters.
Our contributions are:
We propose ODE-GS, which integrates 3D Gaussian Splatting with Transformer based latent ODE. We model scene dynamics as continuous trajectories latent space to enforce smoothness priors, enabling stable extrapolation beyond the observed window.
We validate our modeling strategy that decouples dynamic scene reconstruction from tem- poral forecasting. By first optimizing interpolation model for reconstruction and then training the latent ODE forecaster on generated trajectories, ODE-GS avoids direct reliance on timestamp conditioning and mitigates diittr--iittittti failures.
We incorporate inductive biases as regularizers that encourage smoothness in both latent and decoded trajectories, improving stability and rendering quality at extrapolated timestamps.
ODE-GE GS achieves state-of offtthe-rrtt performance in dynamic scene extrapolation, improving over the best existing extrapolation method by an average of 21.4% PSNR, 7.4% SSIM, and 30.5% LPIPS across synthetic (D-NeRF, NVFi) and real-world (HyperNeRF) datasets.
2

---

Published as a conference paper at ICLR 2026
2 RELATED WORK
Novel View Synthesis (NVS), Recent advances in NVS have explored a diverse range of approaches, including explicit mase- representations (Broxton al., 2020; Dou et al. 2016 Newcombe 2013 Ortts-Escolane et al.] 2016), and implicit neural volume representations Lombardi et al. 2019 Among these, Neural Radiance Fields (NeRF) (Mildenhall et al. 2021 ?) have emerged as a dominant paradigm due to their leading performance in NVS. The foundational success of NeRF has led to numerous extensions for dynamic scene reconstruction (Attal et al., 2023: Du et al. 2021
ct al.
Ost et al. 2021: Park et al. 202 Pumarola et al.)
[021], enabling applications such monocular video-based scene reconstruction (Gao et al. [02 [i et al. 2021, [Tretschk et al. (02T), editable
scene representations (Kania et al., 2022] Park et al. 2021a). and
reconstructions (Peng et al., 2021bla), Notably, NVFi (Li et al., 2023) investigates the problem of dynamic scene
extrapolation by adding geometric priors. However, its dependence on explicit timestamps during training induces out-of-distribution errors when extrapolating.
3DGS and Dynamic Scene Modeling. 3DGS (Kerbl et al. 2023). first used for static NVS, has become a popular choice for representing dynamic scenes due to its speed and explicit nature (Huang
al. 2024; Li al., 2024| Duan al. 2024). Deformable 3D Gaussians (Yang al., 2024) learn Gaussians in a canonical space with a deformation network. By training both the canonical Gaussians and the deformation network simultaneously, they enable continuous- rendering by transforming canonical Gaussians into arbitrary time within the training window. Concurrently, 4D Gaussian Splatting (Wu al. 2024) took the same canonical-deform strategy and additionally integrated 41) neural voxels inspired by HexPlane (Cao & Johnson 2023). Additional efforts, such as explicit time-variant Gaussian features (Luiten et al.J 2024)) achieved interactive frame rates and enabled flexible editing. Gaussian Video (Bond et al., 2025) uses neural ODEs, but for learning smooth camera trajectories rather than scene motion. However, most 3DGS-basee approaches rely on a time-conditioned deformation field, making these models excel in interpolation tasks but unable to extrapolate into unseen time in the future. CaussianPrreicction (Zhao et al., 2024) has recently explored this issue by combining superpoint strategy with Graph Convolution Networks (GCN) that directly conditions on past motion instead of time, but is only capable of sampling at discrete steps when extrapolating.
Neural Ordinary Differential Equations. Neural Ordinary Differential Equations (Neural ODEs) (Chen al. 2018) introduced novel approach for continuous-depth neural networks. Instead of defining discrete layers, Neural ODE specifies the continuous dynamics of hidden state using neural network that parameterizes its derivative. The network's 's output is then determined by numerical ODE solver that integrates these learned dynamics over specified interval. Key advantages include memory-efficient training via the adjoint sensitivity method, inherent handling of mregularly-samplee data, and adaptive computation.
Relevant to dynamic modeling, Latent ODEs (Rubanova et al. 2019) typically encode an input sequence into an initial latent representation whose continuous time evolution is then governed by Neural ODE. decoder can subsequently map these evolving latent states back to the observation space arbitrary times. This is effective for modeling continuous trajectories and is often combined with Variational Autoencoders (VAEs), as in ODE2VAE (Yildiz et al. 2019), to learn distributions over latent paths and capture uncertainty.
The synergy between recurrent methods and Neural ODEs has also been explored. For instance, GRU-ODE (De Brouwer et al.) (019)) adapts GRU-like gating mechanisms to continuously evolving states, while ODE-RNN (Rubanova et al. 2019) interleaves discrete RNN updates at observation points with continuous ODE-based evolution. Recent methods have explored using Neural ODEs for vision tasks like images (Liu et al. 2025), but applications in 3D representation are still limited.
3 METHODOLOGY
In the following sections, we formalize our problem setting and detail each component of ODE-GS: first. the interpolation model that generates dense Gaussian trajectories within the observed window (Sec. [.I]: second, the Transformer based latent ODE architecture for extrapolation (Sec. 3.2): third, dynamic sampling strategy to train the extrapolation model different forecasting horizons
3

---

Published as a conference paper at ICLR 2026
Trajectory Intializatoon
Input Trajectrii
See Tarpet Trggectories
bd Lo09
Gousstan
G
6th
Gitel
Eroopclation
Predicted
Concitioned Beform MLP
Time
Diforenniatle oot Selver
Frod coed Trajecteiiss
Transformer Enoader
a/too]
Decoder
Trainable
OBE
z(t-)
Gbu
Frozes
Regu
Figure 2: We initialize temporal trajectories of 3D Gaussian parameters using the frozen interpo- lation model, which consists of the canonical 3D Gaussian set and a time-conditioned deformation ML.. These trajectories lie entirely within the observed temporal window. 2: Through our dynamic sampling strategy, each Gaussian trajectory is sampled into multiple observed prefix (input) and a held-out suffix (target) trajectories, providing training pairs for the Transformer latent ODE. 3: Latent-ODE training encodes the observed prefix with Transformer, infers latent initial state, and evolves forward with a neural ODE. 4: decoder maps the latent path back to Gaussian parameters, which are supervised against the ground-trut -ound-truth suffixes via an L1 loss and smoothness regularizers.
(Sec. 3.3): ; and finally, the training and regularization objectives that ensure physically plausible and stable extrapolated dynamics (Sec. 3.4). Implementation details are discussed in Sec. B.1
Notation and Formalization. Let {T:} be set of calibrated RGB images, with corresponding camera poses {Vi} capturing dynamic 3D scene at timestamps (4:).
D = (44, V,, V.
4: BxHXX
V, E SE(3). t, EER,
(1)
aim to learn continuous rendering operator F R SE(3) B that generates RGB images for any time / and camera pose V. The operator decomposes as:
Du(t tmin St mmax =) = R(G(), g(t) = Es)
if > fmax
where == ) is learnable canonical set of 3D Gaussians, De is interpolation defor- mation function, E.p. is a peruence-to-sequeen extrapolation model with Transformer Latent ODE architecture, and R. is the differentiable rasterizer from (Kerbl et al. 2023). y is sequence of past
Gaussian parameters that serve as input, which is detailed in See. B
Each canonical Gaussian Gş (μR, Aks Sk, Ck2 ag) comprises position μı R quaternion @ R² scales 8 E R3, opacity 0g E IR, and spherical harmonics (SH) coefficients E R4, where d is the number of SH functions used, usually set 3. For each k. we keep a4 and с consistent across time, so that only Pk. qk. and are time- -ineddepeddent For simplicity, from now on, G(t) refers to only these three parameters. We may then derive the rotation matrix Rg E TR3x3 from (k- the scaling diagonal matrix S₁ E x3x3 from 8k, and the covariance matrix for each Gaussian by Σ, S R
The differentiable rasterizer (Kerbl et al.) 2023) R renders images by projecting each 3D Gaussian onto the image plane, computing per-pixel alpha compositing with front-to-back blending:
R-
C(p) = Σ
(1- kEG(p) 1=1
CROck
where G(p) Gaussians affecting pixel p, G. is the color, and ay is the opacity after 2D projection. We first learn the interpolation model G and Das, then freeze them and train &g for extrapolation.
4

---

Published as a conference paper at ICLR 2026
3.1 INTERPOLATION MODEL FOR TRAJECTORY GENERATION
To obtain trajectories of Gaussian parameters within the observed window, we adopt a canonical-plus deformation strategy (Yang al., 2024 Wu al. 2024) that has become standard in dynamic scene modeling. A canonical set of 3D Gaussians, G, represents the static reference configuration of the (MLP) Dw predicts
scene, while lightweight, conditonde deformation Multi-Layer-Perepptre offsets for position, rotation, and scale at each timestamp t. This enables continuous interpolation of Gaussian states across time. The interpolation model is trained using photometric reconstruction objective:
Lrnnder = (1 A) 10, 4111 + X- (I,
Where I, is rendered using the differentiable rasterizer: 1, = R(g + (t, V.). After training. we freeze both the canonical Gaussians 9 and the deformation network Da., so that given any time I within the observed time window, G& (t) may be generated for all k. SSIM refers the Structural Similarity Index Measure.
3.2 LATENT ODE MODEL FOR EXTRAPOLATION
Viewing temporal prediction mapping between observed sequence and future sequence, we model scene dynamics as a sequence-to-sequene problem. Our extrapolation module & is a Transformer Latent ODE that predicts future dynamics from past Gaussian trajectories.
Given an input sequence
{G((L))))=
uniformly sampled from a context window of length N₂ for Gaussian k we embed each step and add sinusoidal positional encodings preserve temporal order. The resulting sequence is processed by a Transformer encoder
Fg NRRNO Rd
yielding a latent representation ztt) E Rd that summarizes past dynamics. This latent state initializes a neural ODE, parameterized by an MLP:
de = fo((tt))
Numerical integration produces continuous latent trajectory zt)) for any t t> tmax- A decoder then maps the evolved latent states back Gaussian parameters:
8Ø _
IR (t) (z(()))
This combination of sequence encoding and continuous latent evolution allows Eg to generate smooth Gaussian trajectories without explicit timestamp embeddings, enabling extrapolation to arbitrary future horizons where unseen timestamps no longer out-of-distribution.
3.3 DYNAMIC TRAJECTORY SAMPLING
To effectively train the extrapolation module, it is essential to expose the model to prediction tasks spanning wide range of forecasting horizons. Therefore, unlike common approaches where sampled trajectories always occupy the same time span uniformly (Li et al., 2023), we design a dynamic trajectory sampling strategy. The pre-trained interpolation model provides continuous trajectories of Gaussian parameters, from which we extract an observed prefix and a future suffix. The prefix sampled at fixed intervals to ensure consistent input dimensionality, while the suffix varies in temporal span depending on the selected starting time.
Our training dataset is constructed the union over all possible srefix-suffix splits of Gaussian trajectories across all Gaussians and starting times. This design compels the extrapolation model to learn from both short-term and long-term forecasting instances within a unified training procedure, thereby encouraging robust generalization beyond the observed window, as described in Sec. B At inference, the model conditions on the final observed prefix and extrapolates Gaussian trajectories forward in time, which are subsequently rendered into novel frames. At test time, we take the final context segment of length N, from the observed window, generated by the interpolation model D.,. This sequence is encoded and evolved forward by the Latent ODE to produce extrapolated Gaussians (i)(1) for & lmax- The full scene (1) (G) M is rendered with differentiable rasterizer: ī(L.V) R()().
5

---

Published as a conference paper at ICLR 2026
3.4 TRAINING OBJECTIVE
Extrapolation Loss. Given a training pair consisting of a context trajectory Ye and its cone- sponding target trajectory Th = {t, the extrapolation module & produces predictions Ge = {ty) The extrapolation loss supervises these predictions by minimizing the mean absolute error (LI) between target Gaussian parameters generated by the interpolation model and the predicted parameters by the extrapolation model across the target temporal window:
Le
G
This objective ensures that the predicted Gaussian trajectories align closely with true future dynamics before additional regularization terms are applied.
To ensure physical plausibility and prevent overfitting to the observed trajectory, we also introduce two complementary regularizations that promote smoothness latent dynamics and 3D trajectories.
The latent regularization penalizes high-frequence oscillations in the learned ODE function. Given the latent trajectory z(t) evolved by the neural ODE with velocity field E(1 = fo(2(()), we approxi- mate the latent acceleration through finite differencing:
No11
fo(=(lz+1)) [o(=((])))) 2
Riatent
Ne
At,
where (t are the extrapolation timestamps, At, ts+1 tú the step size, fe the neural ODE.
The trajectory regularization enforces smoothness directly in 3D space by penalizing accelerations of the Gaussian positions. For each Gaussian Gx(t), (x(t) E IR3 its position at time We compute:
(x((5+1) - rx mx((t))
Ray
Σ
MN.
At,
k=
where M is the number of Gaussians, and the velocity is approximated as:
vx(tt) 11e(15+1)= mk((1))
(8)
At,
Adaptive weighting for regularization. early stages, strong regularization may inhibit the model from learning meaningful dynamics. To address this, we introduce an adaptive regularization weighting mechanism that dynamically adjusts the contribution of regularization throughout training. At each iteration, we estimate the model's convergence state using the Exponential Moving Average (EMA) of the trajectory prediction loss, which provides a stable signal compared to the raw loss that may fluctuate due to oscillations. As training progresses and the extrapolation loss decreases, the regularization weight is increased. This gradually biases the model toward selecting smoother trajectories among the many plausible solutions, thereby guiding fine-grained trajectory predictions toward stable convergence. The final training loss then becomes:
L Le + St (Atatena Riatent Auna Ruzaj)
where Aasteu and Auaj hyperparameters controlling the regularization strength, and s, is adaptive weighting term. For more details, refer to Sec. A.0.1 Sec.1.0.1
EXPERIMENTS
In this section, we evaluate ODE-GS for extrapolating dynamic 3D scenes. We present quantitative results in Sec. 4. to assess rendering quality on unseen future timestamps across D D-NeRF (Pumarola al. 2021), NVFi (Li et al. 2023), and HyperNeRF (Park et al., 202 and qualitative evaluations in Sec. 4.2 to demonstrate quality of the extrapolated dynamics through visual coherence.

---

Published as a conference paper at ICLR 2026
Table I: Quantitative extrapolation results on D-NeRF dataset. Metries reported include PSNR, SSIM, and LPIPS-vgg. Best metric is highlighted in red, and second best in orange.
Lego
Mutant
Stundup
Trex
Metled
PSNR(T)
SSUMITT)
[.IPS(()
PSNR(T)
L.PISS()
TiNeaVou-B
23.34
.1022
21.77
99199
4D-G8
24.25
9150
.880
22.46
9300
0520
15.61
91100
.840
23,83
94660
2510
Deformable GS
23.25
1939
8679
25.45
9310
.411
21.37
9124
1964
20.74
9421
GrussiaaPrediet
12:25
-7594
26.91
9486
4D Rotee Caussaas
22.32
9178
0705
30,62
.9666
40252
25.79
9305
.6522
20.21
9425
0692
Jumpingiacks
Bouncinghalls
Helwarriir
Hook
Method
PSNR(1))
SS0M(t)
(.PPPS(1)
(SNR(1))
SSIM(t))
[PPPS())
PSNR(11)
SSIM())
[.PIPS())
PSNR(1))
SSIM())
[MIPS())
TiNeaVar-B
19387
.115
.0154
25.92
.9677
.0853
29.36
-9097
.1138
21415
.8877
.103
4D-C8
19.55
0220
.070
16.84
.8790
Deformble-GS
20.32
9162
.099
29.49
.9804
.0337
30.15
9172
.0799
21.60
8876
00020
PrussianPreiict
20.12
91100
.0811
25.09
9779
.0322
30.75
9264
4767
23:75
.9112
10533
Rotic Gauslaas
20.33
9063
Ours
22:18
19243
4015
24.91
9660
.0472
31,80
0366
.6666
28.33
.9893
.0343
Table Quantitative extrapolation results on NVFi dataset scenes. Metrics reported include PSNR, SSIM, and LPIPS. The best metric is highlighted in red, second-best is highlighted in orange.
PSNBR:11
S41k611
(
PSNRR1))
1
PSNNR11
SSIMI1)
1/P1P5:11
(SNN211)
FSN4811
SSIMI1
1.PIPS/11
Dsfemassl-GS
10355
20.11
23307
4D-0S
24.78
2055
8417
22.31
-3635
0370
22.56
2645
.033
1933
3555
12:70
5514
(8333
39.21
3908
1024
25.11
2030
15442
29:51
1991
2006
22.99
3357
1751
21:94
.3811
J3453
NVH
27.17
1670
26/03
3730
30298
25.87
5621
6220
23322
9658
17.50
5630
J3450
33..9
9211
1005
33.86
21235
18.75
6662
96.00
0835
16.59
5984
Meshed
TiNeWNo
50.00
1000
21.36
2521
36.01
1813
13.56
25.36
$220
21.59
22.00
1352
Casmtnelrodkt
2012
.153
21.0
.340
19101
.001
21.33
33541
.2709
XVF
31.30
3791
1910
25.54
2299
30.41
3360
2230
2500
2898
.1700
31.72
5060
.1540
Dars
22.62
0041
1981
33.38
39366
11776
54.16
1911
13155
38.30
3013
11451
04.53
91185
(9806
4.1 QUANTITATIVE RESULTS
D-NeRF, On the D-NeRF dataset (Table 1.. ODE-GS consistently surpasses both interpolation- based baselines (Deformable-Gr 4D-GS, Rotor- and extrapolation-oriente methods (GaussianPredictt, Our model averages 27.30 Peak Signal-to-Noise. Ratio (PSNR). 0.9497 SSIM, and 0.0467 LPIPS-vgg (Zhagg et al. 2018), increasing metric performance against previous SOTA GaussianPredictioon by 18.6%. We have especially large margins in Mutant (+10 dB) and Standup (+7 dB), where scene motion are very smooth and follow a straightforward trajectory. These gains directly validate our motivation: interpolation methods degrade when extrapolated beyond the training window due to their timestamp-condiice
design, while our Latent ODE approach enables extrapolation.
NVFI. The NVFi benchmark (Table 2 emphasizes robustness under both controlled single-object motion (fan, shark, telescope) and complex multi-object indoor dynamics (factory. darkroom, chess- board). ODE-GS achieves new state of the across nearly all sequences, averaging 33.43 PSNR, 0.9471 SSIM, and 0.0603 LPIPS, improving previous SOTA method NVFi by 20%, and GaussianPre- diction by 39.6% on averaging across scenes and metrics. Notably, cluttered or occluded settings like factory and darkroom, model reduces perceptual error (LPIPS) by more than 40% compared to baselines, demonstrating the benefit of decoupling reconstruction from forecasting.
HyperNeRE. On real-world HyperNeRF scenes (Table 3), ODE-GS establishes consistent improve- ments over baselines despite the higher noise and irregular dynamics of captured videos. Our method delivers lower LPIPS in split-cookie, slice- slice-banana, banana, and cut-lemon, and outperforms in both PSNR and SSIM in chickchicken. These results show that ODE-GS avoids overfitting to noisy timestamp signals and remains stable under real-world non-idealities.
7

---

Published as a conference paper at ICLR 2026
Table Quantitative extrapolation results on HyperNeRF Metries reported include PSNR, SSIM, and LPIPS. The best metric is highlighted in bold, second-bent is underlined.
split-cookie
slice-banana
cut-Jemon
Method
PSNR(T)
SSIM(T)
LPIPS(L)
PSNR(()
SSIM(1)
LPIPS(.)
PSNR(T)
SSIM(1)
LPIPS())
TiNeuVox
16.67
.6135
4778
18.44
.6242
.6119
18.84
.6228
.543
Deformable GS
17.84
5698
.9455
21.73
.6330
.3241
21.36
.6950
.3007
GaussiamPredict
16.93
5604
.3336
21.97
.6110
.3749
20.91
.6137
.3220
21.69
.098
hoard
3dprinter
chickchicken
Method
PSNR(T)
SSIM(T)
LPIPS())
PSNR(T)
SSIM(1)
LPIPS())
PSNR(T)
SSIM())
LPIPS())
Vox
19.03
6823
4665
18.16
.878
.9499
15.23
.638
5888
Deformable-GS
6957
19.89
19.03
.3677
GrudsianPredictt
20.13
.6999
.2511
19.96
.6468
21.94
.6933
.3284
Ours
21.06
.7399
.2327
20.22
.6946
.2242
20.29
.7254
.3023
Deformable
Residual
GoussianPrediction
Residual
Residual
standup
lego
hellwarrior
hook
Figure 3: Qualitative visualization on scenes from DNeRF dataset, from left to right are the ground truth image, rendered result from Deformable Ga(Yang et al., 2024), residual of Deformable GS against GT, GaussianPredictionnZZI (Zhao et al. 2024), residual of GaussianPrediction against GT, and finally Our well Ours residual against GT.
4.2 QUALITATIVE RESULTS
To provide more intuitive understanding of our model's performance, we present a qualitative analysis of the extrapolated renderings, shown by Figure which offers compelling side-by-side comparison on five scenes from the D-NeRF dataset. While all models are tasked with predicting the scene at a future, unseen timestamp, the results vary dramatically. We show the difference in predicted motion via residual error maps, which visualize the pixel-wise error between the rendered images and the ground truth, where brighter regions indicate high error and dark regions indicate low error. The error maps for both baselines show bright, structured residuals concentrated around the central object or character, revealing substantial inaccuracies in both shape and position. Conversely, the residual map for ODE-GS is significantly darker and less structured, providing clear visual evidence
8

---

Published as a conference paper at ICLR 2026
Table The effect of additional reprojection loss during training on D-NeRF extrapolation. Metries reported include PSNR, SSIM, and LPIPS-vgg.
Lego
Mutant
Standup
Trex
Medhod
PSNR(T)
SSIMC1)
LPIPS()
PSNR(")
SSIML1>
LFSPS())
PSNR(1)
SSIM/1)
LPIPS(.)
PSNR())
SSIM(T)
LPIPS())
28.91
.9577
.099
22.04
9475
Ours with reprojection
25.63
9068
X649
29.73
9671
0219
29.27
95-65
(090
22.02
9469
0496
Jumpingjacks
Bouncinghalls
Hellwarrtor
PSNR(1)
SSIM
LPIPSL)
PSNRL")
SSEMI1)
LPSPS())
PSNR(1)
SSIM/
LPIPS(..)
PSNRI")
SSDM(T)
LPIPS(
Ours
22.18
.243
0015
24.11
9660
0472
31,00
9365
0696
28.33
9493
0043
Ours with reprojection
31.32
9205
4058
25.96
9684
0436
31,8
9931
1004
2780
9454
0168
of much lower prediction error. This demonstrates that by learning the underlying dynamics in continuous latent space, ODE-GS not only preserves the high-frequency details but also forecasts motion more accurately for photorealistic novel-view synthesis in future frames. In particular, the scene Lego is recognized to have inaccurate poses by previous studies (Zhao et al. 2024 Yang et al. 2.24), but our method can still extrapolate the scene that matches the ground truth image with low eπror, while methods like Gaussian Prediction (Zhao et al., (2024) can be less robust to such noises and fail completely at extrapolating the scene. For additional qualitative on NVFi see Figure
4.3 ABLATION STUDY
We compare ODE-GS against four different abla- tion settings: Without ODE formulation (pure au- toregressive Transformer baseline trained under the same input-output supervision), without regulariza- tions, without adaptive regularization scaling, and without dynamic sampling. The results are summa- rized through average metrics in Table 4 In gen- eral, our full method outperforms all ablation settings. For ablation setting without ODE, we the Transformer encoder and decoder in a pure autore- gressive style. Specifically, this model variation has the same architecture only without the neural ODE. However, the autoregressive Transformer directly pre- dicts the next Gaussian parameters in discrete fixed steps, as each predicted output then fed back as input for subsequent predictions. As shown in Ta- ble 4 the autoregressive bascline significantly under- performs, almost doubling the LPIPS metric. This highlights limitation of discrete autoregressive mod- els also discussed in previous works (Chen al. 2018 Rubanova et al. 2019): they lack the inherent smoothness prior naturally expressed by the ODE formulation. Therefore, juruss jumps or oscillations in the predicted dynamics can occur. For the setting without regularization, we used only the extrapola- tion loss I., for training objective. As shown on Table 4 the additional regularization narrows down solu- tion space further, which results higher average metrics. In Figure 4 we observe these additional constraints helps the model improve on scenes with more complex and diverse motion such the dining and hell-warrior scene. For per-scene results, refer to Table 8 in the Appendix.
Table 4: Ablation study average results over the NVFi dataset.
Method
PSNR(T)
SSIM(1)
LPIPS(U)
W/o ODE
23.71
.79
.113
wio Regularization
32.90
943
066
wlo Adaptive reg.
32.19
.388
.68
wlo Dynamic sampling
31.35
935
069
Ours
33.43
.947
060
reg.
Ours
Dining
Hellweerior
Figure 4: Qualitative comparison on ODE-GS trained using latent and trajectory regulariza- tion vs. using only extrapolation loss on two selected scenes. We highlight the areas within each scene with highest visual disparity.

---

Published as a conference paper at ICLR 2026
Average PSHA,
Time (Higher better]
Avarage 55TH Time
Average LIMPS Time
better)
(Lower is
03 0dtm
014
Oatmm
1.3
012
0.3
1.60
1.0
3.25
045
0.00
6.25
043
6.0
3:00
205
1.80
6.6
0.55
053
6.0
1.6
Figure Visualization of the degrade in metrics through time for extrapolation. Y axis is the corresponding metric and X axis is the time or frame where the metric was evaluated. This graph is the average over all scenes in NVFi dataset.
4.4 ADDITIONAL EXPERIMENTS
Reprojection loss joint training We have experimented with adding the image reprojection loss to the training of the extrapolation model, shown Table Specifically, for every iteration, we sample random camera with ground truth image, and project the gaussians onto the image via the rendering pipeline (Sec. 3). We follow the original 3DGS (Kerbl et al. 2023) and apply LI+SSIM loss. As shown in Table S the projection loss does not provide any significant metric improvement over our base model. This is likely due to the interpolation model being trained using the projection loss and having already accurately represented the training data. Thus, trajectory loss itself is already sufficient as supervision.
Performance degrade analysis Shown in Figure as well as per-scene figure 8 111
we analyzed the performance degrade of the rendered results during extrapolation period. On the NVFi dataset, our model not only shows an overall high metric performance, but also a slower degrade rate in time. Especially the SSIM and LPIPS measures degrade much slower than the baseline Yang al. (024). showing our method's capability of maintaining plausible motion and object appearance as extrapolation length becomes longer.
Effect of different input time spans On certain scenes, the model can benefit from shorter input context time span T.. Specifically, we reduced time span from 0.6 to 0.1 on various train/test time splits over the bouncinghalls scene, ranging from using 60 percent of temporal duration for training to 95 percent, shown in Table 11 in the appendix. Compared to default setting, our method achieves 21.70% average improvement across the three metrics over previous best method in Table
LIMITATIONS AND CONCLUSION
Our model inherits the quality of the underlying interpolation model used to generate Gaussian trajectories. If the interpolation stage fails to accurately reconstruct the scene within the observed window (for instance, in cases of fast-moving small objects like the fallingball scene) then the subsequent extrapolation will propagate these errors forward. In scenarios like bouncingballs and trex where the evolution of the scene undergoes abrupt changes, discontinuities, or fundamentally novel behaviors not foreshadowed by the past, the model's 's predictions degrade. One possible way utilize data-driven priors for extrapolation that can generalize across
to address this limitation scenes instead of overfitting to the observed dynamics of specific scenes. We have conducted preliminary experiment on our model's generalization capability across scenes B.2
In summary, we introduce ODE-GS, which integrates 3D Gaussian Splatting with Transformer- latent neural ODEs to achieve dynamic scene extrapolation. By decoupling scene reconstruction from temporal forecasting, enforcing smoothness through continuous-ting latent dynamics, and incorporating adaptive regularization and dynamic sampling. ODE-GS achieves tftte-of-the-ee on synthetic (D-NeRF, NVFi) and real-world (HyperNeRF) benchmarks. We show that modeling scene evolution as latent flows mitigates out-of tutto--istribution failures common in timestamp-condee conditioned methods and enables accurate extrapolation of scene dynamics beyond the observed time window.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work is supported by NSF 222266 Athena AI Institute and the Yale AI Seed Grant 2025.
ETHICS STATEMENT
This work advances methods for forecasting dynamics in 3D scenes using Gaussian splatting and latent ODEs. Given the scope of the work, we do not identify immediate ethical risks associated with the approach itself. However, as with any machine learning system, outcomes depend on the quality and diversity of the training data. If deployed in downstream applications such robotics or autonomous navigation, inaccurate forecasts of scene dynamics may pose safety risks, and care should be taken validate performance in those contexts.
REPRODUCIBILITY STATEMENT
We provide detailed descriptions of our approach in Sec. including architecture design, sampling strategies, and training objectives. Experimental settings, dataset usage, and evaluation metrics are reported in Sec. B.I with additional implementation details included in Appendix B To ensure reproducibility, we have released the implementation of this paper publicly online.
REFERENCES
Benjamin Attal, Jia- fia-Bin Huang, Christian Richardt, Michael Zollhoefer, Johannes Kopf, Matthew O'Toole, and Changil Kim. Hyperreel: High-fidelity 6-dof video with ray-conditioned sampling. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp- 16610-16620, 2023.
Peter W Battaglia, Jessica B Hamrick, and Joshua B Tenenbaum. Simulation an engine of physical scene understanding. Proceedings of the national academy of sciences, 110(45):1332--18322 2013.
Andrew Bond, lui-Hsien Wang, Long Mai. Erkut Erdem, and Aykut Erdem. Gaussianvideo: Efficient video representation via hierarchical gaussian splatting. arXiv preprint arXiv:2501.0478: 047782 2025.
Michael Broxton, John Flynn, Ryan Overbeck. Daniel Erickson, Peter Hedman, Matthew Duvall, Jason Dourgarian, Jay Busch, Matt Whalen, and Paul Debevec. Immersive light field video with layered mesh representation. ACM Transactions on Graphics (TOG), 39 2020.
Ang Cao and Justin Johnson. Hexplane: fast representation for dynamic scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 130-141, 2023.
Ricky T. Q. Chen, Yulia Rubanova, Jesse Bettencourt, and David K. Duvenaud. Neural ordinary dif- ferential equations. In Advances in Neural Information Processing Systems (NeurIPS), volume 31, pp. 6571-6583, 2018.
Edward De Brouwer, Jaak Simm, Adam Arany, and Yves Moreau. GRU-ODE-Baye Continuous modeling of -poradically-ooserved time series. In Advances in Neural Information Processing Systems (NeurIPS), volume 32, pp. 7366-7376, 2019.
Mingsong Dou, Samch Khamis, Yury Degtyarev, Philip Davidson, Sean Ryan Fanello, Adarsh Kowdle, Sergio Orts Escolano, Christoph Rhemann, David Kim, Jonathan Taylor, et al. Fusion4d: Real-time performance capture of challenging scenes. ACM Transactions on Graphics (ToG), 35 (4):1-13, 2016.
Yilun Du, Yinan Zhang, Hong-Xing Yu, Joshua B Tenenbaum, and Jiajun Wu. Neural radiance flow for 4d view synthesis and video processing. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV) pp. 14304-14314, IEEE Computer Society, 2021.
11

---

Published as a conference paper at ICLR 2026
Yuanxing Duan, Fangyin Wei, Qiyu Dai, Yuhang He, Wenzheng Chen, and Baoquan Chen. 4d-rotor gaussian splatting: towards efficient novel view synthesis for dynamic scenes. In ACM SIGGRAPH 2024 Conference Papers, pp. 1-11, 2024.
Jiemin Fang, Taoran Yi, Xinggang Wang, Lingxi Xie, Xiaopeng Zhang, Wenyu Liu, Matthias Nießner, and Qi Tian. Fast dynamic radiance fields with time-aware neural voxels. In SIGGRAPH Asia 2022 Conference Pupers, pp. 1-9, 2022.
Chen Gao, Ayush Saraf, Johannes Kopf, and Jia-Bin Huang. Dynamic view synthesis from dynamic monocular video. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5712-5721, 2021.
Yi-Hua Huang, Yang-Tian Sun, Ziyi Yang, Xiaoyang Lyu, Yan-Pei Cao, and Xiaojuan Qi. Sc-gs: gaussian splatting for editable dynamic scenes. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4220-4230, 2024.
Kacper Kania, Kwang Moo Yi, Marek Kowalski, Tomasz Trzciński, and Andrea Tagliasacchi. Conerf: Controllable neural radiance fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 18623-18632, 2022.
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Gruph., 42(4):139-1, 2023.
Mina A Khoei, Guillaume Masson, and Laurent U Perrinet. The flash-lag effect as a motion-based predictive shift. PLoS computational biology, 13(1):e105506) 2017.
Patrick Kidger, Ricky T. Q Chen, and Miles Cranmer. torchode: A parallel ode solver for pytorch. https://githun wwtp://cc/pppccicoo kidger/Eorchode 2021.
Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. CoRR, abs/1312.6114, 2013.
URL https://appii semant scmaanicccccccc..... http://ww.......a.ooooooooooaaaa
Jinxi Li, Ziyang Song, and Bo Yang. Nvfi: Neural velocity fields for 3d physics learning from dynamic videos. Advances in Neural Information Processing Systems, 36:34723-34751, 2023.
Zhan Li, Zhang Chen, Zhong Li, and Yi Xu. Spacetime gaussian feature splatting for time
dynamic view synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 3508-8520, 2024.
Zhengqi Li, Simon Niklaus, Noah Snavely, and Oliver Wang. Neural scene flow fields for space-time view synthesis of dynamic scenes. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 6498-6500 2021.
Chen Liu, Ke Xu, Liangbo L Shen, Guillaume Huguet, Zilong Wang, Alexander Tong. Danilo Bzdok, Jay Stewart, Jay C Wang, Lucian V Del Priore, et al. Imageflownet: Forecasting multiscale image- level trajectories of disease progression with pergulaaly-samplee longitudinal medical images. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5. IEEE, 2025.
Stephen Lombardi, Tomas Simon, Jason Saragih, Gabriel Schwartz, Andreas Lehrmann, and Yaser Sheikh. Neural volumes: Learning dynamic renderable volumes from images. arXiv preprint arXiv:1906.07551, 2019.
Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan. Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis. In 2024 International Conference on 3D Vision (3DV), pp. 800-809. IEEE, 2024.
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1)999-106 2021.
Leigh A Mrotek and John F Soechting. Predicting curvilinear target motion through an occlusion. Experimental Brain Research, 178(1):99-1 14, 2007.
12

---

Published as a conference paper at ICLR 2026
Richard A Newcombe, Dieter Fox, and Steven M Seitz. Dynamicfusion: Reconstruction and tracking of non-rigid scenes in real-time. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 143-352,200
Sergio Orts-Escolano, Christoph Rhemann, Sean Fanello, Wayne Chang, Adarsh Kowdle, Yury Degtyarev, David Kim, Philip L Davidson, Sameh Khamis, Mingsong Dou, et al. Holoportation: Virtual 3d teleportation in real-time. In Proceedings of the 29th annual symposium on user interface software and technology, pp. 741-754, 2016.
Julian Ost, Fahim Mannan, Nils Thuerey, Julian Knodt, and Felix Heide. Neural scene graphs for dynamic scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2856-2865, 2021.
Keunhong Park, Utkarsh Sinha, Jonathan T Barron, Sofien Bouaziz, Dan B Goldman, Steven Seitz, and Ricardo Martin-Brualla. Nerfies: Deformable neural radiance fields. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 5865-5874, 2021a.
Keunhong Park, Utkarsh Sinha, Peter Hedman, Jonathan T Barron, Sofien Bouaziz, Dan B Goldman, Ricardo Brruala,,, and Steven M Seitz. Hypemerf: A highhe-ddimnnsional representation for topologically varying neural radiance fields. arXiv preprint 0rXiv:2106.13228, 202
Sida Peng. Junting Dong, Qianqian Wang. Shangzhan Zhang, Qing Shuai, Xiaowei Zhou, and Hujun Bao. Animatable neural radiance fields for modeling dynamic human bodies. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 14314-14323, 2021a.
Sida Peng, Yuanqing Zhang, Yinghao Xu, Qianqian Wang, Qing Shuai, Hujun Bao, and Xiaowei Zhou. Neural body: Implicit neural representations with structured latent codes for novel view synthesis of dynamic humans. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9064,, 2021b.
Albert Pumarola, Enric Corona, Gerard Pons-Moll, and Francese Moreno-Noguer. D-nerf: Neural radiance fields for dynamic scenes. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10318-10327, 2021.
Rajesh PN Rao and Dana H Ballard. Predictive coding in the visual cortex: functional interpretation of some extra-classical receptive-field effects. Nature neuroscience, 2(1):79-87, 1999.
Yulia Rubanova, Ricky T. Q. Chen, and David K. Duvenaud. Latent ordinary differential equations for pregullarly-smmple time series. In Advances in Neural Information Processing Systems (NeurIPS), volume 32, pp.
2019.
Edgar Tretschk, Ayush Tewari, Vladislav Golyanik, Michael Zollhöfer, Christoph Lassner, and Christian Theobalt. Non-rigid neural radiance fields: Reconstruction and novel view synthesis of dynamic scene from monocular video. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 12959-12970, 2021.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017
Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 4d gaussian splatting for real-time dynamic scene rendering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 20310-22 10310-20320, 2024.
Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. Deformable 3d gaussians for high-fidelity monocular dynamic scene reconstruction. In Proceedings of the IEFE/CVF conference on computer vision and pattern recognition, pp.
2024.
Caglar Yildiz, Markus Heinonen, and Harri Lähdesmäki. ODE2VAE: Deep generative second order
ODEs with bayesian neural networks. In Advances in Neural Information Processing Systems (NeurIPS), volume 32, pp. 10280-10290, 2019.
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features perceptual metric. In CVPR, 2018
13

---

Published as a conference paper at ICLR 2026
Boming Zhao, Yuan Li, Ziyu Sun, Lin Zeng, Yujun Shen, Rui Ma, Yinda Zhang, Hujun Bao, and Zhaopeng Cui. Gaussianprediction: Dynamic 3d gaussian prediction for motion extrapolation and free view synthesis. In ACM SIGGRAPH 2024 Conference Papers, pp. 1-12, 2024.
14

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.0.1 DETAILS ON ADAPTIVE WEIGIITING FOR REGULARIZATION
This mechanism operates each training iteration by using the Exponential Moving Average (EMA) of the trajectory prediction loss as proxy for the model's convergence state. The regularization weight increased as the extrapolation loss decreases, allowing the regularizers to more strongly guide the final, fine-grained trajectory predictions.
The scaling factor 8, at iteration computed as:
= exp --1 clip
(LEMA - Lend 0, 1 1)
(10)
Cinit
Lend
where:
LeMA (4) is the EMA of the trajectory prediction loss at iteration I.
Linit and Lend are hyperparameters representing the expected initial and final extrapolation
loss values.
is temperature hyperparameter that controls the decay rate of the scaling. A lower leads to faster decay and more aggressive increase in regularization.
The 'clip" function normalizes the loss into the range [0, 1], ensuring the scaling factor remains between (0, 1].
The EMA of the trajectory prediction loss, LEMA (I), is calculated as:
CEMA (t) = a LEMA 1) + (1 a) (t)
(11)
where (t) is the extrapolation loss at the current iteration and a is the EMA decay rate. The final regularization weights for the latent and trajectory regularizers then scaled by 8; at each iteration before being added to the total loss.
A.0.2 PROBABILISTIC FORECASTING WITH A VARIATIONAL LATENT ODE
Many past Latent ODE works that focus extrapolation and forecasting has been using variational formulation for the model (Rubanova et al. 2019). Specifically, they follow the Variational Autoen- coder (Kingma & Welling 2013) approach. We provide details on our model variant which uses this approach instead of deterministic modeling, as well as quantitative results compare the two methods on the NVFi dataset, as shown in table
Variational Variant for the Transformer ODE architecture. Instead of mapping the observed history to single initial state for the ODE, prior works often formulate trajectory forecasting a probabilistic problem (Chen al. 2018). Given a historical trajectory */ {G2 (t)))) for a Gaussian k, the Transformer encoder produces latent vector hi. A projection head then parameterizes a Gaussian posterior distribution over the initial latent state,
gol (x() m)) , diag(o",))
(12)
where zμ (fo) is the latent state at the start extrapolation. During training, we sample from this distribution via the reparameterization trick:
zx (fa) Mie oz O
c~ T).
(13)
The sampled latent state 24 (to) then evolved forward in time by numerically solving the latent ODE,
m aopesolvele (Jo. (4)),
(14)
and subsequently decoded to predicted Gaussian parameters
Gx(() (zk (1))-
(15)
15

---

Published as a conference paper at ICLR 2026
Table 6: Comparison between our deterministic and variational formulations on NVFi dataset scenes. Metrics reported include PSNR, SSIM, and LPIPS.
Ours (Deterministic)
Ours (Variational)
Scene
PSNR(1)
SSIM(T)
LPIPS(()
PSNR(1)
SSIM(1)
LPIPS(()
factory
34.53
.9183
.1006
22.57
.7851
.218
dining
30.30
.8299
.1451
16.19
.3981
.595
darkroom
34.16
.9199
.1155
22.83
.7098
.233
whale
33,86
.9859
.0135
26.49
.9011
.038
shark
38.73
.9922
.0082
29.43
.9652
.030
chessboard
33.38
.266
.0976
19.69
.6677
.244
bat
36.68
.9255
.0176
28.01
.9544
.043
fan 33.49
.711
.0303
21.67
.8589
.114
telescope 36.57
99844
.0057
22.66
.9283
.053
fallingball 22.62
.9444
.0684
9,45
.7559
.206
Objective. The variational model is trained by maximizing the Evidence Lower Bound (ELBO), which corresponds to minimizing
Lvare
Σ TE
log (Gx(tt) Ga(t)) + KI( 7h) p(=x(to))]
(16)
prediction NLL
where Te denotes the extrapolation timestamps. The first term encourages predicted Gaussian trajectories G& (t) to align with ground truth (t), the second regularizes the latent space toward unit Gaussian prior p(=)(to)). The final objective is then to compose this loss with regularization as discussed in 3.4
While variational formulations provide principled way to capture uncertainty in trajectory fore- casting, results show that the deterministic version of ODB-GS significantly outperforms its variational counterpart across all NVFi scenes. This gap arises because variational training introduces additional complexity through posterior sampling and KL regularization, which can destabilize opti- mization when data is limited and the ground- dynamies are relatively deterministic, In practice, the model tends underfit sharp motion patterns, producing blurred or averaged predictions that reduce both PSNR and SSIM while inflating perceptual error (LPIPS)) By contrast, the deterministic formulation directly learns smooth latent flows aligned with observed trajectories, avoiding posterior collapse and better exploiting the strong spatio-temporal smoothness priors inherent in dynamic 3D scenes.
B DETAILS ON DYNAMIC TRAJECTORY SAMPLING
We provide additional information sampling strategy introduced in Section 3.3
Training data
for the extrapolation module are derived from the pre-trained interpolation model, which can generate Gaussian trajectories arbitrary real-valued timestamps through the deformation function D...
Each training sample is indexed by Gaussian & and a starting time lo- We fix the number of context steps Nº, target steps Ne. and the context time span Te.
The context sequence is uniformly sampled as
A-
T -
7 1 (G + iA.) IN=
(17)
The target sequence begins immediately after the context window at lond = to Te and spans the remaining horizon:
Te fmax
end c
A
Te
(0) = GG + i J.
(18)
This ensures N. and N. remain fixed across all samples, while T. varies with to. The variability in target length naturally generates prediction tasks of differing difficulty, ranging from short- to long-term forecasts.
16

---

Published as a conference paper at ICLR 2026
Table Per-scene ablation over NVFi of our full method against removal of ODE and removal of dynamic trajectory sampling. Metrics reported include PSNR, SSIM, and LPIPS.
w/o ODE
w/o dynumic trajectory sampling
Ours
Scene
PSNR(1)
SSIM(1)
LPIPS())
PSNR(1)
SSIM())
LPIPS(L)
PSNR(1)
SSIM(1)
LPIPS(J)
factory
24.06
814
.179
33.84
.915
.102
34.53
918
.101
dining
20.98
784
.217
25.47
810
.044
30.30
883
145
darkroom
22.08
.731
33.89
.117
34,16
902
chessboard
19.84
.784
.255
32.27
.900
.100
33,38
997
.988
bat
28.42
960
.039
35.37
.999
.033
36.68
.93
.018
telescope
.977
33.09
.066
fallingball
17.14
9004
.944
26.22
.936
.044
22.62
924
.688
fan
23.35
.931
.052
26.97
.948
.04
33.49
.91
.030
shark
29.47
.967
33.17
38.73
.008
whale
28.51
.973
.088
33.21
.984
.015
33.86
.986
.014
Average
23.71
.879
.113
31,35
.955
.069
33.43
.947
.060
The complete dataset is defined as the union over all Gaussian indices and valid starting times:
D
U k=1
(19)
toET
During training, the context To is processed by the Transformer encoder-ODE module to predict Te (Gu(t)eE.. The prediction is supervised with L1 extrapolation loss:
Le
> -G
(20)
At inference, we simply take the final context segment of length N₂ from the observed window and apply the same extrapolation procedure.
B. IMPLEMENTATION DETAILS
Model architecture. The Transformer encoder has dmmddal = 128 latent dimensions with Yleaad = attention heads and Lene 5 encoder layers. For the latent ODE component, set the latent dimension to discen = 64 and use an MLP with Lode = layers and dhidden = 64 hidden units per layer. The decoder network mirrors this structure with Tetec 5 layers and dnidden 128 hidden units. For the ODE function, we use Tanh activations for smooth and bounded outputs. For the interpolation model, we follow Deformable GS (Yang al. 2024) implementation.
Hyperparameters. Our two-stage training pipeline is implemented in PyTorch with the original 3D Gaussian Splatting renderer (Kerbl al., 2023). For the interpolation stage, set the learning rate to x 10-4 with a cosine annealing scheduler that decays to minimum of 1.6 x 10-6 training for 40k iterations on images with timestamps prior to our speesic- train/test split (Sec. B.2.1).
For the extrapolation stage, we solve latent ODEs using the torchode package (Kidger et al. 2021 with the adaptive DOPRIS solver configured with tolerances rtol 10 and atol 10 We train on NVIDIA GPUs (RTX 3090 or A6000) with batch size of 512 for 40 epochs, using an initial learning rate of 10 10-3 and cosine annealing down to 10 The input context sequence length is set to N, 30 and the extrapolation length N, = 10 during optimization. For adaptive trajectory sampling, the temperature parameter is set to 0.05 on D-NeRF and 0.15 on HyperNeRF and NVFi. For adaptive regularization,
Linit 0.02 for D-NeRF and NVFi and 0.05 for HyperNeRF, we set Lend = 0.0 over training on all datasets. Our EMA decay rate set at 0.9 We Auraj 10-1 for the trajectory regularizer and Atatem 10-5 for the latent regularizer.
B.2 MULTI-SCENE GENERALIZATION EXPERIMENT
We evaluate our extrapolation model's capability to generalize across scenes in Table Specifically, set up mult-scene training experiment where collect all generated trajectories from the
17

---

Published as a conference paper at ICLR 2026
Table 8: Per-scene ablation over NVFi of our full method against removal of regularizers and removal of adaptive regularizer scaling. Metrics reported include PSNR, SSIM, and LPIPS.
w/o regularizers
w/o adaptive regularizer scaling
Ours SSIM(T)
Scene
PSNR(1)
SSIM(T)
LPIPS(L)
PSNR(1)
SSIM(1)
LPIPS())
PSNR(1)
LPIPS(L)
factory
34,68
.999
102
33,53
912
.104
34.53
.988
.101
dining
27.26
.411
.180
25.99
828
.187
30.30
.883
.15
darkroom
33.81
.86
123
32.32
880
.127
34.16
.902
.166
chessboard
33,16
.999
.966
32.16
921
.099
33,38
.927
.98
bat
37.43
.985
.017
36.34
.93
.018
36.68
.018
telescope
36.23
.888
006
35.68
988
.006
36.57
.988
.006
fallingball
19.35
918
.086
19.53
918
.085
22.62
.924
.068
fan
34.96
.976
.027
35.23
.974
.028
33.49
.030
shark 38.26
.989
.009
36.63
986
.010
38.73
.989
.008
whale 33.88
.987
.013
34,52
.987
.013
33,86
.986
.014
Average 32.90
33,43
.060
respective interpolation models from each scene in the NVFi dataset, union them into the same dataset, and train one extrapolation model (The Transformer Latent ODE) on all trajectories simultaneously This is equivalent to having the scenes share the same set of extrapolation model weights during training. To assess the generalization of the model, we intentionally hold out the whale scene's observed trajectories to be excluded from this dataset union. The last rows (Ours Multi) of Table 9 shows our quantitative results of extrapolation for each scene using this shared-weight model for extrapolation, conditioned on the last segment of Gaussian trajectories for each scene's 's unique interpolation model. Our quantitative numbers show that our extrapolation model can fit to multiple scene trajectories at the same time, while being capable of generalizing to unseen scene dynamics with simple motion patterns like whale. Although simultaneously training on multiple scene trajectories does not make the extrapolation model exceed metric performs compared to Our default per-scene training, it shows the potential of such and approach for future works to explore.
B.2.1 DATASETS DETAILS
We evaluate ODE GS on three datasets: the D-NeRF dataset (Pumarola et al. 2021), the NVFi dataset (Li et al.) 2023), and the HyperNeRR dataset (Park et al. 2021b). The D-NeRF dataset comprises eight synthetic dynamic scenes (Lego, Mutant, Standup, Trex, Jumpingjacks, Bouncingballs, Hellwarrior, Hook), each containing 100-200 training images and 20 test images with timestamps normalized from 0 to I, rendered at 800x800 resolution with black backgrounds. The NVFi dataset is also synthetic dataset that provides two subcategories: the simpler Dynamic Object Dataset featuring rotating objects (fan, whale, shark, bat, telescope) and the more challenging Dynamic Indoor Scene Dataset (chessboard, darkroom, dining. factory) containing multi-object scenes with occlusions and realistic lighting variations. The HyperNeRF dataset is series of monocular videos day- day-to-day scenes with varying motion complexity. We use the split-cookie, slice-banan, cut-lemon, keyboard, 3dprinter, and chickchicken scenes, picked at random. For the D-Nerf dataset, we use 80% of the temporal sequence for training and reserve the final 20% as ground truth for extrapolation evaluation. For the HyperNeRF dataset, we use the first 90% of the temporal sequence. For the NVFi dataset, we follow the original train-test split where 75% of the temporal sequence is used for training and the other 25% is reserved for testing. Our model's context window time span is set differently for each dataset, with T. 0.6 for D-NeRF, T. 0.5 for NVTi and HyperNeRF.
B.2.2 JOINT TRAINING EXPERIMENT
In order to justify our design of training the extrapolation model while freezing the interpolation model, we'se 'e tested our method with end-to-end training on both models simultaneously. Specifically, we keep the parameters of the interpolation model to be trainable while we train the extrapolation model, supervised by the same loss. As shown by Table 10 the overall performance decreases on all three metrics.
18

---

Published as a conference paper at ICLR 2026
Table 9: Quantitative extrapolation results on NVFi dataset scenes for training the extrapolation model on multi-scenc setting. Scene whale is not included in the training data. Metries reported include PSNR, SSIM, and LPIPS. The best metric is highlighted in red, second-best is highlighted in orange.
fan
dining
factory
Method
PSNR(T)
SSIM(1)
LPIPS(C)
PSNR(1)
SSIM(1)
LPIPS(L)
PSNR(1)
SSIM(")
LPIPS(
TiNeuVox
26.91
.9315
.0643
23.56
.843
.288
25.36
.222
.3722
Deformahle-Gi
23.75
.974
.0519
20.99
.7222
.168
23.63
.107
.1924
4D-GS
24.78
.9565
.0417
22.08
.4999
.8000
23.42
.2522
.3356
Pressi
30.21
.9622
.0324
18.01
.6845
.3811
21.11
.3390
.2708
NVFi
27.17
.6330
.0370
29.01
.8980
.1710
31.72
.9080
.1540
Ours Det.
33.49
.9111
.0303
30,30
.8829
.1451
34.53
.9183
.1066
Ours Multi.
27.30
.9494
.0422
27.61
.8330
.1911
31.84
.9556
.1110
shark
bat
telescope
Method
PSNR(7)
SSIM(1)
LPIPS(J)
PSNR(1)
SSIM(1>
LPIPS(L)
PSNR(1)
SSIM(1)
LPIPS(
Vox
30.95
.6566
.0367
28.65
.9444
0663
27.04
.9977
.507
Deformable-Gi
29.11
9672
.0233
27.07
9456
0482
22.92
9346
0459
4D-GS
22.56
.6488
.0333
19.83
.9565
.0466
22.77
.9114
0432
GaussianPrediet
29.91
.6955
.0995
22.96
.9587
.0761
21.94
.9811
0045
NVFi
28.87
.820
.0210
25.02
.9680
.0420
27.10
96.30
.460
Ours
38.73
.9892
.0082
36.68
.9825
.0176
36.57
.9884
.0057
Ours Multi.
37.31
.9869
.0094
36.72
.9818
.0188
34.19
.9838
.0885
fallingball
chessboard
darkroom
Method
PSNR(*)
SSIM(1)
LPIPS(})
PSNR(1)
SSIM(1)
LPIPS(L)
PSNR(1)
SSIM(1)
LPIPS(
TiNeuVox
30,00
.5000
.0400
21.76
.7567
2421
24.01
.7400
1813
Deformahle GS
24.50
.2000
0600
20.28
.7866
.2277
22.56
7423
2232
4D-GS
22.00
9100
.0700
20.71
81999
.344
21.99
.7375
4036
GaussianPredict
21.50
90000
.0800
20.12
.7283
.1668
20.01
.6371
.840
31.37
.9780
.0410
27.84
.8720
.2100
30.41
.260
.2300
Ours
22.62
.9244
.0684
33,38
.9266
.0976
34.16
.9019
.1155
Multi
23.54
.9199
.0779
29.51
.8949
.1116
29.92
.8489
.1364
whale
average
Method
PSNR(1)
SSIM(t)
LPIPS(J)
PSNR(()
SSIM(1)
LPIPS(L)
ToxeuVox
27.20
94300
.0799
26.54
8826
1005
Deformable-GS
26.58
.9005
.0886
24.14
.8787
.127
4D-GS
22.31
9638
0370
22.25
.9966
1638
GressianPrediet
25.11
.9610
.0442
23.09
.8584
.660
NVFi
26.03
.9780
.0290
28.45
.9336
.1024
Ours
33.86
.9859
.01.35
33.43
.9471
.0603
Ours Multi
29.22
.9693
.0260
30.72
.9285
.0701
Table 10: Comparison between per-scene (Ours) and joint training (Ours joint) on the D-NeRF dataset. Metrics reported include PSNR, SSIM, and LPIPS-vgg.
Method
Metric
Lego
Standup
Jumpingjacks
Bouncingballs
Hellwarrior
Avg
Ours
PSNR(1)
34.53
22.18
28.33
27.31
Ours joint
PSNR(1)
24.93
21.10
20.82
21.19
19.62
19.60
27.75
19.28
21.79
Ours
SSIM
.3788
.9844
.9557
9475
.9243
.9660
.9655
.993
.497
Ours joint
SSIM
9363
8879
.097
9371
.121
.967
.948
.632
.122
LPIPS(
.0126
.0715
Ours_joint
LPIPS())
.065
.1144
.1087
.1091
.0894
.1087
.1339
.0884
19

---

Published as a conference paper at ICLR 2026
GeussianPrediction
Residual
Residual
bat
telescope
chessboard
dining
Figure 6: Qualitative results on 5 scenes from the NVFI (Li al., 2023) dataset, from left to right are the ground truth image, rendered result from Deformable GSO Yang et al., (2024), residual of Deformable GS against GT, GaussianPredictinn against GT, and finally Our well Ours residual against GT.
al. 2024), residual of GaussianPredicttion
(ttigee better)
(Higher la better)
(Lower la batter)
4.5
04bm
310
3.04
360
3.00
054
360
Time Framm
(Rome
Time (Faama
Figure 7: Metric performance on extrapolation over time compared with Deform-GS on scene bat from NVFi
chezaboand
Time (Higher better)
chassboard [Lawer better)
(Higher better)
Deter
313
1.000
1.25
225
8.550
6080
360
022
1100
Time Thoma 120
Time Thume
Tr thume (2)
Figure 8: Metric performance on extrapolation over time compared with Deform-GS on scene chessboard from NVFi
20

---

Published as a conference paper at ICLR 2026
darkrwon Pask (68) vs Time (Higher
darkrros SSiN vo Time
darkroore Leips Time
better)
00E
17.%
A2200
3.7
0390
22
-EC Octer
0325
8.70
Time (hamm R21
Time (hooe
Tiee (hame
Figure 9: Metric performance on extrapolation over time compared with Deform-GS on scene bat from darkroom
dining PSWK (48) Time
dining SSIN Time (Hligher batter)
dieing LPIPS vs Time (Lower
better)
00C
1.30
8:32
039
034
Time: (hamme
Tim (hrme
Tinm: 2hamed
Figure 10: Metric performance extrapolation over time compared with Deform-GS scene dining from NVFi
factory (dik) Time
factiry SSIM us Time (Hiigher
factory LPIPS vs. Time (Lower
(iligher better!
005
1:30
0.2
1.5
014
1.54
0.22
8.20
0EC
Deker
True (Rone LTC
Time Jirvee EX)
Figure 11: Metric performance on extrapolation over time compared with Deform-GS on scene factory from NVFi
fallingbell PENR (d8) Time (fiiher
fallingbal Time better)
fallinghal LPIPS Time
(Lower batter)
375
Derker
770
3.915
362
3.03
360
3555
260
2:01
352
3.09
6352
Time (hree
Time (horse
Time (Wrore
Figure 12: Metric performance on extrapolation over time compared with Deform-GS on scene falling ball from NVFi
21

---

Published as a conference paper at ICLR 2026
fan (JB) vo Time [Higher
fon SSIM vp Time
fan LiPPS vs Time
(Lower
000
2.055
001
17.5
350
0.413
123
300
3005
2.00
350
0.00
Tim thome 100
Tim thooee
Tine thore
Figure 13: Metric performance on extrapolation over time compared with Deform-GS on scene fan from NVFi
shart (tligher better)
shark SCIN Time (Hligher better)
shark LPMPS us Time (Lower
3.99
3:20
1.0%%
0000
39
40.0
Thme Thame 82)
Tim (hrmm
Time: 3hame
Figure 14: Metric performance on extrapolation over time compared with Deform-GS scene shark from NVFi
telescope PSUR (48)) vs Time (Hligeer
telescape SSIM Time better)
telescape LPMPS Time
better)
COC
000
175-
5.5
350
0.7
325
3.04
645
Time Sireee E2g
Tix- (Roves 2P
Time (Riane (T)
Figure 15: Metric performance on extrapolation over time compared with Deform-GS on scene telescope from NVFi
whole (ttigher better)
while SSM vs Time
whale LPIPS Time
(Lower butter)
410
150
3.14
3.00
300
5.00
315
350
Time Drome E21
Ta (horee
Time (Phrne
Figure 16: Metric performance on extrapolation over time compared with Deform-GS on scene whale from NVFi
22

---

Published as a conference paper at ICLR 2026
Figure 17: Qualitative comparison of last frame from scene bat of NVFi by only using an autoregres- sive transformer without the ODE component. From left to right is the GT image, the transformer's rendered image, the baseline (Yang et al. 2024)'s result, the difference between the GT and trans- former's rendered image, and finally the difference between the GT and the baseline's rendered image.
20.45
Figure 18: Qualitative comparison of last frame from scene factory of NVFi by only using an autoregressive transformer without the ODE component. From left right is the GT image, the transformer's rendered image, the baseline (Yang et al. 2024)'s ' result, the difference between the GT and transformer's rendered image, and finally the difference between the GT and the baseline's rendered image.
Table 11: Quantitative extrapolation results on the bouncingballs scene of D-NeRF dataset. Compar- ison between Deformable-GS and Ours across different data splits. The best metric is highlighted in red. For Ours, we set the input time span to be shorten span of 0.1 instead of the default 0.6.
Split 65%
Split 70%
Split 75%
Split 80%
Mathod
1.PPS(())
1PPPS(())
I.PIPS(.)
Deformable GS
24.11
.9614
0833
23.30
.6000
.0500
24.08
9642
.0478
29.49
9804
.0337
26.55
.9115
Split 85%
Split 90%
Split 95%
Average
Method
PSNR(T)
SSIM(1)
LIPPS(II
PSNR(")
SSIM011
LPIPS())
PSNR(")
SSIM(1)
LPIPS())
PSNRRTI
SSIM(T)
LPIPS(.
Deformable-GS
31.44
9642
0197
34,50
9886
0148
34.31
9889
.0131
28.89
9772
0325
Ours
32.97
9860
J0054
36.23
.913
JNL39
39.18
.9944
JNN3E
31.00
9796
J01.38
1T8 DG rander12
11e8 rendert
Figure 19: Qualitative comparison of the fallingball scene in NVFi the interpolation model and the ground truth. As shown, the interpolation model fails to capture meaningful dynamics of the scene while missing the fallingball object.
23


Reviews:
[{"review_id": "77GI3hFgZW", "reviewer": "Reviewer_AuTy", "summary": "The paper points out that existing dynamic 3D reconstruction methods can only handle temporal interpolation and fail to predict future scene dynamics. To address this, it introduces a new task called dynamic scene extrapolation, aiming to forecast future 3D states beyond observed frames. The paper proposes a model called ODE-GS combining 3D Gaussian Splatting with latent neural ODEs. It consists of three main parts: a Gaussian interpolation model for reconstructing observed scenes, a Transformer encoder for encoding motion histories into latent states, and a neural ODE module that evolves these states to predict future dynamics. The overall goal is to achieve physically plausible, temporally smooth, and consistent 3D scene extrapolation.", "questions": "See weaknesses.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "* The paper explicitly defines and tackles dynamic scene extrapolation, which predicting future 3D scene states beyond the observed temporal window. It's a meaningful and underexplored extension of existing dynamic 3D reconstruction research.\n\n* The integration of 3D Gaussian Splatting with latent neural ODEs is conceptually elegant and well-motivated. Modeling Gaussian trajectories as continuous latent dynamics naturally enforces temporal smoothness and physical plausibility.", "weaknesses": "Interesting work. I only have a few questions regarding the experiments:\n\n* I am curious about how ODE-GS performs in dynamic extrapolation on real-world datasets. I could not find clear visualizations of HyperNeRF and more real-world scenes (especially those containing both dynamic and static regions) in the main text or appendix. It would be helpful to see whether the model can accurately distinguish and predict the different evolution trends of dynamic versus static areas in complex real-world scenes.\n\n* I would like to know the effective extrapolation range of the model. The paper does not provide a systematic analysis of how performance degrades with increasing extrapolation distance — i.e., how far into the future the model can extrapolate before failure. This is an important factor for evaluating forecasting models.\n\n* I would like to better understand how ODE-GS performs on the novel view synthesis (NVS) task, both within the observed time window (interpolation) and the future extrapolation period.\n\n* If possible, I would appreciate video demonstrations of the extrapolated results to more intuitively assess temporal consistency and motion realism.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KqFe17NYUZ", "reviewer": "Reviewer_x185", "summary": "This paper proposes to integrates 3D Gaussian Splatting with latent neural ordinary differential equations to enable future extrapolation of dynamic 3D scenes. It models parameter trajectories as continuous-time latent dynamics and ahieves state-of-the-art extrapolation performance. \n\nIt focuses on extrapolation, which is a somehow new problem worth studying. But this is not a totally new field and I believe so many works has been studying this issue. \n\nBut since I am not an expert in this field, I would like read the other reviewer's opinions about the contributions of this work, especially about the novelty.", "questions": "1. Why not supervise the extrapolation module directly with image reprojection error. Another option is using weak supervision or a distillation hybrid approach instead of relying entirely on pseudo-GT Gaussian trajectories produced by the interpolation model.\n2. Beyond smoothness regularization, could you incorporate velocity bounds, momentum conservation, or constrained optimization (e.g., constraint-based pose updates)?\n3. Have you ever run comparisons with end-to-end fine-tuning or mixed supervision?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper combines Neural Ordinary Differential Equations with 3DGS for 4D modeling, which seems to be novel, well-motivated design.\n2. This paper tried to forecast future 3D states in the context of dynamic scene reconstruction (as dynamic scene extrapolation), which is an interesting topic.\n3. The paper proposes a practical and effective two-stage training strategy.\n4. The paper provides consistent gains across D-NeRF, NVFi, and HyperNeRF datasets with comprehensive ablations.", "weaknesses": "1. The foremost weakness is that, there is not enough experimental results on real-world datasets. The results on PlenopticVideo dataset should be included.\n2. It seems that the proposed method \"collapes\" on fallingball, Bouncingballs datasets. The phenomenon should be discussed and more collapsed cases need to be analyzed. The generalizability and robutness need to be discussed.\n3. There has been a lot of advanced SOTA 4D reconstruction methods, but the paper only compare with 4D-GS/ Deformable-GS, which is quite old at this time.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "36nBR40Duq", "reviewer": "Reviewer_MsUn", "summary": "The paper tackles dynamic scene extrapolation problem, predicting future 3D states beyond observed timestamps. The authors propose ODE-GS, which couples 3D Gaussian Splatting with a Transformer-based latent neural ODE: 1) an interpolation stage first fits accurate Gaussian trajectories within the observed window; 2) a Transformer encoder summarizes past trajectories into a latent state; 3) then a neural ODE evolves this state in continuous time, and numerical integration yields smooth, physically plausible future Gaussian trajectories for rendering at arbitrary times. Experiments on D-NeRF, NVFi, and HyperNeRF report consistent rendering quality improvement and demonstrate that ODE-GS achieves state-of-the-art extrapolation task through the proposed latent ODE.", "questions": "1. How are the Gaussian parameters encoded? Which subsets (positions, scales, rotations, opacities, SH features) are fed to the encoder?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- This paper is well-written and easy to understand.\n- This paper targets the underexplored extrapolation problem in dynamic reconstruction and proposes a solution that makes sense.\n- Across both synthetic and real scenes, the results are consistently strong, with visualizations aligning well with the quantitative metrics.", "weaknesses": "- Missing references for some important dynamic reconstruction works:\n  - [CVPR 2024] Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesis, by Zhan Li et al.\n  - [CVPR 2024] SC-GS: Sparse-Controlled Gaussian Splatting for Editable Dynamic Scenes, by Yi-Hua Huang et al.\n- Frozen teacher prevents end-to-end correction. Canonical Gaussians and the deformation network are frozen before training the ODE module, limiting the system’s ability to adjust interpolation when extrapolation exposes inconsistencies. It would be much better if the authors do an ablation about joint training pipeline.\n- Long-horizon robustness not guaranteed. Dynamic trajectory sampling exposes the model to varied horizons, but there’s no quantitative guarantee or analysis of failure rates as horizons grow. It would be much better if the authors could do some experiments to suggest they can prevent potential drift on very long extrapolations.\n- Although ODE-GS leverages a Transformer, it is essentially a per-scene reconstruction method and does not readily scale up into a large model for more generalizable tasks.\n- This method relies heavily on the teacher Gaussians being well fit. When applied to real-world scenes with imperfect camera poses, those errors propagate and can severely degrade the accuracy of ODE-GS extrapolation.\n- Minor typo errors:\n  - L123: `representing dynamics scenes` -> `representing dynamic scenes`\n  - L129: `enables flexible editing` -> `enabled flexible editing`\n  - L129: Add space between `GaussianVideo (Bond et al., 2025))` and `uses`\n  - L680: `focuses on` -> `focus on`\n  - L683-684: `as shown in 5` -> `as shown in Figure 5`", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qOQeQXau8Q", "reviewer": "Reviewer_rwuC", "summary": "The proposed ODE-GS method enhances dynamic scene prediction by first training a pretrained 3D Gaussian deformation (interpolation) network to reconstruct observed scenes and generate continuous Gaussian trajectories. These trajectories are then used to train a Transformer-based latent ODE model, where the ODE solver integrates latent dynamics over time to predict future trajectories.", "questions": "Could the authors provide a visual comparing with and without the ODE component? It would be helpful to understand why the ODE-based model has much better results than the Transformer-only baseline reported in Table 4. If visualization is difficult to include, please report additional quantitative results (e.g., using a simple baseline such as copying the last/nearest frame of interpolation network or scaling up the transformers) on the NVFi dataset for reference. \n\nCould the authors provide training (interpolation pretrained model and the ode part) and inference time comparisons with Deformable 3DGS and GaussianPrediction? This would help assess the computational trade-offs introduced by the latent ODE and Transformer components over baselines.\n\n\nThe paper states:\n“...for the interpolation model, we follow Deformable GS (Yang et al., 2024) implementation.”\nDoes this mean the interpolation model is a pretrained Deformable-GS network reused as a data generator? If so, what is the quantitative performance of this interpolation model itself on HyperNeRF dataset over deformable GS on the interpolation mode?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The idea of introducing an ODE formulation for modeling dynamic 3D Gaussian Splatting (3DGS) is interesting. Representing scene evolution through continuous-time latent dynamics provides a different way to achieve extroplation.", "weaknesses": "1. Complexity and dependence on neural networks\nThe reliance on a neural network make the architecture complicate. It is not entirely clear whether this provides substantial benefits over simply using a learned deformation network.\n\n\n2. Dataset simplicity\nThe benchmark datasets seem to include simple or synthetic trajectories (e.g., Lego, Mutant), which may be easily modeled by smooth ODE dynamics. It remains unclear how the method would perform on more complex or irregular real-world motion patterns, or on camera paths that deviate significantly from known trajectories.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)