WorldTree: Towards 4D Dynamic Worlds from Monocular Video using Tree-Chains
Authors: Qisen Wang
Yifan Zhao
Jia Li
Venue: ICLR 2026 Poster (2026)
Keywords: Novel View Synthesis
Monocular Dynamic Reconstruction
Gaussian Splatting
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Dynamic reconstruction has achieved remarkable progress, but there remain challenges in monocular input for more practical applications. The prevailing works attempt to construct efficient motion representations, but lack a unified spatiotemporal decomposition framework, suffering from either holistic temporal optimization or coupled hierarchical spatial composition. To this end, we propose WorldTree, a unified framework comprising Temporal Partition Tree (TPT) that enables coarse-to-fine optimization based on the inheritance-based partition tree structure for hierarchical temporal decomposition, and Spatial Ancestral Chains (SAC) that recursively query ancestral hierarchical structure to provide complementary spatial dynamics while specializing motion representations across ancestral nodes. Experimental results on different datasets indicate that our proposed method achieves 8.26% improvement of LPIPS on NVIDIA-LS and 9.09% improvement of mLPIPS on DyCheck compared to the second-best method. Code: https://github.com/iCVTEAM/WorldTree.

Published as a conference paper at ICLR 2026
WORLDTREE: TOWARDS 4D DYNAMIC WORLDS FROM MONOCULAR VIDEO USING TREE-CHAINS
Qisen Wang, Yifan Zhao; Jia Li
State Key Laboratory of Virtual Reality Technology and Systems School of Computer Science and Engineering & Qingdao Research Institute, Beihang University (wanggisen, zhaoyf, jial jeuuiaa edu. cn
ABSTRACT
Dynamic reconstruction has achieved remarkable progress, but there remain chal- lenges in monocular input for more practical applications. The prevailing works attempt to construct efficient motion representations, but lack a unified spatiotem- poral decomposition framework, suffering from either holistic temporal opti- mization coupled hierarchical spatial composition. To this end, we propose WorldTree, a unified framework comprising Temporal Partition Tree (TPT) that enables coarse-to- to-fie fine optimization based on the inheritance based partition tree structure for hierarchical temporal decomposition, and Spatial Ancestral Chains (SAC) that recursively query ancestral hierarchical structure to provide comple- mentary spatial dynamics while specializing motion representations across ances- tral nodes. Experimental results on different datasets indicate that our proposed method achieves 8, 8.26% improvement of LPIPS on NVIDIA-LS and 9.09% im- provement of PLPIPS on DyCheck compared to the second-best method. Code: attps://githus: http://wwwb..o/////EE/////////e/
INTRODUCTION
Novel View Synthesis (NVS) has achieved remarkable progress in photo-realistic rendering, e.g. Neural Radiance Field and 3D Gaussian Splatting. Recently, the dynamic NVS, also known as dynamic 3D reconstruction, has achieved rapid advancements, extending the paradigm to tempo- rally varying scenes by leveraging synchronized multi-view video inputs for novel view rendering across arbitrary timestamps. However, significant challenges remain in monocular dynamic recon- struction, which could reduce the dependency synchronized multi-view capturing and enable more practical applications, given the abundance of available monocular videos.
Although the prevailing monocular reconstruction methods have demonstrated significant progress in developing efficient motion representations, there remains a notable gap in sufficiently analyz- ing the video modality in the reconstruction process. An intuitive analysis of previous methods is shown in Fig. [ First, the prevailing methods (Lei et al., 2025 Wang et al. 2024b) commonly adopt global optimization, while disregarding temporal characteristics video modality and relying on the optimization across the entire temporal interval. Furthermore, some researchers (Lei et al. (025) attempt to utilize spatial motion graphs with global spatial fusion but introduce the amorphous rep- resentation and neglect temporal aspects. Others (Liang et al. (2025) decompose 3D deformations into global-loce hicrarchies, yet introduce hierarchical motion coupling. Inspired by the issues en- countered with prevailing methods, aim to develop unified optimization pipeline that adapts to the spatiotemporal characteristics of video modality.
Intuitively, monocular videos exhibit varying deformation patterns across temporal intervals, stem- ming from non-uniform 3D motion of the subject. Furthermore, the deformation characteristics of any temporal interval inherit properties from its parent temporal interval, which simultaneously encompasses the spatial complement to the current interval. The above analysis reveals an inher- ent spatiotemporal hierarchy, i.e. a tree-like structure with ancestral associations. To this end, we propose WorldTree, which includes the Temporal Partition Tree (TPT) and the Spatial Ancestral
"Correspondence should be addressed Yifan Zhao

---

Published as a conference paper at ICLR 2026
Chains (SAC) as shown in Fig. Existing methods employ intire-nteervllwiid optimization (Lei ct al. 2025). whereas TPT implements hierarchical temporal partitioning to handle varying defor- mation patterns. For dynamic representation, existing methods rely global spatial fusion with amorphous representation (Lei et al. 2025) or hierarchical motion composition (Liang et al. 2025). struggling with hierarchical deformation optimization coupling, whereas SAC recursively utilizes hierarchical spatial composition with ancestral specialization to circumvent amorphous representa- tions and optimization coupling.
Specifically, TPT decomposes the monocu- lar sequence into hierarchically organized tem- poral intervals through sequential layering, thereby constructing an inheritance-basee par- tition tree. Additionally, we utilize the attribute independence among nodes the same hier- archical level to enable parallel optimization within the same layer. For each child node in the tree, SAC recursively retrieves its ances- tor chain to obtain hierarchical representations. The node itself models local temporal dynam- ics, while its ancestral chain provides multi- level spatial context. Distinctively, our method specializes the motion representation of each ancestral node to achieve hierarchical motion decoupling. Furthermore, we construct an en- hanced NVIDIA-LS (Yoon et al. (220)) dataset featuring exlended temporal length with varying deformation patterns. To enable more precise qual- ity assessment of dynamic reconstruction, we additionally provide annotated motion masky.
Existing Methods
WorldTree
Estire fnterva
Menorchica Teoyoral Purtiix
Optimization Stratogy
Varying Deformation Patteres
Temporal Partition Tree
Globel Spotta Fasien
Iirrrchia Spatial Compostion
S Ilterrchicaal Modeting Campoootux
Dy Dymamic Representation
Mararshical Madeiing Speciattation
0-
Local wise Interfeenee
0 0-S--6-6
( G-4 D-C G-44
Amorphoss / /Hararchicdl-copping
Spatial Ancertral Chains V
Figure 1: Intuitive illustration of WorldTree.
Our contributions can be summarized as
Temporal Partition Tree builds the inheritance-basee partition tree-structure by adopting the coarse-to-line optimization strategy with hierarchical temporal partition to cope with the varying deformation patterns of monocular videos.
Spatial Ancestral Chains provides a spatially complementary dynamic representation and introduces hierarchical node specialization on ancestral chains, thereby achieving optimiza- tion decoupling while constructing the hierarchical complementary representation.
Enhanced Dataset with extended temporal length, indicating varying motion deformation patterns, and the annotated dynamic masks for precise evaluation.
High-quulity Reconstruction. Experimental results different datasets show that our pro- posed method achieves 8.26% improvement of LPIPS on NVIDIA-LS and 9.09% improve- ment of mLPIPS on DyCheck compared to the second-best method.
RELATED WORK
Novel View Synthesis Novel View Synthesis(NVS) (Avidan & Shashua 1997) focuses on recon- structing the 3D representations for rendering images novel views given set of captured training
Rabby & Zhang 2023 Gao et al,, 20226 Barron et al. 2021 Müller et al.) 2022 Chen al. 2022). milestone in the development of NVS. implicitly represents the 3D scene using the Multi-Layer Perceptron (MLP), and achieves photo-realistic rendering quality. However, its speed of training and rendering is usually slow. 3D Gaussian Splatting (3DGS) (Kerbl et al. 2023 Chen & Wang. 2024 Fei et al. 2024 Yu et al.] 2024), as recently emerged 3D representation, explicitly utilizes a of Gaussian primitives to model the 3D scene, and achieves real-time rendering and high qigh-qualtty reconstruction through dif- ferentiable splatting rendering. Although previous 3D reconstruction work can handle static scenes well, it cannot efficiently handle dynamic scenes.
images. Neural Radiance Field (NeRF) (Mildenhall et al. 2021
Dynamic Novel View Synthesis Dynamic NVS (Zimick et al. 2004; Stich et al. 2008 Gao et al.
2021 Li et al. 2023b 2022) has recently attracted more attention, which aims reconstruct the
2

---

Published as a conference paper at ICLR 2026
Temporal Partition Tree (TPT)
Spatial Ancestral Chain (SAC)
WorldTree
Space
Optrninniie
Tinee
Metecalar Video
Dysamii Represetttion
Spare
Temonal Partition (T.P. Repre. Inberimace
Spuer
Thes
Timse
IPT SAC
Motion Bosis
Ganotan Flalik
Teeppera Partifien
baterval Optiaization
Tras-Nodo
Spatial Cleai
Interval Oppiniiation
Tree-Nade
Y.P.
>>
Sputial Ancestaal dSACy
TPI-SAC Precess
Space
Spoce
paints
Mution
Gassiia deformaation
Mation carve
Ancestral el motion corve
SAC- ①
SAC/O
Time
Thas
>)
Training process
quers
Tereral Parttice Hapee. intorfinne
Temarall Pantbe Regre Inhorture.
TPT distingusshigg leftdrightt
Color SAC distinguishing
Figure 2: WorldTree Pipeline. Our proposed method starts by extracting 2D prior results and then initializes the dynamic representation of the tree root. Furthermore, WorldTree builds TPT to achieve temporal coarse-to-fine optimization from the overall interval to the sub-interval of the video, and utilizes SAC to achieve the complementary spatial dynamic representation at the same time, thereby achieving high-quality dynamic reconstruction.
4D representation of the dynamic 3D scene given a set of time-synchronized videos, and further render images of novel views at different timestamps (Attal et al. [2023] [hao et al. 2022 Liu et al.)
2022 Guo et al. 2022 Song al.) 2023). Similar to the static NVS, the previous works mainly modeled based on NcRF The above methods implicitly construct the deformation field through the
mapping between frames and the canonical space, using neural networks for modeling the captured motions. Recently, due to the rapid development of 3DGS, dynamic scene modeling (Duan et al.
2024 Lin et al. 2023 Li al. 2023a Huang al. 2023 Shao al.]
2024) has been migrated to be JDGS-based and achieved faster rendering speed (Duisterhof et al.
2023 Liang et al., 2023
Yang al., 2023a). The 3DGS-based methods usually have faster training and rendering speeds, but like the based methods (Cao & Johnson 2023 Fridovich- Keil et al. (202)), they require -ynchnnchroo multi-view data, which generally needs to be shot specific scene or requires a certain number of cameras, thus limiting the scope of application.
Monocular Dynamic Novel View Synthesis As mentioned above, time-synchronized multi-view
videos are rare in the real world, but monocular causal videos massive. Therefore, how to use monocular videos with few constraints for high-quality dynamic reconstruction becomes an impor- tant challenge (Liu al.] [0044] Seidenschwarz et al., (2224] Yang et al. [202]] You & Hou 2023] Wu et al. 2022 Tretschk et 2021 Das et al.) 2023). The majority of previous works are limited by the dependency on the multi-view clues, e.g. the utilization of pre-computed initialized points (Wul et al.) 2023 Yang et al. 2023b) from SfM (Schonberger & Frahm 2016). Recent work by (Lei itt al., 2025) employs spatial graphs of motion bases for monocular dynamic reconstruction, yet pro- duces an amorphous dynamic representation. Some work (Wu et al. (025) attempts streamable 4D modeling from monocular video, but lacks an explicit mechanism for coarse-to fine temporal par- tition optimization across the entire sequence. The other work (Liang al. 2025) decomposes 3D motion deformations into global-local hierarchies, but their reliance on precise coarse deformation estimation makes them vulnerable optimization conflicts by local-wise interference, ultimately causing hicrarchical motion coupling. Additionally, some researchers (Kwak et al.) 2025) imple- ment temporal splitting by adaptively adjusting temporal intervals, but this division relies on the manual setting of temporal segments and has limited improvement on reconstruction quality. To this end, we propose WorldTree for the monocular dynamic reconstruction as demonstrated in Sec. T
3 METHOD
In this section, keeping the analysis Sec. T for monocular dynamic reconstruction in our mind, propose WorldTree for this task. Given monocular video including set of images T =
3

---

Published as a conference paper at ICLR 2026
(1,7) and its corresponding poses P [Pi (1 where T is the video length, aim to reconstruct the dynamic scene, which includes the static background and moving foreground objects, namely the monocular dynamic reconstruction. The overview of our pipeline is shown in Fig. 2 which includes two main parts, i.e., Temporal Partition Tree (TPT) and Spatial Ancestral Chains (SAC). We first introduce the basic formulation of monoeular dynamic reconstruction from the previous work (Lei et al. 2025) while defining the core symbols, and then we demonstrate TPT and SAC.
3.1 MONOCULAR DYNAMIC RECONSTRUCTION
Lifting 2D Priors. Similar to previous works (Wagg et al. 2024b; Lei et al. 2025 we first lift several 2D pretrained priors to the 3D scene, which needs be reconstructed. Specifically, given the monocular images I, we compute depths ID (D: (et)) from the monocular depth predictor, 2D points trajectories T {p: (e),77 from the point tracking model where på means pixel coordinates and represents i-th trajectory, and epipolar error maps E FFr, (e from the dense optical flow model. The epipolar error maps can be further utilized to compute the EPI masks, which do not need manual point prompts (Wang et al. 2024b; Park et al. 2025 Liang et al. 2025). These predicted results from 2D priors are utilized to initialize the representation of the dynamic scene for further optimization.
Deformation Representation. The deformation is represented as set of motion basis M =
(M, (*) hee[1 with radius r(()), where v [1, N. and M = [R (+) T( SE(3). For each
motion basis vi we compute its KNN graph ( {M E whose
E
I... (D((),
(1)
where the distance D(v, 17) is the maximum spatial distance across timestamps. The deformation of motions can be represented as the relative deformation of the motion bases. Specifically, given
query of position T( from time Le la. the nearest motion basis the at time l, is M()aa and the
deformation field W(T. (z) t, (())), t,, ta) is defined as
FB (BMM, tta
- Wy, (T("), T( (m=)) } Fn EB(w))
where AMP 1, ing (DQB), and Wy, (T((.) ( r ((m)) exp(- - ( -
M 2 M )11 Fn is blending function which is set to Dual Quaternion Blend-
Dynamic Reconstruction. Given the modeling of the deformation field, the dynamic recon- struction can be represented as a set of dynamic Gaussians in the canonical space G = (G,, (In, R2, 8n, On, Cn, tn which represents the position, rotation, scaling, opacity, color, and canonical time of Gaussian n respectively, with the above mentioned deformation fields. Specifi- cally, the deformation of the Gaussian n to target time т the video interval can be represented as
= G (Vo)) ((T..))) (nn,,,), R), 8n, Oar Cn, T)-
(ina) S(W2),)ttn T)
(3)
W(T( t.
So, given the Gaussians G in the canonical space and the deformation field W(-) constructed by G from M, we can model the dynamic scene through the splatting way. However, although the above modeling provides efficient motion representation, lacks optimization based on temporal characteristics and is limited in spatial representations by the global optimization.
3.2 TEMPORAL PARTITION TREE
As we demonstrated in Sec. the classical paradigm of monocular dynamic reconstruction men- tioned above (Lei et al., 2025 processes the video sequence whole and lacks hierarchical mod- eling optimization of time and space for dynamic scenes. To this end, we propose WorldTree for this issue. We first introduce the Temporal Partition Tree (TPT).
Specifically, we start from the tree root (M,, G,) of tree-depth d, = and representing the whole video interval [T,, TR] obtained from the above reconstruction, which represents the basic
4

---

Published as a conference paper at ICLR 2026
Algorithm 1 BFS-Travel Tree-Building
Input:
Tree root G, with motion bases Mr. Gaussian representation Gy. Empty SAC C, = [. Max depth 8. Initial depth d, = 0.
Output: WorldTree Q.
1: Create a tree O, and 1: G,)CC)).
2: for dj E [dy, 6] do
3: Parallel training for (MM, Q.get(24), 2d,+1 1).
4: for All j do
5:
My. G, temporally partitioned M2n. G2 (Left), and M2j+1. G2f+1 (Right).
6: Left: C2 = Append Gj). Right: C2j+1 = Append Cj: Gj).
7: Left: Q.set(2j; C2j (M2j, C2j); Right: Q.set(2j + +1; (M2 Cayt)):
8: end for
9: end for
and coarse dynamic modeling and includes the motion bases M, and the dynamic Gaussian rep- resentation Gy. Note that r set for the root. We further build this partition tree using BFS traveling, which means that as the depth of the tree expands, the temporal intervals become nar- rower and more refined. For each newly added tree node ((M,; G2), we use similar training process as the root node for optimization to achieve the refinement of the temporal interval. Then, the optimized tree node for the video interval [T TTA] partitioned into left and right child nodes according to the temporal partition point, namely TTj") for the left child S25 (M22, G2j), and [T T for the right child ( G25+1), where T I(T)' + 1/2] the partition point which we obtain using the binary strategy. The partition process includes the division of the motion bases
M2j = (M E M, 11 <TT]
M₁
(4)
M2j+1 (M EMj |t |t21j]
and the division of the Gaussian primitives
C2j = [
G
(5)
In practice we limit the number of Gaussian primitives that a child node inherits from its ancestor balance efficiency and reset its opacity before training escape from local saddle points.
3.3 SPATIAL ANCESTRAL CHAINS
Although the video interval has been partitioned to achieve the temporary coarse-to-fine dynamic modeling optimization, the partition of the temporal interval lacks the spatial visual supplement, because the Gaussian primitives are constantly truncated during the inheritance process. Therefore, we further propose the Spatial Ancestral Chains (SAC) to introduce dynamic representations at different spatial scales.
Specifically, for each tree node (M,, Gý), we introduce the dynamic expression chain C, from the root node ancestor to this node, which can be represented as
C₁ c-= c) CC(M G
E
2a a 1
(6)
where a represents the number upstream levels related to j. and CLM,) G/) means that the common ancestors of different nodes are independent of each other but have the same optimized initialization. Note that the dynamic representation in C, is also optimized the process build-
ing TPT. Thus, we construct a spatial hicrarchical representation the video interval [T T with hierarchical spatial composition and modeling specialization. Furthermore, the final deformed Gaussian points P, at time 7 can be written as
P, (., G E GGlGmME)
(un fos(Me G e GG)] {(J
(7)
5

---

Published as a conference paper at ICLR 2026
The following rasterization process is similar to the common splatting process. By introducing SAC, we introduce motion representations from different spatial aspects to further assist the dynamic modeling of the current node representing the partial video interval.
3.4 TRAINING DETAILS
Parallel Optimization. An intuitive observation is that although nodes at the same depth have the same ancestor chain initialization, they are independent of each other during the optimization pro- cess. Therefore, we provide parallel optimization of the same depth and achieve O(5) optimization efficiency in terms of optimization times when there are enough parallel computing resources. If the optimization is not performed in parallel, 0(2++1) times of independent optimization are required.
-oottOptimizttion We adopt several pre-optimization processes for the root node to better con- struct the dynamic representation during the overall training process. Bundle Adjustment. Since it is often difficult to obtain sufficiently accurate camera poses in dynamic reconstruction, we addition- ally adopt rraseet-based bundle adjustment following (Lei et al. 2025) to better initialize the camera before dynamic optimization. Static Regions Warm-up. In order to make the model focus more on dynamic modeling optimization during dynamic foreground reconstruction, we pre-optimize the static background before dynamic reconstruction to obtain a stable static background as a warm-up for the static regions of the overall dynamic reconstruction.
Photometric Optimization with WorldTree. In Algo. we demonstrate the algorithmic core process of the proposed method. This algorithmic process primarily describes the optimization strategy and its structure, which the core content of this work. More specifically, when training each node, we use several regularization terms to achieve comprchensive optimization, including
regularizations commonly used in the prevailing methods (Lei et al., 2025 Park et al. 2025) such as photometric loss and depth loss. See more implementation details in the Appendix.
3.5 DISSCUSION
We provide a detailed discussion in comparison other prevailing methods. Temporal Partition. MoDec-GS (Kwak et al. 2025) attempts to achieve efficient motion representation through adaptive time intervals and global-loca decomposition of motion, but limited fixed partition initialization and two-stage optimization. Hierarchical Representation. MoSca (Lei et al. 2025) introduces an efficient motion scaffold graph for fusing the motions, but its optimization of global deformation in the overall interval and non- non-hicrarchical motion representation limit its modeling capabilities. HiMoR (Liang et al., 2025) proposes the hicrarchical motion representation with a tree structure for decomposing motions to coarse-to-fine levels, but neglects ficrarchical-couplin of optimization conflicts across different temporal intervals with varying deformation patterns caused by the local- wise interferences. Manual Point Prompts. Besides, the prevailing works (Park et al. 2025 Wang jet al., 2024b; Liang et al.) 2025) tend to introduce the manual point prompts prior to accurately identifying the dynamic regions for better reconstruction, but the manual points conflict with the massive amount of monocular videos, which is further discussed in Sec. 4 To this end, we propose the WorldTree for both achieving hierarchical temporal partitioning for monocular video intervals and spatial complementary hierarchical dynamic representations with modeling specialization.
4 EXPERIMENTS
4.1 EXPERIMENTAL DETAILS
Dataset. We utilize two datasets for evaluating the proposed method. We first re-construct the NVIDIA-LS YYoon et al. 2020) dataset, which is more challenging than the original NVIDIA (Yoon) ct al. 2020) dataset. Specifically, Yoon et al. uses a rig of 12 cameras for capturing different views at the same timestamp, so that they construct a monocular video while obtaining poses from different views. The original dataset only contains 12 frames. We extend the video interval to a long sequence of up to 160 frames. Besides, the previous data division uses the fixed pose corresponding to the first frame of the training view as the test view, which implicitly reduces the difficulty of dynamic reconstruction of monocular video during training. Therefore, we separate the test view from the

---

Published as a conference paper at ICLR 2026
Ground Truth
Ours
MoSca
HiMoR (w/o MPP)
4DGS (W/ CTP)
D3DGS (w/ CIP)
Ground Truth
Ours
SplineGS (w/ MPF)
HiMoR (w/ MPP)
4DGS (w/o CIP)
DIDGS (w/o CIP)
Figure 3: Qualitative comparisons with other methods NVIDIA-LS (Yoon al.
2020
Table 1: Quantitative comparisons with other methods on NVIDIA-LS (Yoon et al.
2020
Metric
Method
CIPMPP
SENRTSSIM]LPISSJAA
MSSIM MAPSSNSSSSSMMMMMMMM
mAVGE
D3DGS t
Yang al.
2023b
X
21.41
0.748
0.209
0.091
16.29
0.575
0.147
D3DGS f
Yang et al.
2023b
/
X
21.33
0.728
0.228
0.096
16.19
0.563
0.154
HiMoR $
Liang et
2025
X
19.24
0.444
0,465
0.160
15.17
0.509
0.215
HiMoR +
Liang et al.
2025)
X
V
19.30
0.442
0.440
0.157
15.28
0.510
0.209
SplineGS
Park al.
22025
21.41
0.687
0.227
0.097
15.56
0.541
0.162
SplineGS
Park ct al.
2025
X
V
22.20
0.725
0.168
0.081
17.07
0.617
0.127
4DGS*
Wu et al.
2023
21.48
0.646
0.197
0.094
16.87
0.604
0.137
4DGS t
Wu et al.
2023
V
X
23.34
0.790
0.128
0.065
17.95
0.651
0.107
MoSca
Le1 et al.
2025
X
23.72
0.792
0.109
0.060
17.89
0.664
0.101
WorldTree (Ours)
X
X
24.06
0.804
0.100
0.056
18.55
0.692
0.092
t: Our reproduction. *: Variants of methods. CIP: Colmap Initial Points. MPP: Manual Points
Prompts.
training view, use the first camera as the test view, and construct the training data structure of the remaining cameras in a similar way to the original dataset. Furthermore, we annotate dynamic masks using Segment-Anythnn Anything (SAM2) (Ravi et al.) 2024) for testing images to better evaluate the reconstruction quality of dynamic regions. See more details of the NVIDIA-LS (Yoon et al. 2020) in the Appendix. Note that SAM2 (Ravi et al. 2024) is only utilized to annotate testing images for evaluation, and not utilized in the optimization process. We also evaluate method on the widely used DyCheck (Gao et al. 2022a) dataset following the protocol of the previous work (Lei et al. 2025).
Implementation Details. MoSca (Lei et al. 2025). We use Metric3D--V2 (Hu et al., (024a) for monocular depth prediction, CoTrracker3 (Karaev et al. [223 2024) for tracking, and RAFT (Teed & Deng 2020 for obtaining optical flow. Note that the LiDAR sensor depth used for the DyCheck (Gao et al. 2022a) dataset following the previous works (Lei et al. 2025). The tree-depth of TPT is set to 2, which counts from See more implementation details in the Appendix.
The preprocessing of our method using 2D fundamental models follows
Baselines. For the NVIDIA-LS (Yoon al. 2020) dataset, compare method with sev- eral ofate-of-the-art methods, including D3DGS Yang et al., 20236), 4DGS (Wu et al.] 2003)) SplineGS (Park al. 20025). MoSca (Lei et al.) 2025). It is supposed to be noted that some of the above methods ( (Yang et al. 2023b; Wu et al. 2023) introduce StM (Schonberger & Frahm 2016) initialization point cloud from Colmap (Schonberger & Frahm 2016) results, while some methods (Park et al. 2025 Liang et al.) 2025) use manual point prompts as conditions to obtain foreground mask, where we use SAM2 (Ravi et al. 2024) for reproduction. For fair comparison, we also report the experimental results without these priors. For the DyCheck (Gao et al. 2022a)
dataset, the reported results are provided by the previous works (Lei et al, 2025 Liu et al. 2023 See details of reproduction in the Appendix.
Metrics. For NVIDIA-LS YYon et al. 2020). we report PSNR, SSIM, and LPIPS (Zhang et al. 2018) for evaluating the entire rendering, which includes both static and dynamic regions. We also
7

---

Published as a conference paper at ICLR 2026
Table 2: Quantitative comparisons with other
methods on DyCheck (Gao et al. 2022a
Method
mP53R
MSSIM
mLPIPS.
I-NeRF
Goo cl 2022a
16.96
0.577
0.379
NSTF
La er Ma
2024
15.46
0.551
0.396
Norfies
Park
Hyp.NeRI Park di dle
10cch
16.81
0.569
0.332
PGDVS
Zhap cl
2024
15.88
0.548
0.340
DyPoint
16.89
DpDy Wang et al
2024B
0.559
0.516
Dyn.Gau Eailen
20124
7.29
0.692
4D GS Wa al. 2024
13.64
0.428
DGMar (Seimms el di
2014
16.72
0.413
DyBluRT [Bo
17.37
CTNeRF (Vlan ai
2014
17.69
0.531
RaDyaRF
D.NPC (Kapnel
16.41
0.582
0.319
RoDyakF LLu &f
2023
17.09
0.534
Figure 4:
Qualitative comparisons with other methods on DyCheck (Gao et al. 2022a).
SoM (Wanr el ai 61744
17.32
0.598
0.296
MoSca (LA 2025
19.32
0.706
0.264
World (UBI))
19.75
0.728
0.240
report the masked PSNR (mPSNR) and masked SSIM (mSSIM) for evaluating the dynamic parts, which are more important for the dynamic reconstruction. Furthermore, we report the Average Error (AVGE), which is computed by the geometric mean of MSE 10 (O--SNR/10 vI- SSIM. LPIPS, and masked Average Error (mAVGE), which replaces PSNR and SSIM with masked metrics. For the DyCheck (Gao et al., 2022a) dataset, we report mPSNR, MSSIM, and mLPIPS using the dataset- provided co-visibility masks following the previous works (Wang et al., 2024b Lei al. 2025).
4.2
COMPARISONS WITH OTHER METHODS
We conduct detailed quantitative experiments the NVIDIA-LS Yoon et al. 2020 dataset as shown in Tab. Our method achieves the improvement of 21.40% on the mp- SNR compared to HiMoR (Liang et al. 2025), the improvement of 12.16% on the MSSIM com- pared to SplineGS (Park et al. 2025). and the improvement of 8.91% on the mAVGE compared to MoSca (Lei al., 2025). When replacing the mask manual prior using SAM2 (Ravi et al. 2024) with manual point prompts to the EPI mask, the improvement of our method goes up to 22.28% on the mPSNR and 27.91% on the MSSIM compared to HiMoR (Liang al. 2025) and SplineGS (Park et al. 2025). respectively. The qualitative results on the NVIDIA-LS Yoon et al., 2020) dataset are shown Fig. 3 Compared to the rendering results of SplineGS (Park et al. 2025) and HiMoR (Liang et al. 2025), ) the rendering results of our method show better overall rendering and reconstruction quality on the dynamic regions. Overall, our proposed method achieves both better reconstruction quality and more detailed visual content.
NVIDIA-LS.
DyCheck. The quantitative experimental results on the DyCheck (Gao et al. 2022a) dataset are reported in Tab. Similar the results on the NVIDIA-LS (Yoon et 2020) dataset, our method achieves state-ol-the-at results compared to prevailing methods. Following the evaluation protocol of the previous works (Liu et al. 2023 Lei al., 2025). we evaluate the reconstruction quality us- ing the masked metrics using the co co-visibility masks. The mLPIPS of our method is improved by 18.92% and 9.09% compared to SoM (Wang et al. (024b) and MoSca (Lei al.) 2025). ) respec- tively. The qualitative results on the DyCheck (Gao et al. 2022a) dataset are shown in Fig. We can notice that MoSca (Lei et al. 2025), as the state-of-the-arr method of 3DGS- based methods for dynamic reconstruction, still exhibits blurry rendering of details in motions of foreground objects, egg. fingers the top line of Fig. See more qualitative comparisons in the Appendix.
4.3 PERFORMANCE ANALYSIS
Why WorldTree achieves better reconstruction quality than prevailing methods? As shown in although the previous works (Lei et al.. 2025) Wang et al. 2024b) propose efficient motion rep- resentations, their training strategy in the overall video sequences, and the global deformation field construction lead to their inadequacy in reconstructing motion details. Meanwhile, the current hi- erarchical representation (Liang et al. 2025) also struggles with hierarchical deformation coupling, thus achieving unsatisfactory reconstruction results. In contrast, our method achieves high-quality reconstruction of dynamic scenes. due to the temporal coarse-to-fine optimization strategy intro- duced by the proposed TPT and the spatial hierarchical representation capability with chain-wise
8

---

Published as a conference paper at ICLR 2026
Table 3: Ablation study on the NVIDIA-LS (Yoon et al. 2020) dataset.
Metric
BA
SW
TPT
SAC
PSNRT SSIMT
LPIPS
AVGEL
mPSNRT
mSSIMT
mAVGE
X
X
X
21.87
0.710
0.139
0.079
16.82
0.620
0.121
X
X
V
X
22.39
0.777
0.121
0.069
17.41
0.652
0.109
X
X
V
22.64
0.778
0.113
0.066
17.85
0.668
0.102
X
X
X
22.08
0.712
0.130
0.076
17.32
0.642
0.113
X
/
22.58
0.764
0.115
0.068
18.02
0.673
0.101
X
/
22.73
0.766
0.108
0.065
18.35
0.685
0.096
X
23.59
0.787
0.115
0.061
17.73
0,655
0.104
23.94
0.801
0.105
0.058
18.36
0.683
0.095
24.06
0.804
0.1
0.056
18.55
0.692
0.092
SW
IIA, SWW
18A, SW
Grvead
1toght
(a) Quantitative comparisons of ablations on the NVIDIA- (b) Qualitative comparisons of ablations on the NVIDIA-LS Yoon al. 2020 dataset.
LS Yoon al. 2020 dataset.
Figure 5: Quantitative and qualitative ablations the NVIDIA-LS Yoon al. 2020 dataset.
specialization introduced the proposed SAC. Overall, our proposed method achieves both better reconstruction quality and more detailed visual content.
Is WorldTree effective? We conduct ablations the NVIDIA-LS (Yoon et al., 2020) dataset shown in Tab. 3 Note that BA represents Bundle Adjustment, SW represents Static Warm-up. TPT represents Temporal Partition Tree, and SAC represents Spatial Ancestral Chains, which are demonstrated in Sec.
Our method achieves the improvement of 18.71% on the LPIPS (the same below) when deactivating BA and SW, 16.92% when deactivating SW, and 13.04% when both activating BA and SW. This demonstrates the ability of WorldTree to optimize and promote the dynamic reconstruction in different situations, confirming its effectiveness.
How does SAC promote TPT? We also conduct ablations to explore the two hierarchical compo- nents WorldTree: TPT and SAC, as shown in Tab. 3 When adding TPT, the LPIPS and mAVGE are improved by 8.70% and 8.65%, respectively. When further adding SAC, the LPIPS and mAVGE are further improved by 4.76% and 3.16%, respectively. This confirms that SAC further supplements the spatial hierarchical representation capability with chain-wise specialization based TPT's tem- poral coarse-to- to-fine optimization. The detailed qualitative results of ablations are shown in Fig. [5b] The rendering quality corresponding to the progressive elimination of different parts of our final method demonstrates the effectiveness of each component. See more ablation results in the Appendix.
How does the height of TPT affect the reconstruction quality? The trend of reconstruction quality varying with TPT height shown in Fig. 5a We can notice that both TPT and SAC have an effective effect on improving reconstruction quality under different initializations. At the same time, the reconstruction quality is highly positively correlated with the height of the tree (i.e. the granularity of the time interval division) over several metrics. To balance quality and efficiency, we ultimately choose the maximum height of 2. More results and discussions on the tree height of TPT and the chain length of SAC are detailed in the Appendix.
Exploration of different external priors. To explore the sensitivity of our method external pri- ors, the experimental results under different external priors are shown in Tab. g While reconstruc- tion quality varies with the initialization using depth and tracking priors, our method consistently achieves better reconstruction quality compared to MoSca (Lei al. 2025), and achieves significant improvements, e.g.. + 10.27% of mAVGE compared to Base when using Metric3D- (Hu et al. 2024a) and BOOISTAPIR (Doersch et al. 2024), and +12.00% of mAVGE compared to Base when using UniDepth (Piccinelli et al.) 2024) and CoTracker3 (Karaev et al. 2023 2024). The experi- mental results confirm the robustness of method different initializations of external priors

---

Published as a conference paper at ICLR 2026
Table 4: Quantitative results of different exter-
Impat
nal priors on NVIDIA-LS Yoon et al. (020).
Metrie
Methods
mPSNR
mSSIMT mAVGE
UniDepth + CoTracker3
MoSca
18.30
0.674
0.097
Base
18.17
0.673
0.100
Ours
18.98
0.714
0.088
Metric3D-V2
BootsTAPIR
MoSca
16.94
0.609
0.142
Base
16.80
0.600
0.146
Ours
17.43
0.640
0.131
Metric3D-V2 + CoTracker3
MoSca
17.89
0.664
0.101
Figure 6: Qualitative results on the DAVIS PPntt-
Base
17.73
0.655
0.104
Ours
18.55
0.692
0.092
Tuset al. 2017) dataset.
Table 5: Evaluation of VRAM costs.
Analysis of computational costs. Further- more, we compare the VRAM usage of our
VRAM(GBB)
Peak VRAM
Avg. VRAM
MoSca
23,99
12.90
method and MoSca (Lei et al. 2025) dur-
Ours (Layer 0)
23.99
8.61
ing photometric optimization on the Balloon scene of the NVIDIA-LS (Yonn et al., (2020)
Ours (Layer 1)
23.99
11.99
Ours (Layer 2)
24.00
12.68
dataset. As shown in Table [ our method ex-
hibits consistently lower average memory consumption across different tree layers compared to MoSca (Lei et al. (025). The peak memory usage is similar for all methods, indicating a shared con- straint inherited from the underlying architecture rather than being introduced by Tree-Chains representation. Notably, our method maintains a significantly lower average VRAM footprint, demonstrating that Tree-Chains do constitute a scalability bottleneck. Additional analysis of computational cost for parallel training is provided in the Appendix.
Evaluation on the wild data. To assess generalization on real-world videos, we present qualitative results on DAVIS (Pont- Tuset et al. 2017) as shown in Fig. 6 We select viewpoints at timestamps with the ratio of [0, 0.25, 0.5, 0.75, I.0] and render novel views to construct a spatio-temporal matrix. This matrix visualizes scene dynamics and 3D spatial information from multiple viewpoints at syn- chronized times. The consistent, high-quality renderings across the matrix confirm that our method generalizes effectively to in-the-wild videos, producing high-fidelity reconstructions and validating its practical applicability. More qualitative results on the wild data are provided in the Appendix.
CONCLUSION AND LIMITATIONS
In this paper, we propose WorldTree, which includes the Temporal Partition Tree and the Spatial An- cestral Chains. These components implement the temporal progressive training strategy for the entire video interval and a complementary hierarchical representation at the spatial level. Further- more, to better evaluate the quality of dynamic reconstruction, we propose the NVIDIA-LS dataset with longer sequence length and annotated dynamic foreground masks for evaluation. Experimen- tal results on different datasets show that our method achieves the state-of-the-art reconstruction quality. The limitation of our work is that the proposed pipeline still relies on the capability of pre- trained foundation models. We can further improve the techniques for prior extraction. We leave as our future work.
ACKNOWLEDGMEETTS
This work is partially supported by grants from the Guizhou Provincial Major Scientific and Tech- nological Program (Qiankehe Zhongda [2025] No. 032), National Natural Science Foundation of China (No.62 22022., Beijing Nova Program (N0.20250484786,, and the Fundamental Research Funds for the Central Universities.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Benjamin Attal, Jia-Bin Huang, Christian Richardt, Michael Zollhoefer, Johannes Kopf, Matthew O"Toole, and Changil Kim. Hyperreel: High-fidelity 6-dof video with ray-conditioned sam- pling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
Shai Avidan and Amnon Shashua. Novel view synthesis in tensor space. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1034-1040. IEEE, 1997.
Jonathan T Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Partin-Bruualla, and Pratul P Srinivasan. Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields. In International Conference on Computer Vision (ICCV), pp.
2021.
Luah--uan Viet Bui, Jongmin Park, Jihyong Oh, and Munchurl Kim. Dyblurf: Dynamic deblurring neural radiance fields for blurry monocular video. arXiv preprint arXiv:2312. 2023.
Daniel Butler, Jonas Wulff, Garrett B Stanley, and Michael J Black. A naturalistic open source movie for optical flow evaluation. In Computer Vision-ECCH 2012: 12th European Conference on Computer Vision, Florence, Italy, October 7-13, 2012, Proceedings, Part VI 12, pp. 611-625. Springer, 2012.
Ang Cao and Justin Johnson. Hexplane: A fast representation for dynamic scenes. arXiv preprint arXiv:2301.09632. 2023.
Anpei Chen, Zexiang Xu, Andreas Geiger, Jingyi Yu, and Hao Su. Tensorf: Tensorial radiance
fields. In Computer Vision-ECCV 2022: 17th European Conference, Tel Aviy, Israel, October 23-27, 2022, Proceedings, Part XXXII, pp. 333-350. Springer, 2022.
Guikun Chen and Wenguan Wang. survey on 3d gaussian splatting. arXiv preprint arXiv:2401.03800 2024.
Devikalyan Das, Christopher Wewer, Raza Yunus, Eddy Ilg, and Jan Erie Lenssen Neural paramet- ric gaussians for monocular non-rigid object reconstruction. arXiv preprint arXiv:2312.01 196, 2023.
Carl Doersch, Pauline Luc, Yang, Dilara Gokay, Skanda Koppula, Ankush Gupta, Joseph Hey- ward, Ignacio Rocco, Ross Goroshin, Joao Carreira, and Andrew Zisserman. Bootstap: Boot- strapped training for tracking-any baccking-any-poiee any-point. Asian Conference on Computer Vision, 2024.
Yuanxing Duan, Fangyin Wei, Qiyu Dai, Yuhang He, Wenzheng Chen, and Baoquan Chen. 4d- rotor gaussian splatting: towards efficient novel view synthesis for dynamic scenes. In ACM SIGGRAPH 2024 Conference Papers, pp. 1-11, 2024.
Bardienus P Duisterhof, Zhao Mandi, Yunchao Yao, Jia-Wei Liu, Mike Zheng Shou, Shuran Song. and Jeffrey Ichnowski. Md-splatting: Learning metric deformation from 4d gaussians in highly deformable scenes. arXiv preprint arXiv:2312.00553,, 2023.
Ben Fei, Jingyi Xu, Rui Zhang, Qingyuan Zhou, Weidong Yang, and Ying He. 3d gaussian as a new vision era: A survey. arXiv preprint arXiv:2402.07188/ 2024.
Sara Fridovich Keil, Giacomo Meanti, Frederik Warburg, Benjamin Recht, and Angjoo Kanazawa. K-planes: Explicit radiance fields in space, time, and appearance. arXiv preprint arXiv:2301. 10241, 2023.
Chen Gao, Ayush Saraf, Johannes Kopf, and Jia-Bin Huang. Dynamic view synthesis from dynamic monocular video. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5712-5721, 2021.
Hang Gao, Ruilong Li. Shubham Tulsiani, Bryan Russell, and Angjoo Kanazawa. Monocular dy- namic view synthesis: A reality check. Advances in Neural Information Processing Systems, 35: 13768-33780. 2022a.
11

---

Published as a conference paper at ICLR 2026
Kyle Gao, Yina Gao, Hongjie He, Dening Lu, Linlin Xu, and Jonathan Li. Nerf: Neural radiance field in 3d vision, a comprehensive review. arXiv preprint arXtv:2210.00379, 2022b.
Xiang Guo, Guanying Chen, Yuchao Dai, Xiaoqing Ye, Jiadai Sun, Xiao Tan, and Errui Ding. Neural deformable voxel grid for fast optimization of dynamic view synthesis. In Proceedings of the Asian Conference on Computer Vision, pp. 3757-3775, 2022.
Mu Hu, Wei Yin, Chi Zhang, Zhipeng Cai, Xiaoxiao Long, Hao Chen, Kaixuan Wang, Gang Yu, Chunhua Shen, and Shaojie Shen. Metric3d v2: A versatile monocular geometric foundation model for zero-shot metric depth and surface normal estimation. 2024a.
Wenbo Hu, Xiangjun Gao, Xiaoyu Li, Sijie Zhao, Xiaodong Cun, Yong Zhang, Long Quan, and Ying Shan. Depthcrafter: Generating consistent long depth sequences for open-world world videos. 2024b.
arXiv preprint
Yi-Hua Huang, Yang-Tian Tian Sun, Ziyi Yang, Xiaoyang Lyu, Yan-Pei Cao, and Xiaojuan Qi.
Sc-gs: Sparse-controlled gaussian splatting for editable dynamic scenes. arXiv preprint arXiv:2312.19937, 2023.
Moritz Kappel, Florian Hahlbohm, Timon Scholz, Susana Castillo, Christian Theobalt, Martin Eise- mann, Vladislav Golyanik, and Marcus Magnor. D-upe: Dynamic neural point clouds for non- rigid view synthesis from monocular video. arXiv preprint arXiv:2446.11000 2024.
Nikita Karaev, Ignacio Rocco, Benjamin Graham, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. Cotracker: It is better to track together. arXiv preprint arXiv:2307.07635, 2023.
Nikita Karaev, Iurii Makarov, Jianyuan Wang, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. Cotracker3: Simpler and better point tracking by pseudo-labelling real videos. 2024.
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splat- ting for real-time radiance field rendering. ACM Transactions on Graphics (TOG), 42(4):1-14, 2023.
Sangwoon Kwak, Joonsoo Kim, Jun Young Jeong, Won-Sik Cheong, Jihyong Oh, and Munchurl Kim. Modec-gs: Global-to-local motion decomposition and temporal interval adjustment for compact dynamic 3d gaussian splatting. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 11338-11348, 2025.
Jiahui Lei, Yijia Weng, Adam W Harley, Leonidas Guibas, and Kostas Daniilidis. Mosca: Dynamic gaussian fusion from casual videos via 4d motion scaffolds. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 6165-6177, 2025.
Jiahe Li, Jiawei Zhang, Xiao Bai, Jin Zheng, Xin Ning, Jun Zhou, and Lin Gu. Dngaussian: Optimiz- ing sparse-view 3d gaussian radiance fields with global-local depth normalization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 20775-20785, 2024.
Tianye Li, Mira Slavcheva, Michael Zollhoefer, Simon Green, Christoph Lassner, Changil Kim, Tanner Schmidt, Steven Lovegrove, Michael Goesele, Richard Newcombe, et al. Neural 3d video synthesis from multi-view video. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
Zhan Li, Zhang Chen, Zhong Li, and Yi Xu. Spacetime gaussian feature splatting for real-time dynamic view synthesis. arXiv preprint arXiv:2312.168/ 2023a.
Zhengqi Li, Simon Niklaus, Noah Snavely, and Oliver Wang. Neural scene flow fields for space- time view synthesis of dynamic scenes. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 6498-6508, 2021.
Zhengqi Li, Qianqian Wang, Forrester Cole, Richard Tucker, and Noah Snavely. Dynibar: Neural dynamic imsge-based rendering, 2023b.
Yiming Liang, Tianhan Xu, and Yuta Kikuchi. Himor: Monocular deformable gaussian reconstruc- tion with hierarchical motion representation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 886-895, 2025.
12

---

Published as a conference paper at ICLR 2026
Yiqing Liang, Numair Khan, Zhengqin Li, Thu Nguyen-Phane Phuoc, Douglas Lanman, James Tompkin, and Lei Xiao. Gaufre: Gaussian deformation fields for real-time dynamic novel view synthesis. arXiv preprint arXfv:2312.11458, 2023.
Youtian Lin, Zuozhuo Dai, Siyu Zhu, and Yao Yao. Gaussian-flood flow: 4d reconstruction with dynamic 3d gaussian particle. arXiv preprint arXtv:23/2.03431, 2023.
Wei Liu, Yan-Pei Cao, Weijia Mao, Wenqiao Zhang, David Junhao Zhang, Jussi Keppo, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Devif: Fast deformable voxel radiance fields for dynamic scenes. arXiv preprint FxXi:22205.15253 2022.
Qingming Liu, Yuan Liu, Jiepeng Wang. Xianqiang Lv, Peng Wang, Wenping Wang, and Junhui Hou. MoDGS: Dynamic Gaussian Splatting from Gausally-Capture Monocular Videos. arXiv 2024.
preprint ar arXiv:2406.0044
Yu-Lun Liu, Chen Gao, Andreas Meuleman, Hung- Yu Tseng, Ayush Saraf, Changil Kim, Yung- Yu Chuang, Johannes Kopf, and Jia-Bin Huang. Robust dynamic radiance fields. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pp. 13-23, 2023.
Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan. Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis. arXiv preprint arXiv:2308.0971 0001130880971,,... 2023.
Xingyu Miao, Yang Bai, Haoran Duan, Yawen Huang, Fan Wan, Yang Long, and Yefeng Zheng- Cinerf: Cross-time transformer for dynamic neural radiance field from monocular video. arXiv preprint ar 0iii::241100066666 2024.
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes neural radiance fields for view synthesis. Communications of the ACM, 65(1)999-106, 2021.
Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics prim- itives with multiresolution hash encoding. arXiv preprint arXiv:2201.05982 2022.
Jongmin Park, Minh-Quan Viet Bui, Juan Luis Gonzalez Bello, Jaeho Moon, Jihyong Oh, and Munchurl Kim. Splinegs: Robust motion-adaptive spline for real-time dynamic 3d gaussians from monocular video. In Proceedings of the Computer Vision and Pattern Recognition Confer- ence, pp. 26866-26875, 2025.
Keunhong Park, Utkarsh Sinha, Jonathan T Barron, Sofien Bouaziz, Dan B Goldman, Steven M Scitz, and Ricardo BautinnBrualla. Nerfies: Deformable neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3865-58744 2021a.
Keunhong Park, Utkarsh Sinha, Peter Hedman, Jonathan T Barron, Sofien Bouaziz, Dan B Goldman, Ricardo Martin-Brualle and Steven M Seitz. Hypermerf: A higher-dimennsional representation for topologically varying neural radiance fields. arXiv preprint 10l::2206.13222 2021b.
Luigi Piccinelli, Hss Yang, Christos Sakaridis, Mattia Segu, Siyuan Li, Luc Van Gool, and Fisher Yu. Unidepth: Universal monocular metric depth estimation. arXiv preprint
arXiv:2403.1891 18913. 2024.
Jordi Pont-Tuset, Federico Perazzi, Sergi Caelles, Pablo Arbeláez, Alex Dorkine-Hornung and Lue Van Gool. The 2017 davis challenge video object segmentation. arXiv preprint
17Xiv:1704.00675. 2017.
AKM Rabby and Chengcui Zhang. Beyondpixels: A comprehensive review of the evolution of neural radiance fields. arXiv preprint 0rXiv:2336.03300, 2023.
Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson. Eric Mintun, Junting Pan, Kalyan Va- sudev Alwala, Nicolas Carion, Chao- Yuan Wu, Ross Girshick, Piotr Dollár, and Christoph Fe- ichtenhofer. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.000 2024. URL //arxi//xvi org abs 2408 00714
13

---

Published as a conference paper at ICLR 2026
Johannes L Schonberger and Jan-Michael Frahm. Structurefroommmotee revisited. In Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4114-4113, 2016.
Jenny Seidenschwarz, Qunjie Zhou, Bardienus Duisterhof, Deva Ramanan, and Laura Leal-Taixé. Dynomo: Online point tracking by dynamic online monocular gaussian reconstruction. arXiv preprint arXiv:2409.022 104, 2024.
Ruizhi Shao, Zerong Zheng, Hanzhang Tu, Boning Liu, Hongwen Zhang, and Yehin Liu. Tensor4d: Efficient neural 4d decomposition for high-fidelity dynamic reconstruction and rendering. arXiv preprint 1.Xiv:2211.116111 2022.
Zhijing Shao, Zhaolong Wang, Zhuang Li, Duotun Wang, Xiangru Lin, Yu Zhang, Mingming Fan, and Zeyu Wang- Splattingavatar: Realistic real-time human avatars with mesh-embedded gaussian splatting, arXiv preprint arXiv:2403.05087 2024.
Liangchen Song, Anpei Chen, Zhong Li, Zhang Chen, Lele Chen, Junsong Yuan, Yi Xu, and An- dreas Geiger. Nerfplayer: A streamable dynamic scene representation with decomposed neural radiance fields. IEEE Transactions on Visualization and Computer Graphics, 2023.
Colton Stearns, Adam W. Harley, Mikaela Uy, Florian Dubost, Federico Tombari, Gordon Wetzstein, and Leonidas Guibas. Dynamic gaussian marbles for novel view synthesis of casual monocular videos. In ArXiv, 2024.
Timo Stich, Christian Linz, Georgia Albuquerque, and Mareus Magnor. View and time interpolation in image space. Computer Graphics Forum, 2008.
Jürgen Sturm, Nikolas Engelhard, Felix Endres, Wolfram Burgard, and Daniel Cremers. bench- mark for the evaluation of rgb-d slam systems. In 2012 IEEE/RSJ international conference on intelligent robots and systems, pp. 573-580. IEEE, 2012.
Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part II 16, pp. 402-419. Springer, 2020.
Zachary Teed and Jia Deng. Droid-slam:
Deep visual slam for monocular, stereo, and rgb-d cameras. Advances in neural information processing systems, 14:16558-1656 2021.
Edgar Tretschk, Ayush Tewari, Vladislav Golyanik, Michael Zollhöfer, Christoph Lassner, and Christian Theobalt. Non-rigid neural radiance fields: Reconstruction and novel view synthesis of a dynamic scene from monocular video. In Proceedings of the IEEE/CVF International Con- ference Computer Vision, pp. 12970, 2021.
Chaoyang Wang, Peiye Zhuang, Aliaksandr Siarohin, Junli Cao, Guocheng Qian, Hsin- Hsin-Ying Lee, and Sergey Tulyakov. Diffusion priors for dynamic view synthesis from monocular videos. arXiv preprint 1=XXx:24410005880 2024a.
Qianqian Wang, Vickie Ye, Hang Gao, Jake Austin, Zhengqi Li, and Angjoo Kanazawa. Shape of motion: 4d reconstruction from a single video. 2024b.
Diankun Wu, Fangfu Liu, Yi-Hsin Hung Yue Qian, Xiaohang Zhan, and Yueqi Duan. 4d-fly: Fast 4d reconstruction from single monocular video. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 16663-16673,, 2025.
Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 4d gaussian splatting for real-time dynamic scene rendering. arXiv preprint arXiv:2310.08528, 2023.
Tianhao Wu, Fangcheng Zhong, Andrea Tagliasacchi, Forrester Cole, and Cengiz Oztireli. D2 nerf: Self-supervised decoupling of dynamic and static objects from a monocular video. arXiv preprint arXiv:2205.15838 2022.
Yuxi Xiao, Qianqian Wang, Shangzhan Zhang, Nan Xue, Sida Peng, Yujun Shen, and Xiaowei Zhou. Spatialtracker: Tracking any 2d pixels in 3d space. In Proceedings of the IEEE/CVE Conference Computer Vision and Pattern Recognition (CVPR), 2024.
14

---

Published as a conference paper at ICLR 2026
Gengshan Yang, Deqing Sun, Varun Jampani, Daniel Vlasic, Forrester Cole, Huiwen Chang, Deva Ramanan, William Freeman, and Ce Liu. Lasr: Learning articulated shape reconstruction from monocular video. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 4925-4455999 4935, 2021.
Zeyu Yang, Hongye Yang, Zijie Pan, Xiatian Zhu, and Li Zhang. Real-time photorealistic dynamic scene representation and rendering with 4d gaussian splatting. arXiv preprint arXiv:2310.10642 2023a.
Ziyi Yang. Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. De- formable 3d gaussians for high-fidelity monocular dynamic scene reconstruction. arXiv preprint arXiv:2309. 13101, 2023b.
Jae Shin Yoon, Kihwan Kim, Orazio Gallo, Hyun Soo Park, and Jan Kautz. Novel view synthesis of dynamic scenes with globally coherent depths from monocular camera. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5336-5345, 2020.
Meng You and Junhui Hou. Decoupling dynamic monocular videos for dynamic view synthesis. arXiv preprint arXiv:2304.017/11 2023.
Zehao Yu, Anpei Chen, Binbin Huang, Torsten Sattler, and Andreas Geiger. Mip-splatting: Alias- free 3d gaussian splatting. In Proceedings of the IEEE/CVF conference on computer vision and pattemn recognition, pp. 19447-19456, 2024.
Richard Zhang, Phillip Isola, Alexei Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as perceptual metric. In Conference Computer Vision and Pattern Recognition (CVPR), pp. 586-595, 2018
Xiaoming Zhao, Alex Colburn, Fangchang Ma. Miguel Angel Bautista, Joshua M. Susskind, and Alexander G. Schwing. Bxeudo-generalized dynamic view synthesis from video, 2024.
Kaichen Zhou, Jia-Xing Zhong, Sangyun Shin, Kai Lu, Yiyuan Yang, Andrew Markham, and Niki Trigoni. Dynpoint: Dynamic neural point for view synthesis. Advances in Neural Information Processing Systems, 36, 2024.
C. Lawrence Zitnick, Sing Bing Kang, Matthew Uyttendacle, Simon Winder, and Richard Szeliski. High-quality video view interpolation using a layered representation. ACM Transactions on Graphics (TOG), 2004.
APPENDIX
A USAGE OF LLMS
In this work, we use Large Language Models (LLMs) to aid or polish writing.
B DATASET DETAILS
Prior to introducing our proposed NVIDIA-LS discuss the limitations of the original NVIDIA Yoon construction efforts. As illustrated in Fig. 7 (left), the original dataset exhibits several critical shortcomings: (1) insufficient video sequence length, (2) implicit inclusion of the test view within the training data. Besides, (3) the original evaluation protocol evaluates the full image rather than the dynamic regions. The above demonstrated limitations significantly bias when benchmarking dynamic reconstruction methods, ultimately compromising the validity of comparative evaluations.
Yoon et al. 2020 dataset, it is necessary to first 2020) dataset that motivated our re-
Consequently, we propose the NVIDIA-LS (Yoon et al.) 2020) dataset based on the original NVIDIA (Yoon et al. 2020) video sequences, shown in Fig. (right). Our pipeline begins by processing the original video sequences to obtain initial coarse camera poses from the first suc- cessful SfM (Schonberger & Frahm 2016) timestamp, and applies consistent undistortion for all
15

---

Published as a conference paper at ICLR 2026
other frames. To address the limitations of the original dataset, we implement three key improve- ments: (1) extending video sequences to a maximum 160 frames (with uniform sampling applied to longer sequences), thereby increasing optimization complexity; (2) setting the first camera the test view and all subsequent views as training data to prevent the implicit test-view inclusion during training; and (3) using SAM2 (Ravi al. 2024) annotate dynamic region masks for Ground Truth images of the test view, enabling more precise evaluation of dynamic reconstruction quality.
C IMPLEMENTATION DETAILS
C.I TRAINING LOSSES
Following MoSca (Lei et al. 2025) and SoM (Wang et al. 2024b), the final training loss can be represented as
Ligh + Adepth + Arnck + Aamp Carap + Amcehace Avelhrvel:
(8)
where Ligh is the photomentric regularization as the main loss, Edepth is the depth loss for regularizing the geometric structures, Lunck is the tracking loss for supervising the track map with predicted 2D tracklets, arap is the physically-nnspired AssSiiid-As-Possible (ARAP) loss for encouraging the preservation of the local distances and coordinates, Lace and Lvel are the acceleration and velocity losses for supervising the temporal smoothness of the reconstructed deformation field.
C.2 TRAINING DETAILS AND
The max TPT height is set to 2 (counting from 0) for all datasets, and we only optimize the camera poses at the height of 0, i.e. the root, of TPT for all datasets. For the optimization of the static regions, we optimize the static regions for all layers of TPT in the NVIDIA-LS (Yoon et al. 2020) dataset, and only optimize the static regions the root of TPT in the DyCheck (Gao al. 2022a)
dataset. The weights for regularizations in Eq. 8 are mostly inherited from MoSca (Lei et al. 2025).
We deactivate the tracking loss of the child-nodes for more stable training on the DyCheck dataset. The optimization steps for each layer are set to 4000, 2000, 2000 and 4000, 1000, 1000, and the start point of the root for optimizing poses is set 1500 and 1000 in the NVIDIA-LS Yoon et al. 2020) dataset and DyCheck (Gao et al. 2022a) dataset, respectively. The inherited Gaussian points number from the parent node is set to 5000, 2000 for the layers except the root. The optimization step for the stage of bundle adjustment is set to 2000 for all datasets. The optimization step for the stage of static warm-up is set to 6000 for all datasets. Most coper-parameters are inherited from MoSca, and partial hyper-pane spper-parameterss are chosen heuristically based on the length of temporal intervals. The setting the randomness seed for the photometric optimization is following MoSca. We select MoSca the codebase of our method. We implement our method using PyTorrhh with CUDA 11.8. All experiments are conducted using RTX 3090.
C.3 3 EVALUATION DETAILS
The quantitative and qualitative results of the NVIDIA-LS ( Yoon et al. 2020) dataset are obtained by our reproduction. The quantitative results of the DyCheck (Gao al., 2022a) dataset are from the reported results of MoSca (Lei et al. 2025) and RoDynRF (Liu et al. 2023). The qualitative
results of MoSca (Lei al., 2025) and RoDynRF (Liu al. (023) the DyCheck (Gao et al., 2022a) dataset are from the released rendering results of them. We report the results of an individual experiment.
D REPRODUCTION DETAILS
D3DGS and 4DGS. We inherit the configuration on the HyperNeRI (Park et al. b)) dataset of D3DGS Yang et al. 2023b) and 4DGS (Wu et al., 2023) for reproducing on the NVIDIA-LS Yoon et al. 2020) dataset. Furthermore, we provide two settings of reproduction, namely using Colmap initial points or using random initial points. For Colmap initial points, we utilize the 3D points from Colmap results of the Colmap processing on the NVIDIA-IS YYon al. 2020) dataset. For random
16

---

Published as conference paper at ICLR 2026
Views
Views
Com-1
Cam-1
Cam-2
Cam-2
Cam-3
Cama
Cam-4
Cam-
Com-
Cam-5
Com-6
Cam-
Com-9
Cam-9
Cam-10
Cam-10
Cam-11
Cam-11
Cam-12
Cam-12
Time
Time
Original NVIDIA Protocol
Our NVIDIA-LS Protocol
Figure 7: The protocol of the NVIDIA-LS dataset. The green points represent training data, and the red points represent testing data.
Ground Truth
Annotated Mask
Figure The example of the annotated mask for evaluation in the NVIDIA-LS dataset.
initial points, we generate random points for initialization following the strategy of DNGaussian LLi et al. 2024).
SplineGS. Since there are differences between datasets, some changes have to be made for repro- duction. Specifically, the original NVIDIA (Yoon et 2020) dataset, the test view is included in the training views as the first camera. So SplineGS (Park et al. (2025) sets the first view of optimized camera poses as the test view, served for further testing rendering. However,
the NVIDIA- LS (Yoon et al. (2020) dataset, the test view is not included in the training views, we supplement the test-time optimization of poses (initialized from the pose the first training view) during test- ing. Besides, since the length of timestamps for the NVIDIA-LS. Yoon et al. (2200) dataset has been extended compared to the original NVIDIA Yoon et al. 2020 dataset, the contimuity of the pro- jection mask cannot be maintained for long sequences, its robustness for successful running. Meanwhile, SplineGS (Park et al. tracking points for tracking the 12-frame original NVIDIA (Yoon al.
make necessary changes improve
2025) takes quite dense
2020) dataset. However,
is not practical for long videos. So we reduce the number of tracking points. Specifically, we start from the number of tracking points provided by Mosca (Lei al. 2025). and employ similar strat- egy of MoSca (Lei et al.) 2025) to control the number of tracking points in long video sequences. Furthermore, since Spline (Park et al.) 2025) requires manual point prompts pre-annotation, we provide two masks when reproducing, one is the EPI mask computed from the epipolar error maps following MoSca (Lei et al., 2025) with the threshold of 0.0001. The other is the SAM mask with our point prompts. Last, we cancel the strategy of selecting the best PSNR result during training and only use the final training result for testing.
dynamic region
HiMoR. Following HiMoR, we pre-process the NVIDIA-LS (Yoon et al. 2020) dataset through the SoM (Wang al. 2024b) pre-processing pipeline. We make necessary changes fit the DROID-SLAM (Teed & Deng 2021) pre-processed data format and use default configurations for the reproduction. However, since the DROID-SLAM (Teed & Deng. 2021) does not include the pose of the test view, we supplement the test-time optimization of poses (initialized from the pose of the first training view) during testing. Besides, since the unrobustness long-sequence uncontinuity tracks, make several changes to improve its robustness of visibility of long-term tracking for successful running. Since HiMoR (Liang et al. 2025) requires manual point prompts for dynamic region pre-annotation, the EPI mask computed from the epipolar enror maps following MoSca (Lei et al. (025) with the threshold of 0.0001 for fair comparison. The other is the SAM mask with our annotated point
HiMoR on processing
reproduce it with two types of masks, one is
17

---

Published as a conference paper at ICLR 2026
Ours
Wi SAC
wio SAC, TPT
w/o
w/o SW, SAC
w/o w/o SW, SAC, TPT
SAC,
Figure 9: More qualitative ablations on the NVIDIA-LS
Yoon al. 2020 dataset.
MoSca
Figure 10: More qualitative comparisons on the DyCheck (Gao et al. 2022a) dataset.
prompts. It is supposed to noted that HiMoR (Liang et al. 2025) shares the same SAM mask as SplineGS (Park al. 2025
MoSca. Since our codebase is MoSca, we only make necessary changes to MoSca to adapt to the NVIDIA-LS (Yoon et al.) (020) dataset format. The maximum number of motion nodes for node densification is set to 32, which is identical to the experimental setting of MoSca on the Sintel (Butler et al. 2012) and Tum-Dynamics (Sturm et al. 2012) datasets.
E MORE RESULTS OF ABLATIONS
More detailed qualitative comparisons of ablations are shown in Fig. Different from the qual- itative ablation studies presented in Fig. 6 of the main manuscript, here we provide the rendering quality of different parts of the staircase elimination under various conditions in final method. This systematic evaluation further validates the contribution of each individual component to the overall performance.
18

---

Published as a conference paper at ICLR 2026
SplineGS MPP)
(N) MIP))
D3DGS
Ground Truth
Ours
SplineGS (wio MPP)
HiMoR (w/o MPP)
4DGS (w/o CIP)
D3DGS CIP)
Figure 11: More qualitative comparisons the NVIDIA-LS
Yoon al.
2020 dataset.
F MORE QUALITATIVE COMPARISONS
More detailed qualitative experiments on the DyCheck (Gao et al. 2022a) dataset and NVIDIA- LSS Yoon et al. 2020) dataset shown in Fig. 10 and Fig. respectively.
G FAILURE CASE ANALYSIS
There remains the challenge in the reconstruction of small, fast- moviin dynamic regions for our method, as well as other prevailing methods, as shown Fig. 12, This challenge arises from weak- ened optimization constraints, consequence of the inherently sparse and incomplete supervisory signals available monocular video data. In the absence of external generative priors to supplement this limited signal, the accurate recovery of small and fast-moving motion details in these areas be- comes particularly difficult. However, this is a common challenge for reconstruction based methods notice that other prevailing methods exhibit more
under monocular constraints. In Fig. 12 pronounced failures in such scenarios.
H DISCUSSION ON THE STATIC CONTENT WITHIN SCENARIOS
Our method leverages common assumption that dynamic scenes often contain a static structural component. The SW and BA stages utilize this to robustly initialize 3D structure and camera poses. This assumption violated in scenarios with large-scale, pervasive non-rigid deformation (e.g., entire scenes deforming). which impair initialization. However, such cases monocular videos, and the problem becomes severely under-constrained without static references, which is challenge common most methods in this situation.
in typical
19

---

Published as a conference paper at ICLR 2026
Table 6: Computational cost analysis for the parallel training.
Parallel Training Sequential Training Acceleration Ratio Ideal Acceleration Ratio
(Hours)
(Hours)
2.21
4.35
49.2 49.26%
57.14%
Table Exploration of different TPT depths on the NVIDIA-LS (Yoon et al. 2020) dataset.
Table 8: Exploration different ancestral chain lengths on the NVIDIA-LS (Yoon et al. 2020) dataset.
Depth
mPSNRR
MSSIM1
LPIPS
17.73
0.655
0.115
Length
mPSNR1
MSSIM1
LPIPS..
18.31
0.681
0.107
4
18.69
0.698
0.095
18.55
0.692
0.100
3
17.44
0.627
0.117
3
18.62
0.695
0,097
2
16.55
0.581
0.128
18.69
0.698
0.095
1
15.54
0.521
0.143
I EVALUATION OF THE PARALLEL TRAINING
We further present the corresponding computational costs analysis the parallel training within the photometric optimization process evaluated on the scene of Balloonl, shown in Tab. 6 In the main manuscript, we mentioned the optimization times (()) for parallel training, and (2**1)) times for sequential training. Note that according to the calculation way of the Big O operator, (()) = O 1), and 012++1)= 0(26+1 - 1). So, for the depth 5 of 2, the number of nodes is 7, and the number of layers is 3, where
and 7 2 1. Thus, the ideal acceleration ratio is (7 - 3)/7 = 57.14%. In practice, we achieve the acceleration ratio of 49.26% because of factors such different parameters for each layer and system I/O time, etc. However, the exploration of computational cost analysis is sufficient to demonstrate the effectiveness of parallel training.
FURTHER EXPLORATION ON THE TREE DEPTH AND CHAIN LENGTH
We provide further exploration on the effect of TPT depths and the ancestor chain lengths of SAC, as shown in Tab. and Tab. The ablation studies on the TPT depth in Tab. 7 reveal a consistent yet diminishing improvement reconstruction quality with increased depth. This trend indicates a convergence in the representational capacity of the model. Thus, select the depth of 2 the trade- off between performance and computational efficiency. Furthermore, based on the reconstruction results at the depth of 4, we evaluate the results by changing the chain length during rendering, as shown Tab. The substantial performance degradation observed with reduced chains underscores the importance of hierarchical spatial information provided by SAC, validating the effectiveness of SAC the achieved optimized reconstruction.
K FURTHER EXPLORATION ON THE STRATEGY OF TEMPORAL SPLITTING
We additionally explore the strategy of temporal splitting for evaluating the robustness of our method, as shown in Tab. 9 Specifically, we present a systematic comparison of three different tem- poral splitting strategies, i.e., gradient-basse based splitting, flow-based splitting, and our binary splitting strategy. Gradient- based splitting partitions the entire sequence by balancing the recorded gradients across intervals, whereas flow-based splitting achieves this partitioning by balancing the magnitude of optical flow within dynamic regions. We can notice that different strategies achieve similar re- sults, demonstrating the adaptability our method to different temporal splitting strategies. The experimental results show that our method does not depend or flow-based based splitting strategies, their
specific splitting strategy but can
adapt to different strategies. When adopting gradient- performance is only slightly different from binary splitting strategy, confirming the robustness of our method, which stems from the inherent advantages of the hierarchical dynamic representation and optimization mechanism of our method.
20

---

Published as a conference paper at ICLR 2026
Ground Truth
Ours
SplineGS (w/ MPP)
HiMoR (w/ MPP)
4DGS (w/ CIP)
D3DGS (w/ CIP)
Figure 12: Failure case in the reconstruction of small dynamic regions with fast motion.
Table 9: Exploration on the strategy of temporal splitting.
Metric AVGE
Settings
PSNRT
SSIM
LPIPS
mPSNRT
MSSIM
mAVGEf
Gradient-based Split
24.04
0.803
0.100
0.056
18.54
0.691
0.092
Flow-based Split
23.98
0.803
0.101
0.056
18.46
0.689
0.093
Binary Split (Ours)
24.06
0.804
0.100
0.056
18.55
0.692
0.092
MORE QUALITATIVE RESULTS ON THE WILD DATA
To evaluate the generalization capability of our approach to real-world, in-the-wild video sequences, present qualitative results on the DAVIS dataset (Pont-Tused Tuset et al. (017). Note that we utilize DepthCrafter (Hu al. 2024b) as depth prior and SpaTracker (Xiao et al. 2024) as tracking prior following the previous work (Lei et al. 2025). As shown in Fig. 13 tamps corresponding to ratios of (0, 0.25, 0.5, 0.75, 1.0] of the video sequence. From these selected viewpoints, render novel views construct spatio-tempord matrix. This matrix provides comprehensive visualization of the dynamics and spatial information of the scene, capturing its ap- pearance from diverse internal viewpoints synchronized timestamps. The results demonstrate consistent, high-fidelity renderings across all cells of this matrix. This spatial and temporal consis- tency indicates that our method successfully learns robust and coherent scene representation under the wild data, which confirms the effectiveness of our method in producing accurate reconstruc- tions from real-world video data, thereby validating its practical applicability for dynamic scene modeling.
select viewpoints times-
21

---

Published as a conference paper at ICLR 2026
NEcwo 58
FiCWOO.50 50
Hew 30
Reumeratted 4D Domam Sezne
Input Video
Spatio-Temports Matrix
Input Video
Spatio-Temport Matrix
Figure 13: More qualitative results on the wild data.
22


Reviews:
[{"review_id": "FUJisrqwKd", "reviewer": "Reviewer_ecpZ", "summary": "This paper proposes a approach for monocular dynamic reconstruction. The core contributions consists of two components:\n(1). Temporal Partition Tree (TPT): a hierarchical binary partitioning of the video time sequence. It can be used to refine the deformation modeling in temporal segments.\n(2). Spatial Ancestral Chains (SAC): a mechanism that propagates Gaussian representations from ancestor nodes to descendant nodes through ancestral chains, which can alleviate information loss during Gaussian point inheritance.\n\nAdditionally, this paper contributes an enhanced dataset based on NVIDIA-LS.", "questions": "(1). Could you provide more experimental validation on how the hierarchical depth of TPT and the ancestor chain length of SAC affect performance?\n(2). If the video is very long, would the binary partitioning in TPT result in an overly deep tree? What impact would this have on performance?\n(3). Does the ancestor chain in SAC introduce redundant information? Has this potential issue been considered?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "For the experimental results, the comparisons as well as the ablation studies support the paper’s claims. For the presentation, the writing is generally clear, for example, the paper provides a comparison diagram with previous works, and the ablation study results are presented comprehensively.", "weaknesses": "For the experiments, the paper mentions an improvement in computational efficiency, but it lacks corresponding experimental data on computational time cost.\nFrom the perspective of novelty, the main contributions of this paper are TPT and SAC. TPT is used for video segmentation, while SAC supplements the current information by reusing information from higher-level ancestors. However, this does not constitute a significant theoretical breakthrough. On the other hand, the proposed mechanisms rely on strong assumptions, such as reasonable temporal segmentation and proper root node initialization. Therefore, these characteristics could also become new limitations of this paper (compared with methods that directly impose various monocular video priors onto 3D Gaussians[1]).\n\n[1] D. Wu, F. Liu, Y.-H. Hung, Y. Qian, X. Zhan, and Y. Duan, “4d-fly: Fast 4d reconstruction from a single monocular video,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2025, pp. 16 663–16 673.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LPdtliOsLq", "reviewer": "Reviewer_Dzb1", "summary": "This paper introduces WorldTree, a comprehensive framework designed to improve the efficiency and quality of 4D dynamic scene reconstruction from monocular video. The central contribution lies in its dual decomposition strategy: the Temporal Partition Tree (TPT), which facilitates coarse-to-fine temporal optimization via inheritance, and the Spatial Ancestral Chains (SAC), which handles hierarchical spatial decomposition. This structural approach directly addresses known bottlenecks in dynamic NeRF/Gaussian Splatting literature, specifically the computational overhead of holistic temporal optimization. The methodology is clever and provides a fresh perspective on spatiotemporal modeling.", "questions": "See weaknesses above.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The division of the 4D space into TPT (Temporal) and SAC (Spatial) is the most substantial contribution. The TPT’s inheritance-based optimization scheme is a highly promising avenue for reducing the redundancy and computational load associated with optimizing motion across long video sequences, potentially leading to better temporal coherence.\n\n2. The framework explicitly aims to overcome the coupling inherent in many hierarchical methods. If the TPT successfully decouples temporal optimization, it offers a crucial step towards scaling dynamic reconstruction to very long, complex videos.\n\n3. Committing to monocular input significantly broadens the practical applicability of the work, moving the field closer to real-world use cases where calibrated multi-view rigs are unavailable.", "weaknesses": "1. Dependency on External Segmentation (SAM): The reliance on external segmentation tools like SAM (mentioned in the Appendix and implied by comparisons to HiMoR/SplineGS) is a significant point of concern. If the performance gains are largely attributed to clean, pre-processed dynamic masks, the \"end-to-end\" nature and robustness of WorldTree in unconstrained settings are compromised. A clearer analysis is needed to quantify the degradation when using noisy or no segmentation masks.\n\n2. Scalability and VRAM Limits: The paper notes a constraint on the maximum number of motion nodes (32) due to VRAM capacity on the NVIDIA-LS dataset. While this is a practical limitation of the current implementation (building on MoSca), it raises critical questions about the theoretical scalability of the Tree-Chains spatial representation itself. Is this limit due to the underlying MoSca architecture, or is the complexity of the Tree-Chains structure the bottleneck? This should be analyzed and discussed in the main text.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XSeu72GVh2", "reviewer": "Reviewer_ReiE", "summary": "The authors present WorldTree, a framework for 4D dynamic scene reconstruction from monocular video. It introduces a hierarchical spatiotemporal decomposition via two key components: (1) Temporal Partition Tree (TPT) – a coarse-to-fine hierarchical temporal optimization structure. (2) Spatial Ancestral Chains (SAC) – hierarchical spatial composition from ancestor nodes to decouple motion representations. These innovations allow improved performance on monocular dynamic reconstruction benchmarks (e.g., NVIDIA-LS and DyCheck) without requiring multi-view input or manual point prompts. The method shows impressive gains in mLPIPS and mPSNR and is backed by a complete ablation study.", "questions": "- How does the method handle occlusion? Is there a single accumulated canonical representation built for the entire scene?\n- Your Temporal Partition Tree (TPT) uses a fixed binary split strategy. Did you consider or experiment with adaptive or learned temporal partitioning based on motion complexity or energy? If so, how did they compare? How sensitive is the reconstruction quality to the tree depth?\n- SAC inherits motion features from ancestor nodes. How does the method ensure that outdated or inaccurate ancestral representations do not propagate errors into child nodes?\n- Are there common failure cases you observed (e.g., fast motion, occlusion, camera jitter)? It would be helpful to know where the method performs poorly.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is well-written and the results are comparable to the state-of-the-art methods if not better.\n- The approach reduces the reliance on expensive external priors such as COLMAP points or manual masks, pushing towards a more practical problem setup.\n- The method achieves state-of-the-art performance on the NVIDIA-LS and DyCheck benchmarks, with comparable results to methods using stronger priors.", "weaknesses": "- While the TPT design that uses coarse-to-fine temporal partitioning is scalable, the binary split heuristic may limit the adaptiveness in scenes with irregular motion patterns.\n- It seems that the transition between subtree boundaries is not explicitly handled, which might lead to edge artifacts in the final reconstruction. It would be nice to see more details on how the method handles the transition.\n- Generalization to real-world videos such as those grabbed from the Internet (or simply DAVIS dataset) is not tested.\n- It is not analyzed how the external priors such as RAFT or Metric3D-v2 would affect the method performance. This might cause a problem for videos with ambiguous geometry or fast motions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ylsaHXkgV4", "reviewer": "Reviewer_2Tyt", "summary": "The author(s) propose a unified hierarchical spatiotemporal decomposition framework for the scene to recover a 4D dynamic world from monocular video. It consists of two key components. First, a Temporal Partition Tree (TPT), which recursively splits the video’s timeline into a tree of sub-intervals (coarse-to-fine) for sequential optimization at multiple time scales, and second, a Spatial Ancestral Chains (SAC), which links each node of the time-tree with a chain of inherited scene representations from its ancestors. In essence, each time segment (node) has its own local dynamic model, while also leveraging ancestral information (broader motion context and coarse geometry from higher levels) as a complementary spatial dynamic prior. This inheritance-based design allows model specialization at different temporal scales without losing global context. The overall optimization iteratively refines the scene: starting from a root (entire video) model, then subdividing in time and re-optimizing each segment with inherited parameters in a tree structure. Notably, the tree nodes are optimized in parallel at each level, leveraging the fact that disjoint time segments can be processed independently. The SAC mechanism ensures that finer segments still incorporate multi-level spatial information by querying the chain of ancestor nodes for each segment. Qualitative results show improved and temporally coherent reconstructions of moving subjects than baselines.", "questions": "Q1. In Sec. 3.3, the paper states that the common ancestors of different nodes are independent of each other but have the same optimized initialization. Kindly clarify what this means in practice? For example, when a parent node (covering a certain interval) is split into two children, do you duplicate the parent's Gaussian primitives and motion parameters into each child's ancestral chain as an initialization, and then allow each branch to optimize those copies independently?\n\nQ2. The current implementation always splits intervals evenly at the midpoint in time. Did the author(s) experiment with other criteria for the partition point? For instance, could one split an interval at a frame where the motion or reconstruction error is highest (indicating a change in dynamics)? If not, how sensitive is the method to the exact placement of the split?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The paper presents a hierarchical decomposition of a dynamic reconstruction problem in both time and space. Prior monocular methods generally either optimized a single global motion field over the entire video or imposed a fixed spatial hierarchy. In contrast, the paper proposes an inheritance-based time segmentation that recognizes that different temporal intervals can exhibit distinct motion patterns. Next, Spatial Ancestral Chains ensure that each temporal segment's model includes multi-scale spatial context rather than starting from scratch. This hierarchical motion specialization may help decouple global and local motions.\n\n2.Using BFS expansion, all segments at a given depth can be optimized in parallel. Next, the use of 3D Gaussian splatting as the underlying scene representation looks like a wise choice, i.e., it provides explicit control over primitives and fast rendering.\n\n3. The use of the SAC mechanism, i.e., aggregating the Gaussians of ancestor nodes when rendering a child's frame, effectively layers coarse and fine geometry, preventing the child from needing to re-learn large static structures or global motion from scratch.", "weaknesses": "1.A notable limitation is the dependence on external 2D vision models for initialization. Specifically, monocular depth estimation, optical flow, and feature tracking are used to lift 2D priors into 3D scene representations.\nThe paper does not show how errors in these inputs can affect the final result. Presumably, inaccuracies in initial depth or camera pose estimation could propagate through the hierarchical optimization.\nSuggestion to improve: It would strengthen the work to either (a) demonstrate some robustness analysis (e.g., running the method with intentionally perturbed or lower-quality priors to see if it can recover) or (b) discuss approaches to reduce reliance on these models.\n\n2. The TPT currently uses a very simple scheme for splitting the video (fixed). Each interval is bisected at the midpoint in time (a binary partition), and this continues to a fixed tree depth. This coarse-grained, uniform partition may not be optimal for every video. Real dynamic scenes can have uneven motion. E.g., a subject may remain still for a while then move rapidly, or different time segments have different complexity. \n\nThe paper could be improved by addressing this design choice. Perhaps providing reasoning for why uniform binary splits were chosen may be simplicity? easier parallelism?, etc.\n\n3. WorldTree's pipeline seems to implicitly assume that the input video has a mixture of static and dynamic content, and that at least a portion of the scene can be treated as static for initialization. The authors perform a \"Static Warm-Up\" and Bundle Adjustment (BA) at the root stage, probably to establish camera poses and a base model of the scene before modeling non-rigid motion. This likely works well if, for example, the background is static and only an object or person is moving. The authors should clarify this assumption. Are there failure cases when the scene violates these assumptions?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)