SplineGS: Learning Smooth Trajectories in Gaussian Splatting for Dynamic Scene Reconstruction
Authors: Jihwan Yoon
Sangbeom Han
Jaeseok Oh
Minsik Lee
Venue: ICLR 2025 Poster (2025)
Keywords: 4D Gaussian splatting
Dynamic scene reconstruction
Novel-view synthesis for dynamic scenes
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We propose SplineGS, representing trajectories in 3D-GS using non-uniform rational B-splines, to reconstruct dynamic scenes. The trajectories are combined based on a multi-resolution hash table and an MLP for spatial smoothness.

Abstract: Reconstructing complex scenes with deforming objects for novel view synthesis is a challenging task. Recent works have addressed this with 3D Gaussian Splatting, which effectively reconstructs static scenes with high quality in short training time, by adding specialized modules for the deformations of Gaussian blobs. However, designing an effective deformation module that incorporates appropriate spatiotemporal inductive biases still remains unresolved. To address this issue, we propose SplineGS in this paper, which utilizes non-uniform rational B-splines (NURBS), an extension of B-spline, to represent temporally smooth deformation. A set of representative trajectories are learned based on NURBS, and the individual trajectories of Gaussian blobs are represented as linear combinations of these trajectories for spatial smoothness. The weights of the combinations are trained based on a multi-resolution hash table and an MLP, with the positions of the Gaussian blobs as the keys. Thanks to this design, the proposed method does not need any regularizers for trajectories, which enables efficient training. Experiments demonstrate that the proposed method provides competitive performance over the existing methods with much shorter training time.

Published as a conference paper at ICLR 2025
SPLINEGS: LEARNING SMOOTH TRAJECTORIES IN GAUSSIAN SPLATTING FOR DYNAMIC SCENE RE- CONSTRUCTION
Jihwan Yoon', Sangheom H Jaeseok Oh', Minsik Lee
1 Department of Electrical and Electronic Engineering, Hanyang University School of Electrical Engineering, Hanyang University ERICA fyks,, gkstkdqja88, oh07020, (eeepaper)ehanyangee. kr
Control points Trajectory
Y
x
(a) Jumping jacks
(b) Stand up
(c) NURBS
Figure 1: propose SplineGS, which represents the trajectories Gaussian splatting using non- uniform rational B-spline (NURBS) to reconstruct dynamic scenes. The learned trajectories are shown with rendering results (a) and (b). (c) shows an example of a trajectory based on NURBS.
ABSTRACT
Reconstructing complex scenes with deforming objects for novel view synthe- sis is a challenging task. Recent works have addressed this with 3D Gaussian Splatting, which effectively reconstructs static scenes with high quality in short training time, by adding specialized modules for the deformations of Gaussian blobs. However, designing an effective deformation module that incorporates ap- propriate spatiotemporal inductive biases still remains unresolved. To address this issue, we propose SplineGS in this paper, which utilizes non-uniform rational B- splines (NURBS), an extension of B-spline, to represent temporally smooth defor- mation. A set of representative trajectories are learned based NURBS, and the individual trajectories of Gaussian blobs are represented as linear combinations of these trajectories for spatial smoothness. The weights of the combinations are trained based on a multi-resolution hash table and an MLP,, with the positions the Gaussian blobs as the keys. Thanks to this design, the proposed method does not need any regularizers for trajectories, which enables efficient training. Exper- iments demonstrate that the proposed method provides competitive performance over the existing methods with much shorter training time.
INTRODUCTION
3D reconstruction a fundamental problem in computer vision and graphics that has been studied for decades. Recently, neural radiance fields (NeRF) (Mildenhall al. 2020) have achieved a markable breakthrough in novel view synthesis (NVS) based on implicit representations. However, NeRF still suffered from limited performance and, especially, slow training times, prompting the de-
velopment of various subsequent methods YYu et al. 2021
Fridovich Keil et al. 2022 Chen et al.
"Corresponding author

---

Published as a conference paper at ICLR 2025
2022 Barron al. 2021 2023). Recently, 3D Gaussian Splatting (3D-GS) (Kerbl et al. 2023 has
emerged as a successful alternative by representing scenes based on 3D Gaussian blobs and employ- ing a differentiable tile rasterizer, resulting in fast, high-quality rendering and thus reduced training time.
Despite the success of 3D-GS in static scenes, applying it to dynamic scenes with deforming objects remains a challenging task due to the difficulty modeling effective trajectory representations. Re- cent works have proposed various methods to model the trajectories of 3D Gaussian blobs over time to reconstruct dynamic seenes. Approaches that expresses deformations based on neural rep- resentations (Yngg et al. 2024b; Wu et al. 2024) optimize canonical 3D Gaussian blobs as well as a neural module to predict temporal deformations. Neural representations in these approaches, however, not provide enough inductive biases for deformations, and additional training mecha- nisms or smoothness regularizers are utilized to handle these. Another approach directly optimizes 4D Gaussian blobs (Yang et al., 2024a, Duan et al. 2024), which introduces challenges in optimiza- tion due to the higher dimensionality (4D). This results in reconstruction artifacts, such floating objects that do not align correctly with the scene. Even though these previous works have shown impressive performance, the trajectory representations they use impose some limitations either performance, convergence speed, or both.
In this paper, propose novel method, i..., SplineGS, that models the trajectories of 3D Gaus- sian blobs as temporally smooth representations based on non-uniform rational B-splines (NURBS) (Piegl & Tiller 2012). extension of B-splines (De Boor (001), reconstruct dynamic scenes. NURBS can represent continuous and smooth trajectories, similar to B-splines, while allowing for more flexible and finer adjustments. Our approach specifically obtains a of representative trajec- tories for both positions and rotations through NURBS, of which the parameters are learned during the training. To ensure spatial smoothness, the individual trajectory of each Gaussian blob is ex- pressed as a linear combination of these representative trajectories. Inspired by Müller et al. (2022), the weights for these combinations are obtained using multi-resolution hash table and multi-layer perceptron (MIP),, where the average position of each Gaussian blob serves as the key. The NURBS representation and the linear combination strategy act as implicit regularizers embedded within the structure. Based on this smooth representation, approach achieves high-fidelity rendering and fast training time without relying on any explicit smoothness regularizers, making it efficient for complex dynamic scenes.
We evaluated the proposed method on two dynamic-scem datasets, i.e. D-NeRF (Pumarola et al, 2021] and Neu3l) (Li et al., 2022). Experimental results demonstrate that the proposed method achieves highly competitive performance, even with shorter training time.
Our contributions are summarized follows:
We propose temporally smooth representations for the trajectories of Gaussian blobs based on NURBS.
We enforce spatial smoothness by introducing a low-rank assumption for the trajectories. Each Gaussian blob's trajectory is modeled as weighted sum of representative trajectories. where the weights are learned by multi-resolution hash table and an MLP. This eliminates the need for a separate smoothness regularizer.
The proposed smooth representation exhibits physically plausible trajectories, which leads to high-fidelity rendering and fast training times.
RELATED WORK
In this section, we briefly review the NeRF-based methods for dynamic scenes. Following this, we introduce the original 3D-GS for static scenes and the methods that extend 3D-GS to dynamic scenes.
2.1 NEURAL RADIANCE FIELDS FOR DYNAMIC SCENES
In recent years, NeRF-based methods have gained significant attention for solving the NVS prob- lem of static scenes. This surge in interest was largely due the impressive performance of vanilla
2

---

Published as a conference paper at ICLR 2025
NeRF, which has inspired subsequent research efforts aimed at improving memory usage, training speed, rendering quality, and rendering speed. In addition, many attempts have been made to ex- tend it to dynamic scenes. D-NeRF (Pumarola et al. 2021) and Nerfies (Park et al. 2021a) utilize the coordinates in a canonical space and time embedding vectors as inputs to the MLP to compute the deformations. HyperNeRF (Park et al.) 202Tb) further enhanced this approach by embedding template NeRF in a higher-dimensional space to better capture topological changes. However, these methods often suffer from inefficiency in rendering due to the high number of queries required for the MLP. Spline-NeRF (Knodt 2022) takes all sampled points along ray as inputs to an MLP, which then outputs control points for Bézier curves. However, it is difficult to represent a complex trajectory, e.g., one that has many temporally local changes, based on a Bézier curve. A Bézier curve a global representation, i.e., changes in some control points affect the entire shape of the curve. Accordingly, it becomes challenging to apply this method to long video sequences. On the other hand, K-Planes (Fridovich- Keil et al. 2023) and HexPlane (Cao & Johnson 2023) represented 4D spacetime by combining multiple 2D planes, improving speed and interpretability through ex- plicit representations. Nevertheless, the complexity of dynamic scenes and the inherent limitations of ray- ray-casting- based rendering prevent these methods from achieving high-fidelity and high-speed rendering required for practical applications.
2.2 3D GAUSSIAN SPLATTING FOR DYNAMIC SCENES
Recently, 3D-GS has emerged as a successful solution for reconstructing static scenes, achieving high-fidelity rendering. fast rendering speed, and fast training time. 3D-GS directly optimizes the means, scales, rotations, opacities, and colors of 3D Gaussian blobs that compose a static scene using a differentiable tile-based rasterizer. Inspired by this success, research efforts have begun to explore the use of 3D-GS for reconstructing dynamic scenes (Wu et al.) 2024 Yang et al. 2024b Huang et al.) 2024 Yang et al. (2024a] Duan et al., [224]] Luiten et al. [024]] Bae et al.) 2024). For example, 4D-GS (Wu et al.] 2024) introduces a spatial-temporal structure encoder, which is com- posed of multi-resolution HexPlane and a tiny MIP, along with additional MI.Ps to compute the deformations. In another study (Yang et al. 2024b), purely implicit networks, i.e., MI.ss are used to compute temporal deformations. In SC-GS (Huang et al. 2024). anchor-based spatial warping and MLPs are learned to represent deformations of the overall 3D Gaussian blobs. However, these approaches rely on neural representations, which do not provide enough inductive biases, so ad- ditional training mechanisms smoothness regularizers are utilized. On the other hand, there methods (Yang et al., 2024a Duan et al., (204)) that directly optimize 4D Gaussian blobs. While this approach allows for flexible representation of complex dynamic scenes, also introduces challenges in optimization due to the increased dimensionality (4D). These challenges can lead to reconstruction artifacts, such as floating objects, and achieving high-quality reconstructions requires additional regularizers or extended training time. Alternatively, polynomial and Fourier bases (Lin et al. 2024) can also be used to represent trajectories. However, since this approach must utilize all the bases to calculate deformation at specific time, it can be more challenging to model temporally local deformations and the computational cost can increase significantly when reconstructing long video sequences.
3 THE PROPOSED METHOD
In this section, we propose SplineGS, which represents deformations based on spatiotemporally smooth representations. First, we review 3D Gaussian Splatting in Section 3.1 In Section 3.2 we introduce the B-spline (non-uniform rational B-splines (NURBS), specifically) representations for the trajectories of Gaussian blobs. Then, in Section 3.3 multi-resolution hash table and an MLP are utilized to enforce spatial smoothness based on linear combinations of representative trajectories. Finally, in Section 3.4 we describe the objective function for optimization. The pipeline of SplineGS is illustrated in Figure
3.1 PRELIMINARY: 3D GAUSSIAN SPLATTING
3D-GS Kerbl et al. (023) reconstructs a static scene by directly optimizing the means, covariance matrices, opacities, and colors of 3D Gaussian blobs. Given N Gaussian blobs, the shape of each
3

---

Published as a conference paper at ICLR 2025
A set of representative trajectories
Multi-resouuiinn bash table
Gaussians
Deformed Ganssinns
MLP
W
Ap AR
The pipeline of the proposed method. For each given 3D Gaussian blob, the positions normalized into the range [0, 1] are used as keys for the multi-resolution hash table. The features derived from the position are then mapped to weights via MLP for the linear combination of representative trajectories. Meanwhile, the relative positions and rotations of the representative tra- jectories time are calculated based NURBS. Finally, the deformation of cach Gaussian blob obtained by a linear combination.
Figure 2:
Gaussian is represented by its mean x; and covariance matrix E;;
G.(m) s==
(I)
The covariance matrix must be positive semi-definite, and this is ensured during optimization by separately optimizing the corresponding scales and rotations, i.c. S, and Rş:
Σ, R,S.S R
To render the 3D Gaussian blobs in the onto 2D image space, the 2D covariance matrix Σ in camera coordinates is derived using a given camera extrinsic transform W and the Jacobian of the affine approximation matrix J (Zwicker al. 2001):
Σ JWE,
(3)
Afterward, the pixel's color C' is computed by blending orddrred blobs that overlap the pixel.
N
k-1
C=
Cii (1,a
II j=1
(4)
where } represents the indices of the overlapping blobs and denotes the color of each blob. a is determined by evaluating the 2D Gaussian with covariance Σ and multiplying it by a learned per-point opacity.
3.2 TRAJECTORY REPRESENTATION BASED ON B-SPLINES
We employ non-uniform rational B-splines (NURBS) (Piegl & Tiller 2012) to represent smooth trajectories over time. Here, we first explain the basic formulation based on B-splines, and then extend it to NURBS.
B-splines. B-splines (De Boor 2001) are defined as a set of piecewise polynomial functions, where each polynomial has specific degree denoted as These polynomial functions are connected con- tinuously at specific points called knots, represented by lo, 4,,..., M++ Here, M is the number of control points which will be defined later. The curve's 'ss shape is determined tion of control points and their corresponding basis functions. When p = O, the influence of the i-th control point at given time is described by the basis function follows:
linear combina-
{
if < li+1, otherwise.
Bi,o(t
(S)
4

---

Published as a conference paper at ICLR 2025
The higher-degree B- B-splines, starting from O, be defined using the Cox-de Boor recursion formula:
t t- (tp litp+1 Bi+1.p-1(t).
BBp(t)
to-p t--p-t -tp t, Once the basis functions are determined, the position at any given time is computed as a linear combination of the control points, with these functions acting weights.
1++p+1 1 t++1
S(t, - P
i=0
where Stt, P) denotes the position of the B-spline curve at time t and {P?} are the control points that define the curve's shape.
To represent more complex trajectories, the trajectory of a Gaus- sian blob can be modeled using non non-uniform rational B-spline (NURBS), extension of B-splines. NURBS allows for finer controls of the trajectory by incorporating an additional weight for each control point, and NURBS curve is expressed follows:
Non-uniform rational B-splines.
S(1, w. P) E/ (i)
EI (t),(
where w, denotes the weight associated with the control point P.. Here, assigning uniform weights makes it equivalent to plain B-splines. Assigning non-unitee non-uniform weights allows for more flexible adjustment over the control points that represent complex motion. Additionally, during simpler mo- tions, simplification can be encouraged by adopting low weights.
In this paper, we utilize NURBS represent the trajectory of Gaussian blob (i.e., translation and rotation), and P and w; are regarded as learnable parameters. Note that the NURBS trajectories in this paper are defined as the differences from the canonical (or static) positions rotations. In other words, the trajectories must be combined with the static positions or rotations before use. The static positions and rotations of blobs are separate learnable parameters in the proposed method, along with colors defined as spherical harmonics, opacities, and scales. The degree and the number of control points M are hyperparameters, and the number of knots determined as + M + The locations of knots are defined equally spaced points in the entire time frame.
3.3 BLENDING REPRESENTATIVE TRAJECTORIES FOR SPATIAL SMOOTHNESS
Optimizing B-splines for each 3D Gaussian blob separately has several drawbacks: First of all, it requires large memory space and increase the overall complexity of optimization. More importantly, spatial smoothness, which is expected in between the trajectories proximity, is not guaranteed. To resolve these issues all at once, instead represent the individual trajectories as weighted sums of a few representative trajectories, which only are defined based on NURBS. The weights for these sums are obtained by a multi-resolution hash table and an MLP, which is inspired by instantNGP (Müller et al. 2022), using the position of each Gaussian blob as the key;
B,
for j = 1, 1,,
where Myy R33 represents the center position of the j-th blob, N denotes the number of Gaussians, 8, R is the weight vector of the corresponding Gaussian blob, and L is the number of represen- tative trajectories. We apply tanh to the weights to limit their range to [-1, 1]. Note here that we do not use any explicit
operations, such as kernel regression or K-nearest neighbors (Huang et al. 2024), and instead learn the weights implicitly by the hash table and MLP. This can effectively prevent the overall computational complexity of the training procedure from increasing.
After obtaining the weights, the positions and rotations of the representative trajectories at time 4, calculated based on NURBS, are combined by the weights. The weighted sum operation is applied in the original 3D space for the positions, while it is applied to the axis-angle representations for the rotations and then converted to quaternions.
(t = Σ B5.u S(t, Wk. P
(10)
k=
5

---

Published as a conference paper at ICLR 2025
T--1
( = 2
_ Bjj S(t, wh, Pkat))
(11)
k=0
(y(() = Pj Aug (1),
(12)
(2(1) = R3 (1)
(13)
where Pk E RMx and Pk E I Mx3 represent the control points for positions and rotations, respectively, for the k-th representative trajectory, and wa E RM denotes the corresponding weights (of the control points) in NURBS. Q denotes the viii-angle-to--uatennnnn conversion. The position and rotation from the first two equations are relative ones (from the static position and rotation, re- spectively), so they are combined with the static ones the last two equations. The above technique effectively suppresses unwanted, abrupt spatial variations and promotes gradual changes. Accord- ingly, the proposed method can ensure spatial smoothness without increasing the overall computa- tional complexity, and also has compact feature representations.
3.4 OPTIMIZATION
The proposed method incorporates the above spline module top of the original 3D-GS to estimate deformations. This process is performed end-to-end, and we term this method SplineGS.
Reconstruction loss. The reconstruction loss is defined as the difference between the ground truth image and the rendered image. Similar to 3D-GS (Kerbl et al. 2023). we include both L and CD-SSIM:
Creson (1 - 1) + ALD-SSIM
(14)
where À is predefined weight.
Sparsity. In complex scenes, particularly with real-world datasets, the diversity and complexity of object structures lead to an increased number of Gaussian blobs to represent fine details. This, on the other hand, generates artifacts, such as floating objects, and increases memory usage. To prevent the creation of unnecessary 3D Gaussian blobs, we add I.1 regularization on the opacity values:
L = Lreonn
Ao
l0y1
(15)
5=1
where A is also predefined weight and 0, is the opacity of the jth blob.
4 EXPERIMENTS
In this section, we evaluate SplineGS on the D-NeRF (Pumarola al. 202T and Neu3D (Li al. 2022) datasets. Performance was measured using PSNR, D-SSIM, training time, and FPS, and com- pared with existing NeRF-based and DD--GS-based methods.
4.1 DATASETS
D-NeRF dataset. The D-NeRF dataset consists of monocular video frames with well-aligned cam- era poses. This synthetic dataset includes 8 scenes: bouncingballs, hellwarrior hook, jumpingjacks, mutant, standup, trex and lego, each of which contains between 50 and 200 training images and 20 test images. The experiments were conducted at resolution of 800x800.
Neu3D dataset. This real-world dataset that includes 6 scenes: coffee_marting cook_spinach, cut red.roasted_bee beef. sear starksteak, ftame_staae and flame_salmon. Each scene, except for flame-salmon, salmon. was recorded for 10 seconds at 30 fps using 15 to 20 fixed cameras, with one camera designated as the test view. The experiments were conducted at half resolution, i.e.. 1352x1014. We did not use flame salmon, which consists of 1200 frames, in the experiments, following the practices of many
salmon.
existing methods (Guo al. 2024 Liu et al. 2024
Lu al. 2024

---

Published as a conference paper at ICLR 2025
Table 1: Quantitative comparison for the D-NeRF dataset. We compared our method existing methods on 800x800 resolution test images. Here, "40K" and "80K" in the parentheses indicate the numbers of training iterations. The asterisks (*) indicate that the results were adopted from the 4D-GS paper (Wu et al. 2024), while the daggers (f) indicate that those were from the original papers. All the other results were reproduced in our experiments. The average PSNR and SSIM are measured across all scenes, with some cells highlighted to represent the best second best and third best Additionally, report the average training time and rendering speed in frames per second (FPS) for each method.
Method
PSNR
SSIM
Training time!
FPSt
K-Planes
Fridovich-Keil et al.
2023
30.67
0.9672
52 mins
0.97
HexPlane
(Cao & Johnson
2023
31.02
0,9680
11 mins
2.5
TiNeu Vox
Fang et al.
2022)
31.35
0,9613
28 mins
1.5
D-3DGS.1
(Yang et al.
2024b)
38.50
0.9857
22 mins
70
4D-GS'
Wu et al.
2024)
34.06
0.9787
13 mins
62
4DGS
Yang
2024a
32.87
0.9649
7.5 hours
135
CoGS
Yu et al.
2024)
37.90
0,9842
CompDynGSt
Katsumata et al.
2024
33.21
0,9770
8 mins 1.1 hours
150
SC-GS
Huang 2024
39.53
0.9906
111
SplineGS (40K)
39.
0.9866
17 mins 35 mins
188
SplineGS (80K)
39.44
0.9868
188
4D-GS
Wu et al
2024
4DGS
Yang al. 2024n
D.DDS
Yang nl.
2024h
SplineG8
GT
Qualitative results for the D-NeRF dataset. We present the rendering results of various methods, including ours, with the ground truth (GT) for standup scene in the D-NeRF dataset. Ad- ditional rendering results for other scenes are provided in the appendix.
Figure 3:
4.2 IMPLEMENTATION DETAILS
We implemented SplineGS by integrating the proposed spline-based deformation module, imple- mented using PyTorch (Paszke et al., 2019), with the original 3D-GS (Kerbl et al. 2023). For the D-NeRF dataset, we randomly sample 100,000 points within a 3D space, where each axis ranges from -1.3 to 1.3, to initialize 3D Gaussian blobs. For the Neu3D dataset, the initial set of 3D Gaus- sians is obtained by calculating structure- seuuctre-frrm--eee from-motion (Schönberger & Frahm (016) for the first frames of each video and downsampling the results. The detailed settings for this were identical to those in 4D-GS (Wu et al. 2024). The initialization of scales, rotations, colors, opacities, and their respective learning rates follows the 3D-GS. Similarly, during training, adopt the clone and split strategies with a threshold of 2 x 10-4 as in 3D-GS. For the D-NeRF dataset, we also utilized the repeated opacity reset suggested in 3D-GS. The training process begins with initial 3K iterations of static 3D-GS training (ignoring any deformations), followed by an end-to-end training with the proposed spline module.
For the spline module, the number of control points (per trajectory) was set as one for every four eight frames collected from a single camera. These control points were initialized with normal distribution with a standard deviation 10-5 The number of representative trajectories was set to either or 128, depending on the dataset. The initial learning rate for the control points was set to 10-3 for the D-NeRF dataset and 5 10 for the Neu3D dataset, with decay factor of 0.99 for every 100 iterations. Similarly, the learning rate of wkk was initialized to 10 for the D-NeRF dataset and 5 x 10-4 for the Neu3D dataset, and also decayed by factor of 0.99 for every 100 iterations. The multi-resolution hash table and MLP were implemented using tiny-cuda-nn (Müller 2021), with 16 levels and 4 features per level. The maximum table size parameter, base resolution,

---

Published as a conference paper at ICLR 2025
Table Quantitative comparison for the Neu3D dataset. We compared our method with existing methods on 1352x1014 resolution test images. Here, "20K" and "40K" in the parentheses indicate the numbers of training iterations. The conventions for typographical marks and color codes are generally identical to Table [ We measured the average PSNR and SSIM across all scenes except flame salmon, following the practice of many existing works (Guo et al., 2024 Liu et al. 2024 et al. (0024). The double dagger (i) denotes that the mean values were calculated across all scenes in the respective work.
Method
PSNRT
SSIMT
Training timel
FPSt
NeRFPlayer' Song et al.
2023
32.12
0,9206
5.5 hours
0.045
K-Planes' (Fridovich- Keil et al.
2023
31.86
0.9658
1.8 hours
0.23
4D-GS* Wu et al. 2024
31.54
0.9444
40 mins
30
Gaussian-Flow (Lin et al. 2024
32.00
0.9700
42 mins
4DGS
Yang et al.
2024a)
32.53
114
STG [Li al. 2024)
32.57
0.9740
42 minsx6
140
SplineGS (20K)
32.52
0,9484
27 mins
76
SplineGS (40K)
32.60
0.9496
55 mins
76
4D-GS (Wu et al.
2024
STG Li ct al.
2024
SplineGS
GT
Figure 4: Qualitative results for the Neu3D dataset. We present the rendering results of various methods, including ours, and the ground truth (GT) for coffee martini the Neu3D dataset. We also provide zoomed-in results in the second row of the figure. Additional results for other scenes are provided in the appendix.
and per-level scale were 19, and 2, respectively, for the D-NeRF dataset and 15, 16, and 1.5, respectively, for the Neu3D dataset. The MLP consisted of 4 hidden layers with hidden dimensions of 128 for all the layers. The optimization of the spline module, like 3D-GS, was performed using Adam (Kingma 2014).
The performance of SplineGS was evaluated after 40K and 80K iterations on the D-NeRF dataset, and 20K and 40K iterations on the DyNeRF dataset, with results averaged over three runs. All the experiments were conducted on single Nvidia Geforce RTX 3090.
4.3 RESULTS
Results on synthetic scenes. Here, we compared the proposed model with various NeRF-based
methods: K- K-Planes (Fridovich- Keil et al. 2023), HexPlane (Cao & Johnson, 2023), and TiNeu Vox (Fang et al. (2022); ; as well as 3D-GS based methods: Yang al. (2024b). 4D-GS (Wu et al. 2024) 4DGS Yang et al. 2024a), CoGS YYu et al. 2024), Katsumata et al. (2024) and SC-GS (Huang et al. ) on the D-NeRF dataset. A quantitative evaluation of average PSNR, SSIM, training time. and FPS is shown in Table T Here, the proposed method ranks second best, but with significantly shorter training time compared to the best performing one while having very small performance gap.
8

---

Published as a conference paper at ICLR 2025
Table 3: Effect of the degree of NURBS. We quantitatively evaluated the effect of the degree of NURBS in our method on synthetic and real-world scenes.
Bouncing Balls
Hook
Coffee Martini PSNRT
Sear Steak
Degree p
PSNRT
SSIM
PSNR
SSIM
SSIM
PSNRT
SSIM
p
36.03
0.9915
28.43
0,9441
28.42
0.9127
33.61
0.9606
40.93
0.9951
38.09
0.9879
28.56
0,9154
33,49
0,9610
=
41.26
0,9954
38.59
0,9890
29.01
0,9178
33.66
0,9621
p 41.81
0.9956
38.71
0.9891
29.26
0,9189
33.82
0.9624
= 39.97
0.9946
26.98
0.9311
28.64
0.9184
33.59
0.9621
Table Ablation study. We quantitatively evaluated the effect of different components of the pro-
posed method on synthetic and real-world scenes.
Bouncing Balls
Hook
Coffee Martini PSNR
Sear Steak
Method
PSNR
SSIM
PSNR1
SSIM
SSIM
PSNR1
SSIM
w/o hash
41.20
0.9949
36.39
0.9813
27.98
0.9041
30.87
0,9473
w/o MLP
38.33
0.9927
37.80
0.9879
28.80
0.9160
33.19
0,9605
w/o NURBS
39.83
0.9944
26.82
0.9306
28.85
0.9181
32.08
0,9559
proposed 41.81
0.9956
38.71
0.9891
29.26
0.9189
33.82
0.9624
This observation suggests that the proposed spline representation is well-suited for dynamic scene reconstruction. More detailed qualitative and quantitative comparisons for each scene are provided in the appendix. Figure 3 shows the qualitative comparison for standup. Here, the proposed method yields more accurate rendering compared to the existing methods.
confirm that
Results Neu3D. For the Neu3D dataset, we compared with the following NeRF-based methods: NeRFPlayer (Song et al. 2023), K- K-Planes Fridovich- et al. (2023); as well as 3D-GS-based methods: 4D-GS (Wu et al. 2024). 4DGS Yang al. 2024a) and STG (Li al., 2024). Table presents the quantitative comparison. For this dataset, the proposed method achieves the best average PSNR after 40K iterations. Moreover, the proposed method always achieves better performance per training time than the existing methods. Here. STG divides the entire video frames (300 frames) into 6 parts (50 frames each) and then processes them separately, which why the training time described as "42 mins 6". Again, the detailed results for each scene are provided in the appendix. Figure 4 shows the qualitative comparisons on the coffee martini scene. Here, the proposed method provides crisper details for the coffee martini.
SplineGS concentrates on the temporal changes of position and rotation, without modeling those of density and color unlike some recent methods (Yang et al. 2024a Li al. 2024 Lin et al. 2024]. Nevertheless, it achieves high-fidelity rendering and fast convergence, which suggests that the proposed spline representation provides a better alternative for modeling deformations.
4.4 ANALYSES AND ABLATION STUDY
Effeet of the degree of NURBS. In NURBS, the value of trajectory specific time instance is influenced by surrounding
control points. Accordingly, as the degree increases, the curve becomes less sensitive to the changes in control points, which makes the trajectory smoother but also increases the computational cost. Table shows the performance variations according to the changes in the degree for bouncing balls, hook, coffee_martin, martini, sear_steak. The results indicate that p 3 generally provides optimal performance for dynamic scenes.
provide an ablation study on the spline module in Table Here. "w/o hash" indicates that the multiresolution hash table was replaced with
Ablation study on the module components.
(-1 with L = 10. In this case, performance generally decreases except for bouncingballs, which contains less complex motions. In the case of "w/o MLP". the hash table directly outputted the weights for the linear combination, which generally
positional encoding (p)) = (sin (2*xp) cos (2*Tp)) k=
resulted in lower performance. In the "w/o NURBS" scenario, the ML. directly outputs the values

---

Published as a conference paper at ICLR 2025
of a trajectory given time instance the features generated by the hash table concatenated with the positional encoding (with L = 6) of t, and then go through the MLP to output the values. This variation leads to lower performance, which suggests that the neural representation alone lacks sufficient spatiotemporal inductive biases.
5 CONCLUSION
We proposed SplineGS, which utilizes smooth representation in both space and time for the tra- jectory of each 3D Gaussian blob, to reconstruct dynamic scenes. Thanks to this representation, the proposed method achieves state-of-the-ar at least competitive performance without the need for a regularizer on the trajectories. The proposed spline representation is an explicit representa- tion, in that the NURBS directly represents the trajectories of Gaussian blobs. This is well suited for dynamic scene reconstruction, providing fast convergence. The proposed model only handles deformations in positions and rotations, so has limitations in reconstructing objects that appear or disappear in scenes, as well as objects with rapidly changing colors due to lighting variations. Incorporating advanced light reflections and accounting for changes in blob density will be explored in future work.
ACKNOWLEDGMENTS
This work partly supported by Innovative Human Resource Development for Local Intellec- tualization program through the Institute of Information & Communications Technology Planning & (VII grant funded by the Korea governmen((MSIT) UUTP-2025-RS-2000-1101777 50%) and partly by the National Research Foundation of Korea(NRF) grant funded by the Korea goVer)men(MMSIT) IRS-2023-NR076576, 50%)
REFERENCES
Jeongmin Bae, Scoha Kim, Youngsik Yun, Hahyun Lee, Gun Bang, and Youngjung Uh. Per Per-gaussian embedding- deformation for deformable 3d gaussian splatting. In European Conference on Computer Vision (ECCV), 2024.
Jonathan T. Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualla, and Pratul P. Srinivasan. Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields. ICCV, 2021.
Jonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul P. Srinivasan, and Peter Hedman. Zip-nerf: Anti-aliased grid-based neural radiance fields. ICCV, 2023.
Ang Cao and Justin Johnson. Hexplane: A fast representation for dynamic scenes, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 130-141, 2023.
Anpei Chen, Zexiang Xu, Andreas Geiger, Jingyi Yu, and Hao Su. Tensorf: Tensorial radiance fields. In European Conference Computer Vision (ECCV), 2022.
Devikalyan Das, Christopher Wewer, Raza Yunus, Eddy Ilg, and Jan Eric Lenssen. Neural para- metric gaussians for monocular non-rigid object reconstruction. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10715-10725, 2024.
Carl De Boor. A practical guide to splines; rev, ed. Applied mathematical sciences. Springer, Berlin, 2001. URL FRRLttps: // cern cern. ch/ record/cood/oooooo 140pd//44411881
Yuanxing Duan, Fangyin Wei, Qiyu Dai, Yuhang He, Wenzheng Chen, and Baoquan Chen. 4d- rotor gaussian splatting: Towards efficient novel view synthesis for dynamic scenes. In ACM SIGGRAPH 2024 Conference Papers, pp. 1-11, 2024.
Jiemin Fang, Taoran Yi, Xinggang Wang, Lingxi Xie, Xiaopeng Zhang, Wenyu Liu, Matthias Nießner, and Qi Tian. Fast dynamic radiance fields with time-aware neural voxels, In SIGGRAPH Asia 2022 Conference Papers, 2022.
10

---

Published as a conference paper at ICLR 2025
Sara Keilvvich-eeil, Alex Yu, Matthew Tancik, Qinhong Chen, Benjamin Recht, and Angjoo Kanazawa. Plenoxels: Radiance fields without neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 5501-5510, 2022.
Sara Fridovich-Keil, Giacomo Meanti, Frederik Rahbæek Warburg, Benjamin Recht, and Angjoo Kanazawa, K-planes: Explicit radiance fields in space, time, and appearance. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12479- 12488, 2023.
Zhiyang Guo, Wengang Zhou, Li Li, Min Wang, and Houqiang Li. Motion-aware 3d gaussian splatting for efficient dynamic scene reconstruction. arXiv preprint arXiv:2403.114477 2024.
Yi-Hua Huang, Yang-Tine Tian Sun, Ziyi Yang, Xiaoyang Lyu, Yan-Pei Cao, and Xiaojuan Qi. Sc- gs: Sparse-controlled gaussian splatting for editable dynamic scenes. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 4220-4230, 2024.
Kai Katsumata, Due Minh Vo, and Hideki Nakayama. compact dynamic 3d gaussian representa- tion for real-time dynamic view synthesis. In ECCV, 2024.
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splat- ting for real-time radiance field rendering. ACM Transactions on Graphics, 42(4), July 2023. URL htt.::ttp://ppptee sam inria. fr/ http:/uuggaph/dddgdaaaaaa innisssinn-nee woann-splatee sn-sslattingggg/////oooaa
Diederik Kingma. Adam: A method for stochastic optimization. arXiv preprint arXiv: arti:v 1r1i::1412.6998) 2014.
Julian Knodt. Continuous dynamic-nerf: Spline- nerf, 2022. URL SKLLhttps://axxxv.rr/////
2203. 3800
Minsik Lee, Jungchan Cho, and Songhwai Oh. Consensus of non-rigid reconstructions. In Pro- ceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 4670-4678, 2016.
Tianye Li, Mira Slavcheva, Michael Zollhoefer, Simon Green, Christoph Lassner, Changil Kim, Tanner Schmidt, Steven Lovegrove, Michael Goesele, Richard Newcombe, et al. Neural 3d video synthesis from multi-view video. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5521-5533 2022.
Zhan Li, Zhang Chen, Zhong Li, and Yi Xu. Spacetime gaussian feature splatting for real-time dynamic view synthesis. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2024.
Youtian Lin, Zuozhuo Dai, Siyu Zhu, and Yao Yao. Gaussian-flow: 4d reconstruction with dynamic 3d gaussian particle. In Proceedings of the IEEE/CVE Conference on Computer Vision and Put- tern Recognition, pp. 21136-200 22146221115551111111111111111111 2024.
Qingming Liu, Yuan Liu, Jiepeng Wang, Xianqiang Lv, Peng Wang, Wenping Wang, and Junhui Hou. Modgs: Dynamic gaussian splatting from causually-capture monocular videos. arXiv preprint arXiv:2406.0004 00434, 2024.
Zhicheng Lu, Xiang Guo, Le Hui, Tianrui Chen, Ming Yang, Xiao Tang, Feng Zhu, and Yuchao Dai. 3d geometry-aware deformable gaussian splatting for dynamic view synthesis. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2024.
Jonathon Luiten, Georgios Kopanas. Bastian Leibe, and Deva Ramanan. Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis. In 3DV, 2024.
Ben Mildenhall, Pratul P. Srinivasan, Matthew Taneik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. In ECCV, 2020.
Thomas Müller.
Einy-cuda-nn tiny-cuda-nn, 4 2021.
URI.
com
Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics prim- itives with a multiresolution hash encoding. ACM transactions on graphics (TOG), 41(4):1-15, 2022.
11

---

Published as a conference paper at ICLR 2025
Keunhong Park, Utkarsh Sinha, Jonathan T. Barron, Sofien Bouaziz, Dan B Goldman, Steven M. Seitz, and Ricardo Martin- Martin-Brualla, Nerfies: Deformable neural radiance fields. ICCV, 2021a.
Keunhong Park, Utkarsh Sinha, Peter Hedman, Jonathan T. Barron, Sofien Bouaziz, Dan B Gold- man, Ricardo Martin-Brualla, and Steven M. Seitz. Hypernerf: A higher-dimensional representa- tion for topologically varying neural radiance fields. ACM Trans. Graph., 40(6), dec 202
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang. Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang. Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high- performance deep learning library. In Advances in Neural Information Processing Systems 32, pp. 8024-8035. Curran Associates, Inc., 2019. URL htt /tt.attte//ppaass...rr....o.oo.. 901 10155-ytorch-aae mperat ve sty oege
ibbary.
pdf
Les Piegl and Wayne Tiller. The NURBS book. Springer Science & Business Media, 2012.
Albert Pumarola, Enric Corona, Gerard Pons-Moll, and Francesc Moreno-Noguer. D-nerf: Neural radiance fields for dynamic scenes. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pultern Recognition, pp. 10318-10327, 2021.
Johannes L. Schönberger and Jan-Michael Frahm. fiructre-from-mottie revisited. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4104 4113, 2016. doi: 10.11099CVPR.2206.445
Liangchen Song, Anpei Chen, Zhong Li, Zhang Chen, Lele Chen, Junsong Yuan, Yi Xu, and An- dreas Geiger. Nerfplayer: A streamable dynamic scene representation with decomposed neural radiance fields. IEEE Transactions on Visualization and Computer Graphics, 29(5):2732-277
2023. doi: 10.
Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 4d gaussian splatting for real-time dynamic scene rendering. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 20310- 20320, June 2024.
Zeyu Yang, Hongye Yang, Zijie Pan, and Li Zhang. Real-time photorealistic dynamic scene rep- resentation and rendering with 4d gaussian splatting. In International Conference on Learning Representations (ICLR), 2024a.
Ziyi Yang, Xinyu Gao. Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. Deformable 3d gaussians for high-fidelity monocular dynamic scene reconstruction. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 20331-20341, 2024b.
Alex Yu, Ruilong Li, Matthew Tancik, Hao Li, Ren Ng, and Angjoo Kanazawa. PlenOctrees for real-time rendering of neural radiance fields. In ICCV, 2021.
Heng Yu, Joel Julin, Zoltán Á Milacski, Koichiro Niinuma, and László A Jeni. Cogs: Controllable gaussian splatting. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 21624-21633, 2024.
Matthias Zwicker, Hanspeter Pfister, Jeroen Van Baar, and Markus Gross. Ewa volume splatting. In Proceedings Visualization, 2001. VIS"O/., pp. 29-538 IEFF, 2001.
12

---

Published as a conference paper at ICLR 2025
APPENDIX
A EXCLUSION OF SCALE VARIATIONS IN SPLINEGS
SplineGS does not model the scale variations of Gaussian blobs. This can prohibit certain objects, such as jelly- or smoke-like structures. However, for most types of objects, their parts exhibit mini- mal scaling changes. It is also reported in the literature that most non-rigid deformations are likely rigid transforms at infinitesimal levels (L.ee et al.) 2016). This perspective is also adequate for Gaus- sian splatting, considering that the blobs represent some small parts forming the objects in the scene. Therefore, we concentrate on modeling deformations in positions and rotations only, In reality, mod- eling scale variations tend to induce overfitting in our experience due to the additional degrees of freedom. We believe that scaling variations should be addressed with careful consideration as a separate research focus.
B DISCUSSION ON COLOR AND DENSITY MODELING
In real-world scenarios, dynamic scenes involve not only geometric deformations of objects but also variations in color and density. Incorporating a model that accounts for these factors can enhance performance. However, color and density exhibit distinct behaviors that require additional consider- ations. Color is highly sensitive to changes in viewpoint and lighting conditions, requiring a model that can capture high-frequency variation. Likewise, density changes, such as the appearance dis- appearance of objects, also require the capacity to represent high-frequency variation. Furthermore, the existing densification strategy in 3D-GS is primarily designed for static scenes and does not ex- plicitly handle temporal variations. Without careful consideration of these factors, modeling color and density variations can unintentionally interfere with the representation of physically plausible trajectories. Accordingly, addressing color and opacity variations require additional modelings pable of handling high-frequence changes. Representations such as higher-degree NURBS, Fourier series, wavelets may offer viable solutions, and further exploration in this direction is left as future work.
C MORE RESULTS
Table presents quantitative comparisons of PSNR and SSIM across all scenes in the D-NeRF dataset. In each scene, the proposed method achieves highly competitive results. Table 6 presents quantitative comparisons of LPIPS across all scenes in the D-NeRF dataset. Here, we can confirm that SplineGS also achieves the best LPIPS performance in most cases. Figure 5Jshows scatter plot comparing the number of blobs, FPS, training time, and PSNR of various methods for the D-NeRF dataset. Here, SplineGS has a high average FPS even though its average number of Gaussian blobs is relatively large compared to those of the other methods. This confirms that the core design of the proposed method is indeed quite efficient, and it still achieves high performance. As illustrated in Figure 6 our method produces more accurate renderings than existing approaches. Similarly. Table 7 shows the quantitative comparisons of PSNR and SSIM across five scenes the Neu3D dataset. Notably, our model achieves the best average performance. Figure provides the rendering results for every 75 frames. Table 8 presents a quantitative comparison of PSNR for three vrig scenes in the HyperNeRF (Park al. 2021b) dataset. Similar to Neu3D, the HyperNeRF dataset provides real-world scenes. Here, SplineGiS achieves performance comparable to other methods. A critical problem with the HyperNeRE dataset is that the provided camera pose information is not accurate. An additional camera pose optimization is required to handle this problem, which is another set of problems left as future work. Table shows the per-scene attributes of SplineGS, such as training time, FPS, and number of blobs, for the D-NeRF and Neu3D datasets.
13

---

Published as a conference paper at ICLR 2025
Table 5: Quantitative comparison for the D-NeRF dataset. We compared our method to existing methods on 800x800 resolution test images. Here, "40K" and "80K" the parentheses indicate the numbers of training iterations. The asterisks (*) indicate that the results were adopted from the 4D-GS paper (Wu et al. 2024), while the daggers (f) indicate that those were from the original papers. All the other results were reproduced in experiments. The average PSNR and SSIM measured across all scenes, with some cells highlighted to represent the best second best and third best
Trex
Jumping Jacks
Hell Warrior
Method
PSNR
SSIM
PSNR1
SSIM
PSNR
SSIM
K K-Planes' HexPlane
FFidovich- Keil et al.
2023
30.43
0.9737
31.11
0.9708
24.58
0,9520
Cao Johnson
2023)
30.67
0.9749
31.31
0,9729
24.55
0.9443
TiNeu Vox
Fang et al.
2022
31.25
0.9666
33.49
0.9771
27.10
0.9638
D-3DGS
Yang et al.
2024b
38.10
0.9933
37.72
0.9897
41.54
0,9873
4D-GS"
Wu et al.
2024)
34.23
0,9850
35.42
0,9857
28.71
0,9733
4DGS
Yang el al.
2024a
30.38
0.9743
32.10
0.9639
34.32
0,9536
CoGS
Yu et al.
2024
37.25
0.9923
37.48
0.9891
40.43
0,9812
NPGs
Das et al.
2024
32.10
0,9818
33,97
0.9828
38.68
0,9780
CompDynGS*
Katsumata
al.
2024
28.17
0.9740
32.93
0.9840
35.36
0.9650
SC-GS
Huang 2024
39.61
0,9981
39.43
0.9964
42.20
0.9925
SplineGS (40K)
37.96
0.9925
38.29
0.9906
42.26
0.9890
SplineGS (80K)
38.77
0.9931
38.66
0.9908
42.52
0.9893
Stand
Bouncing Balls
Mutant
Method
PSNR
SSIM
PSNRT
SSIM1
PSNR1
SSIM
K-Planes
Fridovich et al.
2023
33.10
0.9793
40.05
0.9934
32.50
0.9713
HexPlane
[Cao Johnson
2023
34.40
0.9839
39.86
0.9915
33.67
0,9802
TiNeu Vox
(Fang al.
2022)
34.61
0.9797
40.23
0.9926
30.87
0.9607
D-3DGS'
Yang
2024b
44.62
0.9951
41.01
0.9953
42.63
0.9951
4D-GS"
Wu ct al
2024)
38.11
0,9898
40.62
0.9942
37.59
0,9880
4DGS
Yang
2024a
38.83
0.9857
33.28
0.9842
37.43
0,9842
CoGS
Yu ct al
2024
43.35
0,9929
40.98
0.9958
42.14
0,9937
NPGs
Das al. 2024
38.20
0,9889
36.02
0.9840
CompDynGS'
Katsumata ct al.
2024
40.21
0.9940
33.29
0.9840
38.04
0.9940
SC-GS (Huang et al.
2024
46.47
0,9989
41.39
0.9960
43.42
0,9989
SplineGS (40K)
45.50
0,9940
41.71
0.9956
43.29
0,9960
SplineGS (80K)
46.00
0,9944
41.81
0.9956
43.73
0,9962
Hook
Lego
Mean
Method
PSNRT
SSIMI
PSNR1
SSIM1
PSNR1
SSIM1
K-Planes"
Fridovich- Keil et al.
2023
28.12
0.9489
25.49
0.9483
30.67
0.9672
HexPlane
Can & Johnson
2023)
28.63
0,9572
25.10
0.9388
31.02
0.9680
TiNeu Vox
Fang et al.
2022)
28.63
0,9433
24.65
0.9063
31.35
0,9613
D-3DGS
Yang et al.
2024b
37.42
0.9867
24.94
0.9432
38.50
0.9857
4D-GS*
Wii et al
2024)
32.73
0,9760
25.03
0.9378
34.06
0,9787
4DGS
Yang
2024a
31.92
0,9546
24.70
0.9189
32.87
0,9649
CoGS
Yu et al.
2024
36.43
0,9838
25.16
0.9451
37.90
0,9842
NPGs
Das al.
2024
33.39
0,9735
24.63
0.9312
CompDynGS'
Katsumata ct al.
2024
33.43
0,9810
24.26
0.9400
33.21
0.9770
SC-GS (Huang et al.
2024
38.88
0,9956
24.92
0.9485
39,53
0,9906
SplineGS (40K)
38.61
0,9891
25.29
0.9457
39.11
0,9866
SplineGS (80K)
38.71
0.9891
25.29
0.9457
39.44
0.9868
14

---

Published as a conference paper at ICLR 2025
Table 6: LPIPS comparison for the D-NeRF dataset. We compared our method to existing meth- ods on 800 800 resolution images based on LPIPS. The daggers (f) indicate that those were from the original papers.
Method
Trex
Jumping Jakes
Hell Warrior
4D-GS*
0.0131
0.0128
0.0369
D-3DGS'
0.0098
0.0126
0.0234
SC-GS
0.0119
0.0115
0.0280
SplineGS
0.0047
0.0065
0.0091
Method
Stand up
Bouncing Balls
Mutant
4D-GS*
0.0074
0.0155
0.0167
D-3DGS
0.0063
0.0093
0.0052
SC-GS
0.0072
0.0216
0.0069
SplineGS
0.0027
0.0025
0.0019
Method
Hook
Lego
Mean
4D-GS*
0.0272
0.0382
0.0210
D-3DGS*
0.0144
0,0183
0,0124
SC-GS
0.0139
0.0499
0.0119
SplineGS
0.0065
0.0298
0.0080
160
140
FPS
PSNN
120
-36
100
335
D 3DGs
10
Number of blobs
Figure 5: A scatter plot of various methods based on the number of blobs, FPS, training time, and PSNR. We compared SplineGS to existing methods on a seatter plot for the D-NeRF dataset. The x-axis represents the average number of blobs, and the y-axis represents the average FPS. The size of each circle indicates the average training time, while the color represents the average PSNR.
15

---

Published as a conference paper at ICLR 2025
4D-65
2014
4DGS
2024a
D-DDES
20246
SplineCSS
Bouncing Balls
Hellwarnee
Hack
Jumping Jacks
Lego
Mutani
Thex
Figure 6: Qualitative results on the D-NeRF dataset. We present the rendering results of various methods, including ours, for all scenes in the D-NeRF dataset. The first to third columns show the results of existing methods, while the fourth and fifth columns show those of the proposed method and the ground truth, respectively. Each row shows the results of a different scene.
16

---

Published as a conference paper at ICLR 2025
Table 7: Quantitative comparison for the Neu3D dataset. We compared our method with existing methods 1352x 1014 resolution test images. Here, "20K" and "40K" in the parentheses indicate the numbers of training iterations. The conventions for typographical marks and color codes are generally identical Table [ We measured the average PSNR and SSIM across all scenes except flame salmon, following the practice of many existing works (Guo et al. 2024 Liu et al. 2024 Lu
al. 2024).
Coffee Martini
Spinach
Cut Beef'
Method
PSNR1
SSIM
PSNR
SSIM
PSNR1
SSIM
NeRFPlayer'
(Song et al.
2023
32.05
0.9380
32.06
0.9300
31.83
0,9280
K-Planes'
(Fridovch--Ket et al.
2023
29.99
0.9530
32.60
0.9660
31.82
0.9660
HexPlane
Cao Johnson
2023)
31.86
0.9830
32.71
0.9850
4D-GS"
(Wu et al.
2024
27.34
0.9050
32.46
0,9490
32,90
0.9570
4DGS'
Yang et
2024a
28.33
32.93
33.85
0,9800
STG
(i ct al.
2024)
28.61
0.9585
33.18
0,9785
33.52
0.9795
SplineGS (20K)
29.16
0.9171
33.04
0.9538
33.42
0,9541
SplineGS (40K)
29.26
0.9189
33.09
0,9542
33,49
0,9544
Sear Steak
Flame Steak
Mean
Method
PSNRT
SSIM
PSNR1
SSIM
PSNR
SSIM
NeRFPlayer'
(Song et al.
2023
32.31
0.9400
27.36
0.8670
32.12
0.9206
K- K-Planes
(Fridovich-Ker Keil et al.
2023
32.52
0.9740
32.39
0.9700
31,86
0.9658
HexPlane
CCo Johnson
2023)
32.09
0.9860
31.92
0.9880
4D-GS'
Wu et al.
2024)
32.49
0.9570
32.51
0.9540
31.54
0.9444
4DGS
YYng et al.
2024a
33.51
34.03
32.53
STG
(Li et al.
2024
33.89
0.9826
33.64
0,9824
32.57
0.9740
SplineGS (20K)
33.75
0.9621
33.23
0.9578
32,52
0.9484
SplineGS (40K)
33.82
0.9624
33.33
0,9582
32.60
0.9496
Frame 75
Frame 150
Frame 225
GT
SplineGS
GT
SplineGS
SplineGS
Coffee Mariii
Cat Beef
Flane Steak
Sear Steak
Spinach
Figure 7: Qualitative results on the Neu3D dataset. We show the qualitative results of SplineGS on the Neu3D datasets except for flame falm-s We provide the ground truth and the results of SplineGS for the 75th, 150th, and 225th frames. Each row shows the results of a different scene.
17

---

Published as a conference paper at ICLR 2025
Table 8: Quantitative comparison for the HyperNeRE dataset. We compared our method to ex- isting methods half-resolution test images and measured PSNR for three vrig scenes. The daggers (f) indicate that those were from the original papers.
Method
3D Printer
Chicken
Broom
D-3DGS Yang al. 2024b
20.28
22.76
20.47
4D-GS
Wu et al.
2024)
22.10
28.70
22.00
SplineGS
21.97
29.13
20.56
Table Per-scene attributes SplineGS D-NeRF and Neu3D datasets. We present the training time, FPS, and number of blobs for each scene. The number of representative trajectories L was 64 for mutant and bouncing balls, while that was 128 for the other scenes.
D-NeRF
Neu3D
Scene
Time (min)
FPS
#Blobs
Scene
Time (min)
FPS
#Blobs
Trex
41
133
250K
Coffee Martini
67
62
386K
Jumping Jack
33
197
126K
Spinach
53
81
140K
Hell Warrior
28
263
52K
Cut Beef
52
79
167K
Stand Up
31
227
98K
Sear Steak
52
84
138K
Bouncing balls
38
176
167K
Flame Steak
50
76
143K
Mutant
34
192
165K
Hook
33
186
161K
Lego
42
133
320K
18


Reviews:
[{"review_id": "2F8v2FnA0k", "reviewer": "Reviewer_cEb3", "summary": "This paper proposes SplineGS, a novel method for reconstructing dynamic scenes using 3D Gaussian splatting. The authors propose to utilize Non-Uniform Rational B-Splines (NURBS) to represent the trajectories of Gaussian blobs, aiming to achieve temporally smooth deformations. This approach is a departure from existing methods that rely on neural representations, which often require explicit regularization or specialized training mechanisms to ensure smooth trajectories. The paper demonstrates the effectiveness of SplineGS on two datasets, D-NeRF (synthetic) and Neu3D (real-world), showing competitive or superior performance compared to some existing methods.", "questions": "See the Weaknesses section.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper presents a novel approach by employing NURBS to represent Gaussian blob trajectories in dynamic scene reconstruction, a significant contribution to the field of 3D Gaussian splatting.\n\nBy utilizing NURBS, SplineGS provides smooth deformations without the need for explicit regularizers or complex training mechanisms, leading to faster and more efficient training. This efficiency translates to competitive or state-of-the-art performance on both synthetic (D-NeRF) and real-world (Neu3D) datasets, demonstrating its effectiveness in reconstructing dynamic scenes.", "weaknesses": "1. While the authors claim that NURBS inherently provide temporal smoothness, the paper lacks direct visual evidence to support this claim. The authors should include visualizations of the Gaussian trajectories generated by SplineGS, compared with trajectories from other methods. For instance, plotting the trajectories over time for a specific Gaussian blob in SplineGS and other methods like 4D-GS or D-3DGS would visually illustrate the differences in smoothness.\n\n2. The paper focuses on modeling the deformation of Gaussian centers and rotations but omits incorporating scaling into its deformation model. This omission could limit the expressiveness and applicability of SplineGS, especially when dealing with objects that undergo significant changes in size or shape during motion. The authors should address this limitation and discuss its potential impact.\n\n3. While the paper claims novelty in its use of NURBS for representing Gaussian blob trajectories, this claim requires further examination. The Spline-NeRF paper (Knodt, 2022) explores the use of Bezier splines for trajectory representation in dynamic scene reconstruction, a conceptually similar approach albeit within a NeRF framework. The authors should cite and acknowledge this related work and elaborate on the specific novelties of their NURBS-based method within the context of 3D Gaussian splatting.\n\n4. The evaluation primarily focuses on datasets with limited camera motion. To comprehensively assess SplineGS's capabilities, the evaluation should be broadened to include datasets with more pronounced camera motion, such as HyperNeRF dataset. This would provide valuable insights into SplineGS's performance under challenging conditions and enable a more direct and thorough comparison with baseline methods like 4D-GS and Deformable 3D Gaussians, which are evaluated on HyperNeRF.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "whgV0V3Cfx", "reviewer": "Reviewer_AxDz", "summary": "This paper proposed a new gaussian splatting method to reconstruct the dynamic scene. This paper investigate the usage of non-uniform rational B-splines (NURBS) in temporally smooth deformation representation. Based on the NURBS, they learned a set of trajectories and than linearly combine these trajectories to get the individual trajectories. They can achieve the competitive performance over the existing methods with much shorter training time.", "questions": "Compared with 4DGS, why this method has a significant advantage on D-NeRF dataset but has a relative small advantage on Neu3D dataset?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper combined a new representation, called non-uniform rational B-splines (NURBS), with gaussian splatting to reconstruct the dynamic scene, which is declared to be effective in representing smooth deformation.\n2. The results show that this method can achieve higher or competitive performance with existing SOTA methods in both the monocular dynamic and multi-view dynamic benchmarks.", "weaknesses": "1. To some content, this method still relies on the canonical space (the initial gaussians), but using different strategy to model the deformation. So how about the performance in the non-rigid scenes (scenes like HyperNeRF-data[1])? \n2. Actually, there are some similar strategies or representations can achieve the same effect, like Fourier (Gaussion-flow [2] adopted this presentation). The crucial point is finding a more robust representation to model the deformation or trajectory. So what's the difference or advantage of the used NURBS over other representations like Fourier?\n\n[1] HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fields.\n[2] Gaussian-Flow: 4D Reconstruction with Dynamic 3D Gaussian Particle.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bYi2tvN80n", "reviewer": "Reviewer_x185", "summary": "This paper proposes SplineGS, which reconstructs dynamic scenes using the non-uniform rational B-splines (NURBS).\nSplineGS can learn temporally smooth trajectories based on NURBS,  and learn weights of the combinations by a multi-resolution hash table and an MLP.\nSplineGS leads to high-fidelity rendering and fast training times.", "questions": "See `Weaknesses`. More,\n1. The number of Gaussians per scene, the number of representative trajectories, the training time, and the FPS of SplineGS for the D-NeRF dataset and Neu3D dataset should be provided.\n2. `LPIPS` metric also should be reported. As shown in D-3DGS, results on the “Lego” scene of the D-NeRF dataset should not be reported, nor should performance on the validation set be reported.\n3. I would like to know the performance of replacing the multi-resolution hash table and MLP with the Gaussian kernel RBF adopted by SC-GS.\n4. It would be better to provide some videos to intuitively demonstrate the performance of SplineGS.\n5. Are there any failure cases? For example, how SplineGS performs on the “vrig-broom” scene on the Hyper-NeRF dataset.\n6. Laking the results of 4DGS on the terms of `SSIM` and `Training time` for the Neu3D dataset.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- SplineGS can obtain smooth trajectories utilizing NURBS.\n- The paper is well written, and the idea is easy to understand.", "weaknesses": "- The novelty is limited. \n   1. Although SplineGS does not need any regularizers for trajectories, NURBS also can be treated as a regularizer. More importantly, this paper does not demonstrate the usefulness of the learned smooth trajectories.\n   2. To obtain the weights, SplineGS employs a multi-resolution hash table and an MLP, which can be treated as learning the weights of LBS through the neural network. Compared to other methods of obtaining LBS weights (such as Gaussian kernel RBF), what are the advantages of using neural networks?\n  3. In my opinion, the shorter training time and higher FPS are more due to L1 regularization of the opacity value in Eq. (15), which reduces the number of Gaussians.\n- Although this paper achievies state-of-the-art or at least competitive performance, hower the performance is not consistent. For example, compared with 4DGS (Yang et al., 2024a), SplineGS achieves higher PSNR and FPS on the D-NeRF dataset, while achieving similar PSNR and lower FPS on the Neu3D dataset. Experiments on more real datasets are needed to evaluate the effectiveness of SplineGS.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "E401XFjbd0", "reviewer": "Reviewer_J4YF", "summary": "This paper proposes SplineGS, a model that enables Gaussian Splatting to handle trajectories of Gaussian smoothly. SplineGS first predict position and rotation using an MLP over time and employ non-uniform rational B-splines.", "questions": "1. I believe that for the Flame Salmon scene, experiments can be conducted using only the first 300 frames, as done in 4DGS, STG, and Gaussian-Flow. Could you provide results of the scene?\n\n2. Are there trajectory results on the Neu3D dataset?\n\n3. Can you provide LPIPS results?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. SplineGS enables smooth representation of dynamic scenes without regulerizations.\n\n2. SplineGS provides competitive performance compared to the existing methods.", "weaknesses": "1. Lacks significant advantages over other models. For example, SC-GS shows better rendering quality on the D-NeRF dataset, and on the Neu3D dataset, SplineGS’ PSNR is only 0.03 dB higher than STG. It seems like SplineGS' main contribution is creating smooth trajectories for Gaussians, but only the visualization in Figure 1 supports this. I suggest adding visualizations that shows the Gaussians being learned in a smooth trajectory on various scenes.\n\n2. In line 85, the B-spline functions as a smooth regularizer, but according to line 231, rendering requires $p + M + 1$ additional points. Using the smooth regularizer should allow rendering with just one point, which seems inefficient. Therefore, a comparison of the trade-off between rendering speed and rendering quality is necessary, especially in contrast to using the regularizer.\n\n3. The appearance or disappearance of objects cannot be handled, as mentioned in the conclusion. However, in real-world scenes like Neu3D, I believe that such cases also exist, but there is a lack of analysis on this aspect. In Fig. 6, the flame streak scene in Neural 3D Video shows flames appearing and disappearing, suggesting that the method may handle disappearing objects. However, it remains unclear how SplineGS manages this. I recommend an analysis, such as visualizing the trajectories of the Gaussians.\n\n4. SplineGS performs video view synthesis, but no video results are provided.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)