Gaussian Splatting Lucas-Kanade

## Abstract

Abstract Reconstructing dynamic 3D scenes from 2D images and generating diverse views over time presents a significant challenge due to the inherent complexity and temporal dynamics involved. While recent advancements in neural implicit models and dynamic Gaussian Splatting have shown promise, limitations persist, particularly in accurately capturing the underlying geometry of highly dynamic scenes. Some approaches address this by incorporating strong semantic and geometric priors through diffusion models. However, we explore a different avenue by investigating the potential of regularizing the native warp field within the dynamic Gaussian Splatting framework. Our method is grounded on the key intuition that an accurate warp field should produce continuous space-time motions. While enforcing the motion constraints on warp fields is non-trivial, we show that we can exploit knowledge innate to the forward warp field network to derive an analytical velocity field, then time integrate for scene flows to effectively constrain both the 2D motion and 3D positions of the Gaussians. This derived Lucas-Kanade style analytical regularization enables our method to achieve superior performance in reconstructing highly dynamic scenes, even under minimal camera movement, extending the boundaries of what existing dynamic Gaussian Splatting frameworks can achieve.

## 1 Introduction

The task of reconstructing dynamic 3D scenes from monocular video presents a significant challenge, particularly when constrained by limited camera movement. This limitation leads to a scarcity of parallax and epipolar constraints, which are crucial for accurately estimating scene structures ; . Implicit neural representation methods like NeRF have shown promise in mitigating this issue by modeling dynamic scenes using an implicit radiance field ; ; . Explicit approaches like Gaussian splatting , while more efficient and often improves rendering quality, face additional hurdles. Gaussian splatting relies on an explicit representation of the scene using volumetric Gaussian functions. This necessitates a high degree of accuracy in geometric understanding to ensure proper scene depiction. Consequently, despite advancements in dynamic Gaussian splatting techniques, the requirement for diverse viewing angles to effectively constrain the placement of these Gaussians remains a limiting factor. This inherent limitation restricts their effectiveness for capturing scenes from static viewpoints or dealing with objects exhibiting rapid and complex movements.

Figure: Figure 1: Data-driven depth and optical flow supervisions produce inaccurate geometries. We instead derive the analytical warp field of Gaussians to refine geometries and motions.
Refer to caption: x1.jpg

Recent Gaussian splatting frameworks for modeling dynamic scenes commonly utilize a canonical Gaussian space as their foundation. This space is typically initialized using Structure from Motion (SfM) point clouds, with which each Gaussian is assigned attributes that describe its position, orientation, and lighting-dependent color. This canonical space acts as a reference point from which deformations are applied to represent the dynamic scene at different time steps. The process of deforming the canonical space to depict the scene at a particular time step is achieved through a warp field, often referred to as a forward warp field due to its function of warping the canonical representation forward to a specific point in time. This warp field is typically modeled using learnable Multi-Layer Perceptrons (MLPs) or offset values, as seen in frameworks like DeformableGS and Dynamic-GS . These approaches have laid the groundwork for dynamic Gaussian scene modeling.

The community has explored structural cues such as optical flow and monocular depth to help regulate the temporal transitions of Gaussians. However, enforcing structural constraints with respect to a generic forward warp field as in the DeformableGS is non-trivial. Intuitively, scene flows can be computed by forward warping of canonical scenes towards two queried frames, then evaluating the travelled trajectories. This approach is problematic since it relies on accurate structural priors, which are often unavailable for monocular casual sequences. Performing time-integration on an auxiliary network predicting the velocity field will introduce computational overhead and is theoretically unconstrained. Counter to the data-driven approach, we show that we can actually derive velocity fields directly from a generic warp field tailoring Lucas-Kanade for Gaussian Splatting, which can then be time-integrated for structural comparisons. Regulating the Gaussian motions through analytical velocity fields addresses the deficiencies of data-driven structural regularizations, as it promotes warp field learning on a continuous time domain without introducing additional learnable parameters.

In real-world captures, our approach outperforms state-of-the-art methods, including other dynamic Gaussian techniques. We demonstrate that our warp field regularization accommodates deformations in complex scenes with minimal camera movement, achieving results competitive with NeRF frameworks and enabling Gaussian splatting to model highly dynamic scenes from static cameras. To summarize, our approach offers two key advantages:
Reduced Bias: Methods relying solely on data-driven learning of warp fields can be biased towards the visible time steps, neglecting the overall movement of Gaussians. Our method, applicable to forward warp field techniques, ensures accurate Gaussian trajectories throughout the sequence.
Improved Tractability: The derived analytical solution for warp field velocities provides greater tractability compared to directly supervising the flow field as the difference in estimated deformations between consecutive frames.

## 2 Related Works

Figure: Figure 2: (a) Visualization of Gaussians’ travel distance $||\boldsymbol{p}-\boldsymbol{p}_{o}||_{2}$. For both of the scenes, the humans are stationary with dinosaur balloons moving around. Our result correctly identifies the dynamic regions, whereas the baseline model forms motions in the background and on supposedly stationary humans to compensate for photometric correctness. (b) 3D visualization of the motion trajectories. Our result shows clean trajectories from the waving balloon.
Refer to caption: x2.png

Neural Radiance Fields (NeRFs). NeRFs have demonstrated exceptional capabilities in synthesizing novel views of static scenes. Their approach uses a fully-connected deep network to represent a scene’s radiance and density. Since its introduction, NeRF has been extended to dynamic scenarios by several works ; ; ; ; ; . Some of these methods leverage rigid body motion fields ; while others incorporate translational deformation fields with temporal positional encoding ; . Despite the advantages of NeRF and its dynamic extensions, they often impose high computational demands for both training and rendering.

Gaussian Splatting.
The computational complexity of Neural Radiance Fields (NeRF) has spurred the development of alternative 3D scene representation methods. 3D Gaussian Splatting (3DGS) has emerged as a promising solution, leveraging 3D Gaussians to model scenes and offering significant advantages over NeRF in terms of rendering speed and training efficiency.

While initially focused on static scenes, recent works ; have extended 3DGS to the dynamic domain. These approaches introduce a forward warp field that maps canonical Gaussians to their corresponding spacetime locations, enabling the representation of dynamic scene content. DeformableGS employs an MLP to learn positional, rotational, and scaling offsets for each Gaussian, creating an over-parameterized warp field that captures complex spacetime relationships. 4DGaussians further refine this approach by leveraging hexplane encoding ; to connect adjacent Gaussians. DynamicGS incrementally deforms Gaussians along the tracked time frame. Despite these advancements, existing dynamic 3DGS methods struggle with highly dynamic scenes and near-static camera viewpoints. To overcome these limitations, we propose a novel approach that grounds the warp field directly on approximated scene flow.

Flow supervision.
Optical flow supervision has been widely adopted for novel view synthesis and 3D reconstruction. Dynamic NeRFs ; ; ; ; ; have explored implicit scene flow representation with an Invertible Neural Network, semi-explicit representation from time integrating a learned velocity field, and explicit analytical derivation from the deformable warp field.

Several Gaussian Splatting works have begun exploring flow supervision. Motion-aware GS applies flow supervision on the cross-dimensionally matched Gaussians from adjacent frames, without explicitly accounting for the flow contributions of Gaussians to each queried pixel. Gao et al. on avatar rendering apply flow supervision for adjacent frames after rendering the optical flow map with $\alpha$-blending . This work regularizes the warp field through data-driven flow supervision, which is only suitable for enforcing short-term geometric consistency but becomes intractable for out-of-distribution time steps ; . In the context of dynamic Gaussian Splatting, we propose to regularize the deformable warp field through analytical time integration to ensure geometric consistencies.

## 3 Preliminary

Dynamic Gaussian Splatting.
Dynamic 3D Gaussian Splatting frameworks follow a similar optimization pipeline as static Gaussian Splatting. Each Gaussian is defined by parameters for its mean $\mu$, covariance $\Sigma$, and spherical harmonics $SH$ color coefficients. To project Gaussians from 3D to 2D, we calculate the view space covariance matrix as follows:

$$ $\Sigma^{\prime}=\boldsymbol{JW\Sigma W^{T}J^{T}},$ (1) $$

where $\boldsymbol{J}$ is the Jacobian of the projective transformation and $\boldsymbol{W}$ is the viewing transform. To represent a scene’s radiance, the covariance $\Sigma$ is decomposed into scaling matrices $\boldsymbol{S}$ and rotation matrices $\boldsymbol{R}$ as:
$\Sigma=\boldsymbol{RSS^{T}R^{T}}$.
This decomposition supports differential optimization for dynamic Gaussian Splatting. The color $C$ is derived using a NeRF-like volumetric rendering equation:

$$ $\mathbf{C}=\sum_{i=1}^{N}T_{i}\left(1-\exp(-\sigma_{i}\delta_{i})\right) \mathbf{c}{i}\quad\text{with}\quad T_{i}=\exp\left(-\sum_{j=1}^{i-1}\sigma_{j} \delta_{j}\right).$ (2) $$

Adaptive density control manages the density of 3D Gaussians, allowing dynamic variation over iterations. For more details, refer to .

Dynamic Gaussian Splatting frameworks maintain a canonical space and learn warp fields $\mathcal{F}$ that transform canonical Gaussians $\mathcal{G}$ to the deformed space at time $t$:

$$ $\delta\mathcal{G}=\mathcal{F}_{\theta}(\mathcal{G},t).$ (3) $$

Following deformation, a novel-view image $\hat{I}$ is rendered using differential rasterization with a view matrix $M=[R,T]$ and target time $t$. While MLP-based deformation mapping often suffers from overfitting to training views, leading to degraded novel-view reconstruction, we address this by enforcing analytical constraints on the warp field. This ensures Gaussian motions adhere to the expected scene flow field, mitigating the issues of overfitting and SfM-initialized point cloud limitations.

Warp Field with twist increments. Let $P_{\tau}\in\mathbb{R}^{N_{1}\times 3}$ and $P_{s}\in\mathbb{R}^{N_{2}\times 3}$ be the target and source point clouds respectively, where $N$ denotes the number of points respectively. The warp field that defines the rigid transformation that aligns the source point cloud to the template point cloud in the special Euclidean group can be expressed as $\mathcal{W}(\xi)=exp(\sum^{6}_{p=1}\boldsymbol{\xi_{p}}\boldsymbol{T_{p}})\in
SE
(3)$. The twist parameters in exponential mapping are the $\boldsymbol{\xi}\in\mathbb{R}^{6}$, and the generator matrices are $\boldsymbol{T}$. The warp field can be optimized by minimizing the differences between the source and the target point clouds as:

$$ $argmin_{\boldsymbol{\xi}}||\phi(\mathcal{W}(\boldsymbol{\xi})\cdot\boldsymbol{ P}_{\mathcal{S}})-\phi(\boldsymbol{P_{\tau}})||^{2}_{2},$ (4) $$

where $\phi:\mathbb{R}^{N\times 3}\rightarrow\mathbb{R}^{K}$ is an encoding function that either explicitly extracts geometric features such as edges and normals, or implicitly encodes the point clouds into feature vectors. The $(\cdot)$ denotes the warp field transformation.
The optimization of the warp field incrementally learns the twist parameter as $\Delta\boldsymbol{\xi}$ that best aligns the encoded source point cloud to the target point cloud.

This optimization can be solved iteratively given the twist parameter Jacobian matrix $\boldsymbol{J}$ that can be further decomposed into a product of warp field gradient and the encoding functional gradient as:

$$ $\boldsymbol{J}=\frac{(\mathcal{W}^{-1}(\boldsymbol{\xi})\cdot\boldsymbol{P}_{ \tau})}{\partial\boldsymbol{\xi}^{T}}\cdot\frac{\partial\phi(\mathcal{W}^{-1}( \boldsymbol{\xi})\cdot\boldsymbol{P}_{\tau})}{\partial(\mathcal{W}^{-1}( \boldsymbol{\xi})\cdot\boldsymbol{P}_{\tau})^{T}}.$ (5) $$

The transformations are applied iteratively until convergence.

## 4 Gaussian Splatting warp field regularization by scene flow

We study the problem of fitting deformable Gaussian Splatting from a monocular video sequence. Using structure-from-motion methods, we initialize the canonical Gaussians and camera parameters. For a given 3D Gaussian set $\mathcal{G}$ at time $t$, the warp field deforms the Gaussians from canonical space to their positions at time $t$. We seek the optimal network parameters $\theta$ for the forward warp field $\mathcal{W}_{c\rightarrow t}$ so that the scene flow mapping $\mathcal{T}_{t\rightarrow t+1}$ and the rasterized image $\hat{\mathcal{I}_{t}}$ match the expected flow field and input image $\mathcal{I}_{t}$:

$$ $argmin_{\theta}\Sigma_{t}\left[\Sigma_{t^{\prime}\in\{t\pm\Delta t\}}||(\hat{ \mathcal{T}}_{t\rightarrow t+1}-\mathcal{T}_{t\rightarrow t+1})||_{p}+||\hat{ \mathcal{I}_{t}}-\mathcal{I}_{t}||^{2}_{2}\right].$ (6) $$

Figure: Figure 3: Analytical scene flow from warp field. With canonical Gaussians $\mathcal{G}_{c}$, we transform them forward in time to $\mathcal{G}_{t}$, then perform time integration from warp field velocities $v(\mathcal{G};t)$ to derive $\mathcal{G}_{t+\Delta t}$. The Gaussian offsets $\mathcal{G}_{t+\Delta t}-\mathcal{G}_{t}$ are compared to reference scene flow.
Refer to caption: x3.jpg

The scene flow can be further decomposed into optical flow and depth estimations corresponding to the in-plane and out-of-plane motions. Suppose a point $\boldsymbol{p}=\boldsymbol{p}(t)$ moves in the scene. The projected location of this point from a static viewpoint camera $i$ is $\boldsymbol{u}_{i}=\boldsymbol{u}_{i}(t)$, with $\boldsymbol{u}_{i}$’s change rate with respect to time as
$\frac{d\boldsymbol{u}_{i}}{dt}=\frac{\partial\boldsymbol{u}_{i}}{\partial
\boldsymbol{p}}\frac{d\boldsymbol{p}}{dt}.$
Although the mapping from image coordinates to the scene coordinates is not unique, the relationship between $\boldsymbol{p}$ and $\boldsymbol{u_{i}}$ can be loosely expressed with dependencies on time as $\boldsymbol{p}=\boldsymbol{p}(\boldsymbol{u}_{i}(t);t)$ . Now differentiating this mapping expression to time gives:

$$ $\frac{d\boldsymbol{p}}{dt}=\frac{\partial\boldsymbol{p}}{\partial\boldsymbol{u }_{i}}\frac{d\boldsymbol{u}_{i}}{dt}+\frac{d\boldsymbol{p}}{dt}\Bigr{|}_{ \boldsymbol{u}_{i}}.$ (7) $$

This describes the motion of a point in 3D space using the scene flow projection on the image plane and the out-of-plane scene flow component. The in-plane component, $\frac{d\boldsymbol{u}_{i}}{dt}$, is obtained by projecting the instantaneous motion on the image plane, and the out-of-plane term represents the motion along $\boldsymbol{u}_{i}$’s corresponding ray. This decouples regularization on the optical flow field and the depth field from the estimated scene flow. The scene flow regularization then becomes:

$$ $\mathcal{L}_{scene}=||(\hat{\mathcal{T}}_{t\rightarrow t+1}-\mathcal{T}_{t \rightarrow t+1})||_{p}=\alpha\mathcal{L}_{o_{t\rightarrow t+1}}+\beta\mathcal {L}_{d_{t\rightarrow t+1}}$ (8) $$

Decoupling the motions alleviates constraints on the supervision dataset and adds flexibility to the optimization. The regularization pipeline is visualized in Fig. 3. Implementation details are provided in Appendix A.1.

### 4.1 Forward warp field velocity

With the aforementioned problem statement, we show that the velocity field can be derived from the forward warp field. Restating the objective, we seek to find a warp field $\mathcal{W}$ that transforms the canonical Gaussians $\mathcal{G}$ forward in time such that it minimizes the appearance loss and scene flow loss at the target time step. To do this, we derive the velocity of the forward warp field as analogous to the twist Jacobians. The velocity field $v(\mathcal{G};t)$ expresses the changes in Gaussian parameters in the world coordinate at a given time $t$. Gaussian Splatting maintains an explicit canonical Gaussian space so intuitively the velocities can be computed from its forward warp field $\mathcal{W}_{c\rightarrow}(\mathcal{G},t)$ as:

$$ $v(\mathcal{G};t)=\frac{\partial\mathcal{W}_{c\rightarrow}(\mathcal{G},t)}{ \partial t}\Big{|}_{(\mathcal{W}_{c\leftarrow}(\mathcal{G},t);t)}.$ (9) $$

The explicit expression of 3D Gaussians ensures that the scene flow mapping between the adjacent frames is bijective. In other words, the resulting correspondences among the 3D points in each frame are cyclic consistent since they are deformed from the same canonical Gaussian. This allows us to analytically compute the velocity field $v(\mathcal{G};t)$ for each Gaussian in the canonical space.

The velocity field $v(\mathcal{G};t)$ requires an analytical representation of the warp field’s gradient concerning time and warp parameters. Although the direct analytical solution is costly to compute, the chain rule can be applied to decompose the compute into two partial terms: the feature gradient of the warp field with respect to the corresponding Gaussian parameter, and the analytical warp Jacobian ; :

$$ $v(\mathcal{G};t)=\boldsymbol{J}^{-1}_{\mathcal{G}}\frac{\partial\mathcal{W}_{c \rightarrow}(\mathcal{G};t)}{\partial t}=\frac{\partial\mathcal{W}_{c \rightarrow}(\mathcal{G};t)}{\partial\mathcal{G}}\frac{\partial\mathcal{W}_{c \rightarrow}(\mathcal{G};t)}{\partial t}.$ (10) $$

The warp field feature gradient $\frac{\partial\mathcal{W}_{c\rightarrow}(\mathcal{G};t)}{\partial t}$ describes how each output Gaussian parameter offsets changes with respect to the input canonical Gaussian parameters. The warp Jacobian $\frac{\partial\mathcal{W}_{c\rightarrow}(\mathcal{G};t)}{\partial\mathcal{G}}$ describes how changes in the warp field output affect the transformation on the Gaussians.

We note that the expression holds for any Gaussian parameter that is an input to the deformation network $\mathcal{W}_{c\rightarrow}$ as long as the respective input and output dimensions are the same. However, in practice, the warp Jacobian can experience numerical instabilities with $det(\boldsymbol{J}_{\mathcal{G}})<\epsilon$, especially when the canonical Gaussian scene has not stabilized. However, discarding unstable motions also affects the downstream densification and pruning process. We instead replace $\boldsymbol{J}^{-1}_{\mathcal{G}}$ with Moore–Penrose pseudoinverse operation $\boldsymbol{J}^{+}_{\mathcal{G}}$. The scaling terms and spherical harmonics of Gaussians have minimal changes between adjacent frames, we thus choose to exclude them from the compute for efficiency. Overall, we compute the velocity fields for Gaussians’ displacement and rotation as:

$$ $v(\boldsymbol{p};t)=\boldsymbol{J}^{+}_{\boldsymbol{p}}\frac{\partial\mathcal{ W}_{c\rightarrow}(\boldsymbol{p};t)}{\partial t};v(\boldsymbol{q};t)= \boldsymbol{J}^{+}_{\boldsymbol{q}}\frac{\partial\mathcal{W}_{c\rightarrow}( \boldsymbol{q};t)}{\partial t},$ (11) $$

where $v(\boldsymbol{p};t):\mathbb{R}^{3}\times\mathbb{R}\rightarrow\mathbb{R}^{3}$ and $v(\boldsymbol{q};t):\mathbb{R}^{4}\times\mathbb{R}\rightarrow\mathbb{R}^{4}$ respectively describes the positional and rotational velocity of a Gaussian in the world coordinate.

### 4.2 Scene field from time integration

Time integration on velocity field. Given the velocity field from equation 10, and 3D Gaussians $\mathcal{G}(t)$ observed at time step $t$, we can apply time integration using a Runge-Kutta numerical solver on the velocity field to obtain the offset parameters for Gaussians at time $(t+\Delta t)$:

$$ $\mathcal{G}(t+\Delta t)=\mathcal{G}(t)+\int_{t}^{t+\Delta t}v_{\theta}( \mathcal{G}(t);t)dt.$ (12) $$

We then train the warp field network to minimize the difference between the predicted scene flow derived from the analytical velocity field, and the ground truth scene flow decomposed into optical flow and depth describing the in-plane and out-of-plane Gaussian motions.

Optical flow and depth rendering from Gaussians.
For each Gaussian at time $t$, we compute each of the corresponding positions and rotations at $(t+\Delta t)$. For rendering the optical flow and depth map, we consider the full freedom of each Gaussian motion.

For optical flow rendering, we want to calculate the motion’s influence on pixel shifts from all Gaussians in the world coordinate. In the original 3D Gaussian Splatting, a pixel’s color is the weighted sum of the projected 2D Gaussian’s radiance contribution. Analogous to this formulation, the optical flow at a queried pixel is the weighted sum of the 2D Gaussians’ contribution to the pixel shift:

$$ $\hat{flow}^{G}_{t,t+\Delta t}=\Sigma_{i\in N}flow_{i}\alpha_{i}\prod_{j=1}^{i- 1}(1-\alpha_{j})={\sum}_{i=1}w_{i}[\Sigma_{i,t_{2}}\Sigma^{-1}_{i,t_{1}}(x_{t_ {1}}-\mu_{i,t_{1}})+\mu_{i,t_{2}}-x_{t_{1}}],$ (13) $$

where $flow_{i}$ denotes the optical flow of Gaussian $\mathcal{G}_{i}$, and $w_{i}=\frac{T_{i}\alpha_{i}}{\Sigma_{i}T_{i}\alpha_{i}}$ denotes the weighing factor of each Gaussian from $\alpha$-blending.

Similarly, the per-pixel z-depth estimates $\hat{d}$ can be rendered from the discrete volume rendering approximation accounting for the per-Gaussian contributions:

$$ $\hat{D}^{G}_{t,t+\Delta t}=\Sigma_{i\in N}d_{i}\alpha_{i}\prod_{j=1}^{i-1}(1- \alpha_{j}),$ (14) $$

where $d_{i}$ denotes the z-depth value of Gaussian $\mathcal{G}_{i}$ from the viewing space. We normalize the optical flow and depth estimates for numerical stability. The optical flow and depth maps are rasterized simultaneously in the same forward pass with color.

## 5 Experiments

In this section, we first provide the implementation details of the proposed warp field regulation and then validate our proposed method on four dynamic scene datasets captured with different levels of camera movements. We showcase the quantitative and qualitative results of the Dynamic Scenes dataset , Planoptic Video dataset , HyperNeRF dataset , and DAVIS dataset . Our method achieves state-of-the-art results on both static and dynamic cameras.

Figure: Figure 4: Qualitative comparisons on the Dynamic Scenes dataset. Compared to the baseline method, our approach can achieve superior rendering quality on real datasets with lower EMFs.
Refer to caption: x4.jpg

### 5.1 Implementation details

We implement the warp field using PyTorch , leveraging its Autodiff library for gradient and Jacobian computations. The framework is optimized with Adam as with 3DGS on a NVIDIA A100 Tensor Core . We use the Torchdiffeq library for numerical integration. Modified from DeformableGS, the inputs to the warp field include 3D Gaussian positions ($\boldsymbol{p}$), time ($t$), and 3D Gaussian rotations ($\boldsymbol{q}$) in quaternions. Velocity derivations are enabled by removing the stop gradient operation from the network inputs. Positional encoding is applied to extend inputs’ frequency band . We refer readers to Appendix A.2 for more implementation details.

### 5.2 Evaluation

Here we analyze the performance of our method quantitatively and qualitatively. We aim to study if the analytical scene flow regularization on the warp field helps disambiguate the dynamic scene geometry and promote the reconstruction of low EMF scenes.

Benchmarked datasets. We evaluate our method using real-world monocular datasets with varying camera motion. These include Dynamic Scene , captured by a stationary multi-view 8 camera rig with significant scene motion; Plenoptic Video , captured using a static rig with 21 GoPro cameras ; Hypernerf , which captures objects with moving topologies from a moving camera, suitable for quasi-static reconstruction; and sequences from DAVIS 2017 dataset containing near-static monocular videos.

Quantitative baseline comparisons.
We evaluate our method’s novel-view synthesis capabilities on three datasets: Plenoptic Videos, Hypernerf, and Dynamic Scenes, ordered by decreasing Effective Multi-view Factor (EMF) . These datasets present increasing challenges, with Dynamic Scenes posing the greatest difficulty due to its low EMF and unreliable object motion. We assess visual quality using LPIPS , SSIM , and PSNR , comparing our method to static Gaussian Splatting , DeformableGS , and 4DGaussians , as shown in Table 4 and Table 2. Our method consistently outperforms these baselines on scenes with higher EMFs (Plenoptic Videos and Hypernerf).

**Table 1: Quantitative evaluation of novel view synthesis on the Dynamic Scenes dataset. See Sec. 5.2 for descriptions of the baselines.**
| method | Playground | Balloon1 | Balloon2 | Umbrella |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | PSNR $\uparrow$ | SSIM $\uparrow$ | LPIPS $\downarrow$ | PSNR $\uparrow$ | SSIM $\uparrow$ | LPIPS $\downarrow$ | PSNR $\uparrow$ | SSIM $\uparrow$ | LPIPS $\downarrow$ | PSNR $\uparrow$ | SSIM $\uparrow$ | LPIPS $\downarrow$ |
| NSFF | 24.69 | 0.889 | 0.065 | 24.36 | 0.891 | 0.061 | 30.59 | 0.953 | 0.030 | 24.40 | 0.847 | 0.088 |
| NR-NeRF | 14.16 | 0.337 | 0.363 | 15.98 | 0.444 | 0.277 | 20.49 | 0.731 | 0.348 | 20.20 | 0.526 | 0.315 |
| Nerfies | 22.18 | 0.802 | 0.133 | 23.36 | 0.852 | 0.102 | 24.91 | 0.864 | 0.089 | 24.29 | 0.803 | 0.169 |
| (w flow) | 22.39 | 0.812 | 0.109 | 24.36 | 0.865 | 0.107 | 25.82 | 0.899 | 0.081 | 24.25 | 0.813 | 0.123 |
| Flow-sup. NeRF | 16.70 | 0.597 | 0.168 | 19.53 | 0.654 | 0.175 | 20.13 | 0.719 | 0.113 | 18.00 | 0.597 | 0.148 |
| Deformable GS | 24.82 | 0.646 | 0.343 | 22.40 | 0.833 | 0.137 | 24.19 | 0.818 | 0.153 | 22.35 | 0.711 | 0.186 |
| 4DGaussians | 21.39 | 0.776 | 0.204 | 24.48 | 0.849 | 0.144 | 24.72 | 0.801 | 0.219 | 21.29 | 0.560 | 0.332 |
| Ours | 26.34 | 0.756 | 0.184 | 26.35 | 0.848 | 0.133 | 25.89 | 0.911 | 0.151 | 23.02 | 0.746 | 0.176 |

To further assess our method’s performance, we compare its novel-view synthesis results with NSFF , NR-NeRF , Nerfies , and Flow-supervised NeRF on the Dynamic Scenes dataset, see Table 1. These comparisons highlight the effectiveness of our analytical scene flow approach compared to other flow-guided rendering frameworks. Notably, our method outperforms NSFF, which uses a similar supervision style but a different scene flow derivation. The comparison with Flow-supervised NeRF, which also uses analytical scene flow derived from an implicit radiance field, serves as an ablation study, demonstrating the effectiveness of our derived scene flow on an explicit scene representation. Finally, we support the aforementioned decomposed structural supervision as in Table 4. More discussions on alternative setups are provided in Appendix Sec. A.3.

**Table 2: Quantitative evaluation of novel view synthesis on the Plenoptic Videos dataset. See Sec 5.2 for an analysis of the performance.**
| method | cook spinach | cut roasted beef | sear steak | Mean |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | PSNR $\uparrow$ | SSIM $\uparrow$ | LPIPS $\downarrow$ | PSNR $\uparrow$ | SSIM $\uparrow$ | LPIPS $\downarrow$ | PSNR $\uparrow$ | SSIM $\uparrow$ | LPIPS $\downarrow$ | PSNR $\uparrow$ | SSIM $\uparrow$ | LPIPS $\downarrow$ |
| Deformable GS | 32.97 | 0.947 | 0.087 | 30.72 | 0.941 | 0.090 | 33.68 | 0.955 | 0.079 | 32.46 | 0.948 | 0.085 |
| 4DGaussians | 31.98 | 0.938 | 0.056 | 31.56 | 0.939 | 0.062 | 31.20 | 0.949 | 0.045 | 31.58 | 0.942 | 0.055 |
| Ours | 33.91 | 0.951 | 0.064 | 32.40 | 0.957 | 0.084 | 34.02 | 0.963 | 0.057 | 33.44 | 0.954 | 0.068 |

**Table 3: Quantitative evaluation on the Hypernerf dataset. See Sec. 5.2 for detailed explanations.**
| method | PSNR $\uparrow$ | SSIM $\uparrow$ | LPIPS $\downarrow$ |
| --- | --- | --- | --- |
| 3DGS | 20.84 | 0.70 | 0.45 |
| Deformable GS | 26.47 | 0.79 | 0.29 |
| 4DGaussians | 26.98 | 0.78 | 0.31 |
| Ours | 27.38 | 0.81 | 0.26 |

Trajectories from scene flow fields. We visualize the scene flow trajectories from Dynamic Scenes Dataset sequences to assess the quality and smoothness of the derived flow field. The Gaussians are subsampled from the dynamic regions of the canonical Gaussian space, and then time integrated to produce their respective displacements across time, visualized as colored trajectories in Fig. 2. We note that the trajectories are smooth and follow the expected motions of the dynamic objects. The visualized trajectories of the sequences demonstrate the framework’s potential to be extended for tracking in dynamic 3D Gaussian scenes. More trajectories on the DAVIS dataset will be shown in the appendix.

Geometric consistency. Fig. 4 compares our method with DeformableGS, which learns Gaussians’ time dependency without scene flow regularization. While DeformableGS renders visually accurate novel views, the underlying scene structures are inaccurate, leading to blurry dynamic objects and artifacts in depth maps. Our method, leveraging analytical scene flow regularization, achieves more accurate geometries, reflected in both visual and quantitative results (Table 1). This also enables accurate Gaussian motion tracking (Fig. 2).
To explore alternative design choices, we compare with direct flow and depth supervision on DeformableGS (Fig. 1). These comparisons indicate that our approach, which enforces regularization on the expected scene flow over a continuous time frame, results in more plausible scene structures compared to discrete time step supervision.

Figure: Figure 5: Our method struggles with insufficient point cloud initialization due to reliance on non-rigid warping of scene geometry. In the ’skating’ scene, this results in unstable geometry at certain angles. Inaccurate camera calibrations also degrade our method, as shown in the ’toby-sit’ sequence, where miscalibration distorts the scene geometry.
Refer to caption: x5.jpg

Static and dynamic motion separation. Our approach formulates deformations within a canonical Gaussian space. To assess the fidelity of our method in separating static and dynamic regions, we visualize the travel distance of each Gaussian from its canonical projection to a queried viewing camera in Fig. 2. The plots are color-coded by the absolute travel distance, with yellow indicating larger distances and purple indicating smaller distances.

In the Playground scene featuring human-object interactions, the visualized travel distance is greatest at the far-side boundaries of the moving object as it recedes from the human. The scene’s background is correctly rendered as static with minimal motion. This visualization demonstrates the effectiveness of our method in accurately identifying dynamic regions. Moreover, these results suggest promising future directions for optimizing dynamic scene rendering by filtering out static regions from the optimization process, potentially reducing computational costs.

## 6 Limitations

Sensitivity to SfM Initialization: Our approach is sensitive to the quality of the Structure-from-Motion (SfM) Gaussian space initialization. Accurate initialization is crucial for 3D Gaussian Splatting, particularly without other scene priors. As shown in Fig. 5.a, reconstruction quality deteriorates when dynamic regions have insufficient initialized points.

Camera Parameter Sensitivity: Our method is sensitive to inaccuracies in bundle-adjusted camera intrinsic and extrinsic parameters. Errors in these parameters lead to inaccurate rasterization projections, causing noisy geometry or failed reconstruction, as shown in Fig. 5.b.

Computational Complexity: The pseudo-inverse operation for velocity field computation requires $(N\times d)$ square matrix decompositions, where $N$ is the number of Gaussians, and $d$ is the number of Gaussian parameters. Potential mitigations include optimized implementations or pre-filtering to remove static Gaussians.

## 7 Conclusion

This paper presents a method for incorporating scene flow regularization into deformable Gaussian Splatting. We derive an analytical scene flow representation, drawing upon the theoretical foundations of rigid transformation warp fields in point cloud registration. Our approach significantly enhances the structural fidelity of the underlying dynamic Gaussian geometry, enabling reconstructions of scenes with rapid motions.

Comparison with other Deformable Gaussian Splatting variants demonstrates that regularizing on the warp-field derived scene flow produces more accurate dynamic object geometries and improved motion separation due to its continuous-time regularization. Additionally, comparison with flow-supervised NeRFs reveals the advantages of using Gaussians’ explicit scene representations.

While our method exhibits limitations as discussed in the previous section, the accurate geometries learned from dynamic scenes open up promising possibilities for future applications, including 3D tracking from casual hand-held device captures and in-the-wild dynamic scene rendering.

## Appendix A Appendix / supplemental material

### A.1 Scene Flow Regularization Formulation

Recall that in addition to the photometric regularization as detailed in the original 3D Gaussian Splatting , $\mathcal{L}_{scene}$ is used as the structural regularization for the derived scene flow consisting of weighted depth and optical flow terms $\mathcal{L}_{scene}=\alpha\mathcal{L}_{o_{t\rightarrow t+1}}+\beta\mathcal{L}_
{d_{t\rightarrow t+1}}$.

The optical flow term minimizes the absolute error between the rendered optical flow as computed from accounting for all Gaussians’ 2D projected motion contributions to each rendered pixel, and the reference flow from RAFT . In particular, the optical flow term is expressed as:

$$ $\mathcal{L}_{o_{t\rightarrow t+1}}=\frac{1}{n}\sum_{j=1}^{n}|o_{j}-\hat{o}_{j}|,$ (15) $$

where $j$ denotes the queried pixel from a rasterized image containing $n$ total pixels.

As noted in SparseNeRF , monocular depth maps are scale-ambiguous thus directly using monocular depth priors with $p$-norm regularizations hinders the spatial coherency. We apply a local depth ranking loss on the rendered depth maps. We transfer the depth ranking knowledge from monocular depth $d_{mono}$ to rendered $d_{r}$ with:

$$ $\mathcal{L}_{d_{t\rightarrow t+1}}=\sum_{d^{k_{1}}_{mono}<d^{k_{2}}_{mono}}max (d^{k_{1}}_{r}-d^{k_{2}}_{r}+m,0),$ (16) $$

where $m$ is a tunable small margin defining the permissible depth ranking errors, and $k_{1},k_{2}$ are the indices of the randomly queried pixels. If the depth ranking of the monocular depth pixels and the rendered pixels are not consistent, then it entails inaccuracies in the warped Gaussians at time $(t+\Delta t)$, penalizing the learned warp field.

The coefficients $\alpha$ and $\beta$ reweight the magnitude of loss terms to compensate for regularizing with monocular depth cues. In practice, we notice that setting the two coefficients with similar magnitude yields satisfactory results for scenes initialized with dense point clouds. Whereas with insufficient SfM initialization, the motion coefficient $\alpha$ needs to be weighed down in earlier iterations to allow the canonical scene to stabilize.

### A.2 Additional implementation Details

Scene initialization.
We define the initialization stage as the first $N$ iterations without deploying scene flow regularizations. To warm up the optimization, during the initialization stage, we only compute the photometric losses with the warp field network unfrozen. Note that we discarded the static scene initialization stage typically adopted by other dynamic Gaussian splatting frameworks as it provided incremental benefits.

The length of the warm-up period is defined based on the quality of the SfM scene. For scenes with rich point clouds to start with, we employ $3000$ iterations for initialization to refine the canonical structures. Some highly dynamic scenes contain only a few points or no points in the dynamic regions. Since our method relies on analytical warping from the canonical scene and needs Gaussians at the dynamic regions to start with, we extend the warm-up period to $5000$-$10,000$ to recover as many Gaussians as possible at the dynamic regions.

Motion masking.
The initialized Gaussians generally carry inaccurate motions to compensate for photometric accuracies, regardless of whether the Gaussians are located at the presumably static or dynamic regions. Thus we first randomly sample pixels from the rendering to correct motions of all Gaussians equally across the scene. Since static regions are usually larger than dynamic regions and are easier to converge, this allows us to disambiguate the static and dynamic regions.

After ensuring that most Gaussians in the static regions have been sufficiently constrained, we apply a large motion mask to reconstruct small and fast-moving objects. The motion mask is defined similarly as prior work as a binary segmentation mask at pixels with normalized motions larger than $0.1$. This additional motion masking step allows us to focus on rapid motions when most of the scene motions have converged.

Training details. The hyperparameters for learning the Gaussian parameters are kept consistent as the original 3D Gaussian Splatting . For comparable analysis on the analytical warp field regularization, we follow setups in the baselines ; . The learning rate of the warp field network is empirically set to decay from 8$e$-4 to 1.6$e$-6, and Adam’s $\beta$ range is set to $(0.9,0.999)$. As with data preparation, we used COLMAP and Reality Capture to estimate camera intrinsics and extrinsics, which are kept fixed during optimization. We use Marigold and RAFT for depth and flow maps.

### A.3 Further Discussions on Motion Learning

Geometry enhances novel view reconstruction quality.
Fig. 6 presents novel view synthesis results and learned geometries from our method alongside two baseline models. This comparison highlights the impact of accurate underlying geometries on synthesized novel views. All three models converge on the training views, so we focus on the results for unseen viewpoints.

Figure: Figure 6: Visualization of Gaussian trajectories and comparison of novel view synthesis results with learned geometries.
Refer to caption: x6.jpg

4DGaussians learns the warp field without incorporating motion cues. This approach leads to a noisy underlying geometry with artifacts around specular areas, likely introduced to compensate for photometric inaccuracies. Consequently, its synthesized dinosaur exhibits blurriness around the edges.

Another baseline predicts velocities directly instead of relying on analytical derivation as in Du et al. . We adopt the velocity field model from Du et al. and perform time integration to derive scene flow. However, this data-driven velocity field tends to produce an "averaged" geometry, resulting in blurriness around the dinosaur’s head area.

In contrast, our method generates the most crisp depth map and the highest quality synthesized result. This improvement stems from our ability to accurately learn dynamic geometries. The left-hand side of the figure demonstrates that even with monocular sequences, our method effectively separates the foreground, background, and carried object. Our method serves as a general tool for enhancing dynamic Gaussian splatting frameworks without requiring additional network modules.

Motion thresholding visualizations.
Fig. 7 presents a comparison of motion thresholds at different travel distances and our learned trajectories. The first row visualizes the 3D trajectories of Gaussians, color-coded by z-depth magnitude and in learned color. This visualization clearly demonstrates the accurate motions of a human waving a balloon.
The second row displays the motion plot generated by our method, while the third row showcases the results from the DeformableGS baseline model . The baseline model exhibits a significant error, moving the stationary human instead of the balloon. Notably, the side of the balloon furthest from the human experiences the least movement, while the human’s torso exhibits the most displacement. This outcome directly contradicts expectations.
In contrast, our method accurately captures the motions, correctly ranking their magnitudes. This highlights the effectiveness of our approach in distinguishing and representing the dynamic elements within the scene.

Figure: Figure 7: 3D trajectories and thresholded motion plots. The first row of motion plots are results from our method and the second row are results from baseline model DeformableGS.
Refer to caption: x7.jpg

Sensitivities to SfM initialization continued.
As discussed in the limitations section, our method relies on analytical warping from canonical space, making it sensitive to Structure-from-Motion (SfM) initialization, particularly when dynamic regions lack sufficient point correspondences. Fig. 8 illustrates this sensitivity using the "train" scene from the DAVIS dataset .
The left side of the figure shows the SfM-initialized mesh, where the toy train traveling on the circular track is not captured. The top right figure displays the learned trajectories from the baseline model, exhibiting irregular Gaussian movements within the dynamic area.

The bottom right figure presents the learned trajectories from our method. While our method captures the expected trajectory of the moving train, it struggles to reconstruct dense Gaussians for the train due to an insufficient number of initialized Gaussians in this region. Additionally, the Gaussians are not accurately placed at the correct depth, as the monocular depth ranking loss only enforces the correct depth ranking order. Employing alternative depth regularization techniques could improve the learned depth in absolute scale.

Figure: Figure 8: SfM initialized mesh and the reconstructed toy train trajectories. The right top subfigure shows trajectories from the baseline model, and the right bottom subfigure shows trajectories from our model.
Refer to caption: x8.png

More results on real scenes.
More qualitative results on dynamic scenes can be found in Fig. 9.

Figure: Figure 9: Qualitative comparisons on Dynamic Scenes Dataset
Refer to caption: x9.jpg

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: Q1: Given the analytical form of the derivation, it would be interesting to study classic optical flow problems for Gaussian flow: What is the aperture problem in the Gaussian case. What happens when the Gaussian covariance is rank deficient? What scene structures would cause an ambiguous flow? 

Q2: What would happen when the spherical harmonic coefficients would change? 

Q3: What happens with a local deformation like scale change or shear?

Q4: What exactly is the difficulty of including second-order terms?

Q5: How is the method's sensitivity to the choice of canonical frame? DGS methods have been proposed that do not require a choice of canonical frame.
soundness: 4
strengths: + Deriving analytical solutions is elegant and makes an approach interpretable. 

+ The analytical derivation enables a sound application of the flow term in the Dynamic Gaussian case. 

+ The paper is a pleasure to read.

+ A direct flow regularization makes a network overfit on viewpoints.

+ The idea of flow supervision in Gaussian splatting increases performance.
summary: This is a paper about the fundamental question of how to derive the velocity of Gaussian means in Gaussian splatting from the finite forward warp transformation. 
It assumes that these transformations are small and that one can live with a first-order Taylor expansion. 

This involves the derivative of the Gaussian parameters after the motion with respect to the Gaussians in the canonical frame and the warp Jacobian.
Then, assuming that scaling and directional reflectivity do not change, one can derive the Jacobian with respect to the mean and the Jacobian with respect to the orientation.
weaknesses: While it is nice for a paper to be self-contained, everything 
in this paper until eq. (10) is identical to the "Flow supervision for Deformable NeRF" paper. Eq. (12) is the same as Eq. (7) in the latter. Optical flow rendering is same as in GaussianFlow. The reader is wondering, what exactly is the contribution of this paper when the main ingredients are in the Flow Supervision and the GaussianFlow paper. It really hurts to say that because any reader would enjoy such elegant analytical derivations. Please explain in your response.

---

confidence: 3
contribution: 3
initial_rating: 8
presentation: 2
questions: Three limitations are mentioned in the paper. 
I would like to add:
- "Sensivity to noise". What is the behavior under noise? is that possible to further reduce the effect of noise?
- Computational complexity and related aspects are not mentioned at all. What is the advantage of the approach? (compared to pure learning approaches) 
- The paper mentions a motion masking step. How is the threshold defined? Do theses parameters generalize?
soundness: 3
strengths: The strength of the paper is to propose a direct analytical solution to the problem of the estimation of the warp field. The solution is mathematically transparent and provides superior results compared to a purely data-driven approach. 
It is a fully new solution which is very relevant for dynamic gaussian estimation and related applications. It is strongly inspired by Lucas-Kanade point tracking and has the same level of generality.
summary: The paper addressed the problem of Gaussian Splatting estimation in dynamics scene and with small movements of the camera or the objects. The authors propose an analytical solution to the deformable warp field used to regularize the estimation instead of a learnable  field or offset-values as applied in the state of the art approaches. 
The key contribution is to provide a detailed derivation and stable estimation, basically a a Gaussian Splatting warp field regularization by scene flow. The approach is then evaluated on various datasets and show improvements compared to the existing solutions.
weaknesses: - A reference implementation should be provided since the approach is very relevant to the community and may include many small details, which have their importance and not detailed in the paper.
- The method relies on external algorithms to provide depth and optical flow information, which are used to supervise the scene flow regularization. The appendix discusses experiments with different depth and flow estimation methods. However, this introduces an additional source of potential error and highlights the importance of using high-quality depth and flow estimation techniques.
- EMF is not explained when it was mentioned but later in the text.
- One reference is missing

---

confidence: 4
contribution: 2
initial_rating: 6
presentation: 2
questions: - What exactly is the network architecture of the warp field?
- Compared to fully MLP-based Deformable 3DGS, grid-based warp fields such as 4DGS should struggle less with incorrect motion correlations triggered by a single global MLP representation. How much improvement does the proposed method offer for 4DGS-type warp fields?
soundness: 2
strengths: - The analytical formulation of the vector field is interesting.
- More tractable than fully MLP-based Deformable 3DGS or other 2D prior-based supervision approaches.
- Competitive benchmark performance.
- The paper is clearly written and easy to follow.
summary: This paper proposes a novel dynamic Gaussian splatting approach based on an analytical velocity field formulation. Similar to Lucas-Kanade, the method linearizes the warp function at specific time intervals and approximates it with a first-order expansion. This approach effectively regularizes scene flow, resulting in improved dynamic reconstruction performance on benchmark datasets.
weaknesses: - Limited evaluation dataset. Since the method builds on Deformable GS, it would be beneficial to evaluate on the same benchmark datasets, such as DNeRF, HyperNeRF, and NeRF-DS.
- While the method claims tractability as an advantage due to the analytical formulation, reliance on a black-box MLP weakens this claim.
- The video supplementary does not show particularly impressive results; there is noticeable unnatural blurring in the GSLK_network video

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: 1. The author should provide more details about the experimental setup, e.g., time step ∆t  in Eq.12. Please clarify how is it determined? does the method still satisfy the small motions assumption stated in Line 273 when ∆t is large?

2. The authors highlight in Section 6 that their method is sensitive to SfM initialization. Nevertheless, in my experience, 3DGS can still produce high-quality renderings even when initialized with random point clouds. I find it puzzling that the authors emphasize this aspect, and I would appreciate it if they could provide a more in-depth explanation for this claim.

3. It would be beneficial if the authors could furnish a detailed analysis of the time complexity for each component of the algorithm, as well as the memory requirements during the training process. This information is essential for assessing the algorithm's efficiency and scalability.

Currently, I would rate this paper as borderline. However, I am willing to reconsider my evaluation if the authors can provide additional clarifications and engage in a more in-depth discussion to address the concerns I have raised.
soundness: 3
strengths: The proposed method analytically derives the expression of warp field velocities and provides better tractability compared to directly supervising the flow field as the difference in estimated deformations between consecutive frames. Besides, the method is applicable to forward warp field techniques, and ensures accurate Gaussian trajectories throughout the sequence.
In real-world application, the author demonstrates that the warp field regularization accommodates deformations in complex scenes with minimal camera movement, achieving results.
The paper is well-written, concise, and has excellent formatting of figures and formulas.
summary: This paper tackles the challenges posed by dynamics and the lack of motion parallax that Gaussian Splatting methods face in real-world settings. The author proposes an analytical solution that adapts Lucas-Kanade method to dynamic Gaussian splitting. The proposed method leverages the forward warp field and derives an analytical velocity field for accurate scene flow computation. The experiment shows that “GAUSSIAN SPLATTING LUCAS-KANADE” outperforms state-of-the-art methods on both synthetic and real-world scenes, and demonstrates accurate reconstructing capability in complex scenes with minimal camera movement.
weaknesses: Despite the authors' efforts to derive an analytical velocity field and incorporate flow and depth supervision to regularize the Gaussian optimization, I have identified several questions that require attention.

1. The author suggests that the velocity field can be derived from the forward warp field by utilizing a modified Lucas-Kanade expression, and computing the fields corresponding to the displacement and rotation of Gaussians. Nevertheless, I argue that this claim is not entirely accurate. A crucial oversight in the authors' derivation is that optical flow or warp fields are inherently tied to both camera pose and Gaussian motion. Yet, the analysis seems to only consider the scenario flow resulting from the motion of the Gaussian alone, as evident in equations[8][11]. The motivation behind the authors' decision is unclear, but it is likely that incorporating camera pose variations into the Jacobian expression would lead to improved outcomes.

1. The authors should provide a more comprehensive explanation of their implementation, as the description of time integration on the velocity field in Section 4.2 remains unclear. Specifically, what is the value of the time step ∆t employed by the authors, and how is it determined? Furthermore, does the method still satisfy the small motions assumption stated in Line 273 when ∆t is large?

1. The proposed algorithm's reliance on computationally intensive operations, such as the (N × d) pseudo-inverse and time integration, raises concerns about its efficiency. To facilitate a more comprehensive assessment of the method's practical viability, it would be beneficial for the authors to provide a detailed analysis of the time complexity for each algorithmic component, as well as the overall memory requirements. This is essential for evaluating the algorithm's real-world applicability.

1. In my opinion as a reader, the paper's formulas are somewhat verbose, as the Autodiff library is leveraged for efficient gradient and Jacobian computations, obviating the need for manual CUDA implementation. Consequently, it may not be essential to include the detailed derivation process in the main body of the paper.

1. Although the paper's experimental design is thorough, it still exhibits some limitations. The experimental setup is somewhat brief and insufficient. I believe that relevant methods, such as [1][2], should be included as baselines to more thoroughly validate the effectiveness of this method. Additionally, I highly recommend that the authors incorporate a comparison with PhotoSLAM to further bolster the experimental setup.

[1] Motion-aware 3D Gaussian Splatting for Efficient Dynamic Scene Reconstruction

[2] DynMF: Neural Motion Factorization for Real-time Dynamic View Synthesis with 3D Gaussian Splatting

Meta-review: The paper presents an analytical approach to regularize dynamic Gaussian splatting using a modified Lucas-Kanade framework to compute velocity fields of Gaussian motion. It aims to improve the tractability and accuracy of dynamic scene reconstruction by using analytical formulations for scene flow instead of data-driven approaches. The proposed method achieves competitive results on monocular sequences and datasets with minimal camera motion and complex object dynamics.

Strengths:
- Analytical formulation: a novel analytical solution on velocity fields for dynamic Gaussian splatting for better tractability and interpretability.
- Practical settings: designed for realistic scenarios with minimal camera motion, where current algorithms have difficulties.
- Strong experimental results on monocular datasets and dynamic scenes.

Weaknesses:
- Unclear technical novelty: significant overlap in mathematical formulation with existing work such as "Flow Supervision for Deformable NeRF" and "GaussianFlow", raising concerns about originality.
- Limited experimental scope: evaluation on standard benchmarks such as DNeRF and NeRF-DS would better validate the generalizability of the method.
- Computational complexity: computationally intensive operations like pseudo-inverse and time integration may limit practical applications.

The paper provides a clear analytical contribution that improves the understanding and application of Gaussian splatting in dynamic scenarios. It demonstrates competitive experimental results in real-world applications with minimal camera motion. 
The theoretical originality of the paper is undermined by its significant overlap with previous work, but the rebuttal partially addresses the issue.

Decision: Accept (Poster)