ICLR 2025

Published as a conference paper at ICLR 2025
GAUSSIAN SPLATTING LUCAS-KANADE
Liuyue Xie1, Joel Julin1, Koichiro Niinuma2, L´aszl´o A. Jeni1
1Carnegie Mellon University, 2Fujitsu Research of America
{liuyuex,jjulin}@andrew.cmu.edu, laszlojeni@cmu.edu, kniinuma@fujitsu.com
ABSTRACT
Gaussian Splatting and its dynamic extensions are effective for reconstructing 3D
scenes from 2D images when there is significant camera movement to facilitate
motion parallax and when scene objects remain relatively static. However, in many
real-world scenarios, these conditions are not met. As a consequence, data-driven
semantic and geometric priors have been favored as regularizers, despite their bias
toward training data and their neglect of broader movement dynamics.
Departing from this practice, we propose a novel analytical approach that adapts
the classical Lucas-Kanade method to dynamic Gaussian splatting. By leveraging
the intrinsic properties of the forward warp field network, we derive an analytical
velocity field that, through time integration, facilitates accurate scene flow compu-
tation. This enables the precise enforcement of motion constraints on warp fields,
thus constraining both 2D motion and 3D positions of the Gaussians. Our method
excels in reconstructing highly dynamic scenes with minimal camera movement,
as demonstrated through experiments on both synthetic and real-world scenes. 1
Figure 1: Data-driven depth and optical flow supervisions produce inaccurate geometries. Instead,
we derive the analytical warp field of Gaussians to refine geometries and motions.
1
INTRODUCTION
The task of reconstructing dynamic 3D scenes from monocular video presents a significant challenge,
particularly when constrained by limited camera movement. This limitation results in a lack of
parallax and epipolar constraints, which are crucial for accurately estimating scene structures Gao
et al. (2022); Sch¨onberger & Frahm (2016). Implicit neural representation methods like NeRF have
shown...