Splatter Image: Ultra-Fast Single-View 3D Reconstruction
Authors: name: Stanislaw Szymanowicz; name: Christian Rupprecht; name: Andrea Vedaldi
Published: 2023-12-20 16:14:58

Abstract: We introduce the Splatter Image, an ultra-fast approach for monocular 3D
object reconstruction which operates at 38 FPS. Splatter Image is based on
Gaussian Splatting, which has recently brought real-time rendering, fast
training, and excellent scaling to multi-view reconstruction. For the first
time, we apply Gaussian Splatting in a monocular reconstruction setting. Our
approach is learning-based, and, at test time, reconstruction only requires the
feed-forward evaluation of a neural network. The main innovation of Splatter
Image is the surprisingly straightforward design: it uses a 2D image-to-image
network to map the input image to one 3D Gaussian per pixel. The resulting
Gaussians thus have the form of an image, the Splatter Image. We further extend
the method to incorporate more than one image as input, which we do by adding
cross-view attention. Owning to the speed of the renderer (588 FPS), we can use
a single GPU for training while generating entire images at each iteration in
order to optimize perceptual metrics like LPIPS. On standard benchmarks, we
demonstrate not only fast reconstruction but also better results than recent
and much more expensive baselines in terms of PSNR, LPIPS, and other metrics.