GaussianImage: 1000 FPS Image Representation and Compression by 2D
  Gaussian Splatting
Authors: name: Xinjie Zhang; name: Xingtong Ge; name: Tongda Xu; name: Dailan He; name: Yan Wang; name: Hongwei Qin; name: Guo Lu; name: Jing Geng; name: Jun Zhang
Published: 2024-03-13 14:02:54

Abstract: Implicit neural representations (INRs) recently achieved great success in
image representation and compression, offering high visual quality and fast
rendering speeds with 10-1000 FPS, assuming sufficient GPU resources are
available. However, this requirement often hinders their use on low-end devices
with limited memory. In response, we propose a groundbreaking paradigm of image
representation and compression by 2D Gaussian Splatting, named GaussianImage.
We first introduce 2D Gaussian to represent the image, where each Gaussian has
8 parameters including position, covariance and color. Subsequently, we unveil
a novel rendering algorithm based on accumulated summation. Remarkably, our
method with a minimum of 3times lower GPU memory usage and 5times faster
fitting time not only rivals INRs (e.g., WIRE, I-NGP) in representation
performance, but also delivers a faster rendering speed of 1500-2000 FPS
regardless of parameter size. Furthermore, we integrate existing vector
quantization technique to build an image codec. Experimental results
demonstrate that our codec attains rate-distortion performance comparable to
compression-based INRs such as COIN and COIN++, while facilitating decoding
speeds of approximately 1000 FPS. Additionally, preliminary proof of concept
shows that our codec surpasses COIN and COIN++ in performance when using
partial bits-back coding.