AlphaTablets: A Generic Plane Representation for 3D Planar
  Reconstruction from Monocular Videos
Authors: name: Yuze He; name: Wang Zhao; name: Shaohui Liu; name: Yubin Hu; name: Yushi Bai; name: Yu-Hui Wen; name: Yong-Jin Liu
Published: 2024-11-29 18:59:52

Abstract: We introduce AlphaTablets, a novel and generic representation of 3D planes
that features continuous 3D surface and precise boundary delineation. By
representing 3D planes as rectangles with alpha channels, AlphaTablets combine
the advantages of current 2D and 3D plane representations, enabling accurate,
consistent and flexible modeling of 3D planes. We derive differentiable
rasterization on top of AlphaTablets to efficiently render 3D planes into
images, and propose a novel bottom-up pipeline for 3D planar reconstruction
from monocular videos. Starting with 2D superpixels and geometric cues from
pre-trained models, we initialize 3D planes as AlphaTablets and optimize them
via differentiable rendering. An effective merging scheme is introduced to
facilitate the growth and refinement of AlphaTablets. Through iterative
optimization and merging, we reconstruct complete and accurate 3D planes with
solid surfaces and clear boundaries. Extensive experiments on the ScanNet
dataset demonstrate state-of-the-art performance in 3D planar reconstruction,
underscoring the great potential of AlphaTablets as a generic 3D plane
representation for various applications. Project page is available at:
https://hyzcluster.github.io/alphatablets