Feed-Forward SceneDINO for Unsupervised Semantic Scene Completion
Authors: name: Aleksandar Jevtić; name: Christoph Reich; name: Felix Wimbauer; name: Oliver Hahn; name: Christian Rupprecht; name: Stefan Roth; name: Daniel Cremers
Published: 2025-07-08 17:59:50

Abstract: Semantic scene completion (SSC) aims to infer both the 3D geometry and
semantics of a scene from single images. In contrast to prior work on SSC that
heavily relies on expensive ground-truth annotations, we approach SSC in an
unsupervised setting. Our novel method, SceneDINO, adapts techniques from
self-supervised representation learning and 2D unsupervised scene understanding
to SSC. Our training exclusively utilizes multi-view consistency
self-supervision without any form of semantic or geometric ground truth. Given
a single input image, SceneDINO infers the 3D geometry and expressive 3D DINO
features in a feed-forward manner. Through a novel 3D feature distillation
approach, we obtain unsupervised 3D semantics. In both 3D and 2D unsupervised
scene understanding, SceneDINO reaches state-of-the-art segmentation accuracy.
Linear probing our 3D features matches the segmentation accuracy of a current
supervised SSC approach. Additionally, we showcase the domain generalization
and multi-view consistency of SceneDINO, taking the first steps towards a
strong foundation for single image 3D scene understanding.