Towards Learning to Complete Anything in Lidar
Authors: name: Ayca Takmaz, name: Cristiano Saltori, name: Neehar Peri, name: Tim Meinhardt, name: Riccardo de Lutio, name: Laura Leal-Taixé, name: Aljoša Ošep

We propose CAL (Complete Anything in Lidar) for Lidar-based shape-completion
in-the-wild. This is closely related to Lidar-based semantic/panoptic scene
completion. However, contemporary methods can only complete and recognize
objects from a closed vocabulary labeled in existing Lidar datasets. Different
to that, our zero-shot approach leverages the temporal context from multi-modal
sensor sequences to mine object shapes and semantic features of observed
objects. These are then distilled into a Lidar-only instance-level completion
and recognition model. Although we only mine partial shape completions, we find
that our distilled model learns to infer full object shapes from multiple such
partial observations across the dataset. We show that our model can be prompted
on standard benchmarks for Semantic and Panoptic Scene Completion, localize
objects as (amodal) 3D bounding boxes, and recognize objects beyond fixed class
vocabularies. Our project page is
https://research.nvidia.com/labs/dvl/projects/complete-anything-lidar