3DGS-DET: Empower 3D Gaussian Splatting with Boundary Guidance and
  Box-Focused Sampling for 3D Object Detection
Authors: name: Yang Cao; name: Yuanliang Jv; name: Dan Xu
Published: 2024-10-02 15:15:52

Abstract: Neural Radiance Fields (NeRF) are widely used for novel-view synthesis and
have been adapted for 3D Object Detection (3DOD), offering a promising approach
to 3DOD through view-synthesis representation. However, NeRF faces inherent
limitations: (i) limited representational capacity for 3DOD due to its implicit
nature, and (ii) slow rendering speeds. Recently, 3D Gaussian Splatting (3DGS)
has emerged as an explicit 3D representation that addresses these limitations.
Inspired by these advantages, this paper introduces 3DGS into 3DOD for the
first time, identifying two main challenges: (i) Ambiguous spatial distribution
of Gaussian blobs: 3DGS primarily relies on 2D pixel-level supervision,
resulting in unclear 3D spatial distribution of Gaussian blobs and poor
differentiation between objects and background, which hinders 3DOD; (ii)
Excessive background blobs: 2D images often include numerous background pixels,
leading to densely reconstructed 3DGS with many noisy Gaussian blobs
representing the background, negatively affecting detection. To tackle the
challenge (i), we leverage the fact that 3DGS reconstruction is derived from 2D
images, and propose an elegant and efficient solution by incorporating 2D
Boundary Guidance to significantly enhance the spatial distribution of Gaussian
blobs, resulting in clearer differentiation between objects and their
background. To address the challenge (ii), we propose a Box-Focused Sampling
strategy using 2D boxes to generate object probability distribution in 3D
spaces, allowing effective probabilistic sampling in 3D to retain more object
blobs and reduce noisy background blobs. Benefiting from our designs, our
3DGS-DET significantly outperforms the SOTA NeRF-based method, NeRF-Det,
achieving improvements of +6.6 on mAP@0.25 and +8.1 on mAP@0.5 for the ScanNet
dataset, and impressive +31.5 on mAP@0.25 for the ARKITScenes dataset.