Text-guided Sparse Voxel Pruning for Efficient 3D Visual Grounding
Authors: name: Wenxuan Guo; name: Xiuwei Xu; name: Ziwei Wang; name: Jianjiang Feng; name: Jie Zhou; name: Jiwen Lu
Published: 2025-02-14 18:59:59

Abstract: In this paper, we propose an efficient multi-level convolution architecture
for 3D visual grounding. Conventional methods are difficult to meet the
requirements of real-time inference due to the two-stage or point-based
architecture. Inspired by the success of multi-level fully sparse convolutional
architecture in 3D object detection, we aim to build a new 3D visual grounding
framework following this technical route. However, as in 3D visual grounding
task the 3D scene representation should be deeply interacted with text
features, sparse convolution-based architecture is inefficient for this
interaction due to the large amount of voxel features. To this end, we propose
text-guided pruning (TGP) and completion-based addition (CBA) to deeply fuse 3D
scene representation and text features in an efficient way by gradual region
pruning and target completion. Specifically, TGP iteratively sparsifies the 3D
scene representation and thus efficiently interacts the voxel features with
text features by cross-attention. To mitigate the affect of pruning on delicate
geometric information, CBA adaptively fixes the over-pruned region by voxel
completion with negligible computational overhead. Compared with previous
single-stage methods, our method achieves top inference speed and surpasses
previous fastest method by 100\% FPS. Our method also achieves state-of-the-art
accuracy even compared with two-stage methods, with +1.13 lead of Acc@0.5 on
ScanRefer, and +2.6 and +3.2 leads on NR3D and SR3D respectively. The code
is available at
https://github.com/GWxuan/TSP3D{https://github.com/GWxuan/TSP3D}.