A Review of 3D Object Detection with Vision-Language Models
Authors: name: Ranjan Sapkota; name: Konstantinos I Roumeliotis; name: Rahul Harsha Cheppally; name: Marco Flores Calero; name: Manoj Karkee
Published: 2025-04-25 23:27:26

Abstract: This review provides a systematic analysis of comprehensive survey of 3D
object detection with vision-language models(VLMs) , a rapidly advancing area
at the intersection of 3D vision and multimodal AI. By examining over 100
research papers, we provide the first systematic analysis dedicated to 3D
object detection with vision-language models. We begin by outlining the unique
challenges of 3D object detection with vision-language models, emphasizing
differences from 2D detection in spatial reasoning and data complexity.
Traditional approaches using point clouds and voxel grids are compared to
modern vision-language frameworks like CLIP and 3D LLMs, which enable
open-vocabulary detection and zero-shot generalization. We review key
architectures, pretraining strategies, and prompt engineering methods that
align textual and 3D features for effective 3D object detection with
vision-language models. Visualization examples and evaluation benchmarks are
discussed to illustrate performance and behavior. Finally, we highlight current
challenges, such as limited 3D-language datasets and computational demands, and
propose future research directions to advance 3D object detection with
vision-language models. >Object Detection, Vision-Language Models, Agents,
VLMs, LLMs, AI