ICLR 2025

Published as a conference paper at ICLR 2025
UNI2DET: UNIFIED AND UNIVERSAL FRAMEWORK FOR
PROMPT-GUIDED MULTI-DATASET 3D DETECTION
Yubin Wang1∗, Zhikang Zou2∗, Xiaoqing Ye2, Xiao Tan2, Errui Ding2, Cairong Zhao1†
1School of Computer Science and Technology, Tongji University, 2Baidu Inc.
ABSTRACT
We present Uni2Det, a brand new framework for unified and universal multi-
dataset training on 3D detection, enabling robust performance across diverse
domains and generalization to unseen domains. Due to substantial disparities
in data distribution and variations in taxonomy across diverse domains, train-
ing such a detector by simply merging datasets poses a significant challenge.
Motivated by this observation, we introduce multi-stage prompting modules for
multi-dataset 3D detection, which leverages prompts based on the characteristics
of corresponding datasets to mitigate existing differences. This elegant design
facilitates seamless plug-and-play integration within various advanced 3D detection
frameworks in a unified manner, while also allowing straightforward adaptation
for universal applicability across datasets. Experiments are conducted across mul-
tiple dataset consolidation scenarios involving KITTI, Waymo, and nuScenes,
demonstrating that our Uni2Det outperforms existing methods by a large margin in
multi-dataset training. Notably, results on zero-shot cross-dataset transfer validate
the generalization capability of our proposed method. Our code is available at
https://github.com/ThomasWangY/Uni2Det.
1
INTRODUCTION
With the ability to capture precise geometric information of entire scenes, LiDAR has become
an essential sensor for most autonomous vehicles. Due to the rapid development of large-scale
annotated 3D LiDAR datasets such as Waymo (Sun et al., 2020), nuScenes (Caesar et al., 2020), and
KITTI (Geiger et al., 2012), LiDAR-based models play a significant role in various critical perception
tasks for autonomous vehicles, particularly in 3D object detection....