DINO-X: A Unified Vision Model for Open-World Object Detection and
  Understanding
Authors: name: Tianhe Ren, name: Yihao Chen, name: Qing Jiang, name: Zhaoyang Zeng, name: Yuda Xiong, name: Wenlong Liu, name: Zhengyu Ma, name: Junyi Shen, name: Yuan Gao, name: Xiaoke Jiang, name: Xingyu Chen, name: Zhuheng Song, name: Yuhong Zhang, name: Hongjie Huang, name: Han Gao, name: Shilong Liu, name: Hao Zhang, name: Feng Li, name: Kent Yu, name: Lei Zhang

In this paper, we introduce DINO-X, which is a unified object-centric vision
model developed by IDEA Research with the best open-world object detection
performance to date. DINO-X employs the same Transformer-based encoder-decoder
architecture as Grounding DINO 1.5 to pursue an object-level representation for
open-world object understanding. To make long-tailed object detection easy,
DINO-X extends its input options to support text prompt, visual prompt, and
customized prompt. With such flexible prompt options, we develop a universal
object prompt to support prompt-free open-world detection, making it possible
to detect anything in an image without requiring users to provide any prompt.
To enhance the model's core grounding capability, we have constructed a
large-scale dataset with over 100 million high-quality grounding samples,
referred to as Grounding-100M, for advancing the model's open-vocabulary
detection performance. Pre-training on such a large-scale grounding dataset
leads to a foundational object-level representation, which enables DINO-X to
integrate multiple perception heads to simultaneously support multiple object
perception and understanding tasks, including detection, segmentation, pose
estimation, object captioning, object-based QA, etc. Experimental results
demonstrate the superior performance of DINO-X. Specifically, the DINO-X Pro
model achieves 56.0 AP, 59.8 AP, and 52.4 AP on the COCO, LVIS-minival, and
LVIS-val zero-shot object detection benchmarks, respectively. Notably, it
scores 63.3 AP and 56.5 AP on the rare classes of LVIS-minival and LVIS-val
benchmarks, both improving the previous SOTA performance by 5.8 AP. Such a
result underscores its significantly improved capacity for recognizing
long-tailed objects.