Does DINOv3 Set a New Medical Vision Standard?
Authors: name: Che Liu; name: Yinda Chen; name: Haoyuan Shi; name: Jinpeng Lu; name: Bailiang Jian; name: Jiazhen Pan; name: Linghan Cai; name: Jiayi Wang; name: Yundi Zhang; name: Jun Li; name: Cosmin I. Bercea; name: Cheng Ouyang; name: Chen Chen; name: Zhiwei Xiong; name: Benedikt Wiestler; name: Christian Wachinger; name: Daniel Rueckert; name: Wenjia Bai; name: Rossella Arcucci
Published: 2025-09-08 09:28:57

Abstract: The advent of large-scale vision foundation models, pre-trained on diverse
natural images, has marked a paradigm shift in computer vision. However, how
the frontier vision foundation models' efficacies transfer to specialized
domains remains such as medical imaging remains an open question. This report
investigates whether DINOv3, a state-of-the-art self-supervised vision
transformer (ViT) that features strong capability in dense prediction tasks,
can directly serve as a powerful, unified encoder for medical vision tasks
without domain-specific pre-training. To answer this, we benchmark DINOv3
across common medical vision tasks, including 2D/3D classification and
segmentation on a wide range of medical imaging modalities. We systematically
analyze its scalability by varying model sizes and input image resolutions. Our
findings reveal that DINOv3 shows impressive performance and establishes a
formidable new baseline. Remarkably, it can even outperform medical-specific
foundation models like BiomedCLIP and CT-Net on several tasks, despite being
trained solely on natural images. However, we identify clear limitations: The
model's features degrade in scenarios requiring deep domain specialization,
such as in Whole-Slide Pathological Images (WSIs), Electron Microscopy (EM),
and Positron Emission Tomography (PET). Furthermore, we observe that DINOv3
does not consistently obey scaling law in the medical domain; performance does
not reliably increase with larger models or finer feature resolutions, showing
diverse scaling behaviors across tasks. Ultimately, our work establishes DINOv3
as a strong baseline, whose powerful visual features can serve as a robust
prior for multiple complex medical tasks. This opens promising future
directions, such as leveraging its features to enforce multiview consistency in
3D reconstruction.