Challenge for Vision-Language Modeling in 3D Medical Imaging (VLM3D)
Creators: name: Hamamci, Ibrahim Ethem
name: Shit, Suprosanna
name: Sekuboyina, Anjany
name: Xu, Murong
name: Prabhakar, Chinmay
name: Menze, Bjoern
name: Bluethgen, Christian
name: Er, Sezgin
name: Simsek, Ayse Gulnihan
name: Durugol, Omer Faruk
name: Esirgun, Sevval Nil
name: Dasdelen, Muhammed Furkan
name: Simsek, Neslihan
name: Akan, Gulhan Ertan
name: Wang, Chenyu
name: Dai, Weicheng
name: Batmanghelich, Kayhan
name: Zhang, Xiaoman
name: Rajpurkar, Pranav
name: Bassi, Pedro R. A. S.
name: Li, Wenxuan
name: Yuille, Alan
name: Zhou, Zongwei
name: Reynaud, Hadrien
name: Kainz, Bernhard
name: Wu, Chaoyi
name: Xie, Weidi
name: Hou, Benjamin
name: Lu, Zhiyong
name: Xu, Daguang
name: Yang, Dong
name: Guo, Pengfei
Publisher: name: Zenodo
Year: 2025
Subjects: 3D Chest CT
Radiology Report
Multimodal AI
MICCAI 2025 challenge

description: Three-dimensional (3D) medical imaging, particularly chest computed tomography (CT), plays a vital role in diagnosing thoracic abnormalities by offering detailed insights into complex anatomical structures. However, interpreting 3D CT data is a time-consuming and challenging process, especially given the increasing global demand for CT scans. While artificial intelligence (AI) has demonstrated significant potential in automating radiological tasks such as report generation and abnormality detection, its application to 3D medical imaging remains limited due to the lack of large-scale, paired datasets and the computational challenges associated with processing 3D data.

To address these challenges, we introduce the Challenge for Vision-Language Modeling in 3D Medical Imaging (VLM3D), built around the open-source CT-RATE dataset. CT-RATE pairs over 50,000 3D chest CT volumes with corresponding radiology reports, annotated for 18 clinically significant abnormalities. The VLM3D Challenge presents participants with four tasks: (1) radiology report generation from chest CT volumes, (2) multi-abnormality classification, (3) self-supervised multi-abnormality localization, and (4) text-conditional 3D chest CT generation.

These tasks address key aspects of radiological diagnostics and treatment planning. Radiology report generation automates the creation of accurate and detailed reports from CT volumes, reducing radiologists' workloads. Multi-abnormality classification enables quicker and more precise detection of pathologies in CT scans. The self-supervised multi-abnormality localization task focuses on identifying specific pathological regions, including pericardial effusion, pleural effusion, consolidation, lung opacity, and lung nodules, without the use of ground-truth labels during training. This task will be evaluated using a manually labeled dataset to assess how effectively models can localize abnormalities in a self-supervised manner. Text-conditional 3D chest CT generation supports the creation of realistic CT volumes from textual descriptions, with applications in data augmentation, education, and generative modeling research.

The challenge evaluation will utilize two datasets: an internal closed test set containing 2,000 cases and an external closed test set from Boston University Hospital with 1,024 cases. This dual evaluation framework ensures that models are assessed on diverse and clinically relevant data, supporting their real-world applicability. The training and validation datasets are derived from the CT volumes in the CT-RATE dataset.

The VLM3D Challenge aims to advance AI in 3D medical imaging by providing a benchmark for vision-language models. Biomedically, it seeks to improve diagnostic accuracy, streamline radiological workflows, and enhance patient care. Technically, it encourages the development of scalable and generalizable AI systems that integrate visual and textual modalities, fostering innovation and collaboration in research and clinical applications.