CAPTOR

CAPTOR (Chest Abdomen Pelvis foundaTion model for Oncologic Radiology) is a CT vision-language model. It is trained CLIP-style to align 3D CT regions (thorax, abdomen, pelvis, ...) with the corresponding parts of radiology reports.

• Image encoder: 3D-inflated ResNet-152 (I3ResNet)
• Text encoder: Clinical-Longformer
• Architecture: based on Merlin
• Code: https://github.com/Guerbet-AI/CAPTOR
• Paper: CAPTOR: a vision-language foundation model for Chest-Abdomen-Pelvis Oncologic CT

Checkpoints

Each checkpoint is in its own subfolder. Only the one you request is downloaded.

CAPTOR_30K : ~30K CT studies from our proprietray dataset

CAPTOR_100K : ~100K CT studies from our proprietray dataset and public datasets : Merlin, CT-RATE, INSPECT, BIMCV-R

Quick start

pip install git+https://github.com/Guerbet-AI/CAPTOR.git

from captor import MerlinArchitecture

model = MerlinArchitecture.from_pretrained("Guerbet-AI/CAPTOR", subfolder="CAPTOR_30K")
print(model)

Weights, config and tokenizer are fetched automatically. No ResNet-152 or Clinical-Longformer initialization weights are needed.

The model is returned in eval mode. Call model.train() to fine-tune.

Image embeddings only

model = MerlinArchitecture.from_pretrained(
    "Guerbet-AI/CAPTOR", subfolder="CAPTOR_30K", ImageEmbedding=True
)

Offline use

On a machine with internet:

from huggingface_hub import snapshot_download

snapshot_download("Guerbet-AI/CAPTOR", local_dir="./captor_weights",
                  allow_patterns=["tokenizer/*", "CAPTOR_30K/*"])

Copy the folder to the offline machine, then:

model = MerlinArchitecture.from_pretrained("./captor_weights", subfolder="CAPTOR_30K")

Input preprocessing

The model expects CT volumes cropped around an anatomical region. Segmentation with TotalSegmentator, crop-center extraction and the dataloaders are part of the pipeline in the GitHub repository, which also documents the expected data format. Loading the model alone does not preprocess raw NIfTI files.

License and acknowledgements

CAPTOR builds on Merlin, Clinical-Longformer, torchvision's ResNet-152 and TotalSegmentator. Please respect their licenses.

Citation

@misc{jeanmactoux2026captorvisionlanguagefoundationmodel,
      title={CAPTOR: a vision-language foundation model for Chest-Abdomen-Pelvis Oncologic CT}, 
      author={Séléna Jean-Mactoux and Sophie Loizillon and Ghina Jardali and Amandine Crombé and Nathalie Lassau and Alexandre Bône},
      year={2026},
      eprint={2610.14226},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2610.14226}, 
}