Model card for vit_huge_patch16_radio_v3.c_radio

A C-RADIOv3 (Agglomerative Vision Foundation Model) image feature encoder by NVIDIA. C-RADIOv3 was trained by multi-teacher distillation from DFN5B CLIP ViT-H/14-378, SigLIP2-SO400M, DINOv2-g/14 (w/ registers), SAM ViT-H. This is the VisionTransformer variant. The position embedding grid was resampled once, from RADIO's 128x128 grid to 32x32 (512x512), with RADIO's resize method; outputs match the original model at 512x512. Other resolutions (via img_size or dynamic_img_size=True) resample from this reduced grid and will drift from the original model, use naflexvit_huge_patch16_radio_v3 for original behaviour at all resolutions.

NOTE: This checkpoint is a native timm remap of NVIDIA's original nvidia/C-RADIOv3-H weights, with no additional training. Licensed by NVIDIA Corporation under the NVIDIA Open Model License, see LICENSE.pdf and NOTICE.

Model Notes

• The 8 prefix tokens are one CLS (summary) token per teacher followed by registers. timm models them as a cls_token plus reg_token. The default global_pool='token' uses the first CLS token. RADIO's summary output is the concatenation of CLS tokens [0, 1, 2] (DFN CLIP, SigLIP2, DINOv2), see usage below.
• Teacher adaptor heads are not included, so zero-shot text alignment from the original repository is not available.
• RADIO normalizes [0, 1] inputs internally with OpenAI CLIP mean / std. The timm pretrained_cfg applies the same normalization in the data transforms.
• The original bias-free patch embedding linear projection is remapped to a convolution with a zero bias. Position embeddings are resized with bilinear interpolation without antialiasing, to a square of the larger grid dimension then cropped top-left, with align_corners=True matching the original Hub model code for this checkpoint.

Model Details

• Model Type: Image Feature Encoder
• Model Stats:
  • Params (M): 632.0
  • GMACs: 738.0
  • Activations (M): 969.3
  • Image size: 512 x 512
• Source revision: d7fd0e2b0a1761f1af150582e06c41e9a99b0bf8
• Original: https://github.com/NVlabs/RADIO
• Original weights: https://huggingface.co/nvidia/C-RADIOv3-H
• License: NVIDIA Open Model License
• Training data: NV-CC-Img-Text-Dataset (per NVIDIA model card)
• Teachers:
  • DFN5B CLIP ViT-H/14-378
  • SigLIP2-SO400M
  • DINOv2-g/14 (w/ registers)
  • SAM ViT-H
• Papers:
  • RADIOv2.5: Improved Baselines for Agglomerative Vision Foundation Models: https://arxiv.org/abs/2412.07679
  • FeatSharp: Your Vision Model Features, Sharper: https://arxiv.org/abs/2502.16025
  • PHI-S: Distribution Balancing for Label-Free Multi-Teacher Distillation: https://arxiv.org/abs/2410.01680
  • AM-RADIO: Agglomerative Vision Foundation Model - Reduce All Domains Into One: https://arxiv.org/abs/2312.06709
  • PyTorch Image Models: https://github.com/huggingface/pytorch-image-models

Model Usage

Image Embeddings and Spatial Features

from urllib.request import urlopen
from PIL import Image
import torch
import timm

img = Image.open(urlopen(
    'https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/beignets-task-guide.png'
))

model = timm.create_model('hf-hub:timm/vit_huge_patch16_radio_v3.c_radio', pretrained=True).eval()
data_config = timm.data.resolve_model_data_config(model)
transforms = timm.data.create_transform(**data_config, is_training=False)
x = transforms(img).unsqueeze(0)

with torch.inference_mode():
    tokens = model.forward_features(x)  # (1, 1032, 1280): prefix + patch tokens
    embedding = model.forward_head(tokens, pre_logits=True)  # (1, 1280): first CLS token

# RADIO style outputs
summary = tokens[:, [0, 1, 2]].flatten(1)  # (1, 3840)
spatial = tokens[:, model.num_prefix_tokens:]  # (1, 1024, 1280)

Intermediate Feature Maps

with torch.inference_mode():
    maps = model.forward_intermediates(x, indices=4, output_fmt='NCHW', intermediates_only=True)
for feature_map in maps:
    print(feature_map.shape)  # (1, 1280, 32, 32)

Model Comparison

Results as reported by NVIDIA in the original C-RADIOv4 model cards. Zero-shot results use the original SigLIP2 adaptor and text encoder, which are not included here.

Benchmark	C-RADIOv3-B	C-RADIOv3-L	C-RADIOv4-SO400M	C-RADIOv3-H	C-RADIOv4-H
ImageNet Zero-Shot (Top-1)	71.30	79.95	82.01	82.65	83.09
ImageNet kNN (Top-1)	81.22	84.33	85.75	86.23	86.68
ADE20k Semantic Segmentation (mIoU)	49.79	51.87	55.14	52.75	55.20
Pascal VOC Semantic Segmentation (mIoU)	84.68	86.12	87.22	86.41	87.24

Citation

@misc{heinrich2025radiov25improvedbaselinesagglomerative,
  title={RADIOv2.5: Improved Baselines for Agglomerative Vision Foundation Models},
  author={Greg Heinrich and Mike Ranzinger and Hongxu and Yin and Yao Lu and Jan Kautz and Andrew Tao and Bryan Catanzaro and Pavlo Molchanov},
  year={2024},
  eprint={2412.07679},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={https://arxiv.org/abs/2412.07679},
}

@inproceedings{ranzinger2025featsharp,
  title={FeatSharp: Your Vision Model Features, Sharper},
  author={Mike Ranzinger and Greg Heinrich and Pavlo Molchanov and Bryan Catanzaro and Andrew Tao},
  booktitle={Forty-second International Conference on Machine Learning},
  year={2025},
  url={https://openreview.net/forum?id=lioemOcq3H}
}

@misc{ranzinger2024phisdistributionbalancinglabelfree,
  title={PHI-S: Distribution Balancing for Label-Free Multi-Teacher Distillation},
  author={Mike Ranzinger and Jon Barker and Greg Heinrich and Pavlo Molchanov and Bryan Catanzaro and Andrew Tao},
  year={2024},
  eprint={2410.01680},
  archivePrefix={arXiv},
  primaryClass={cs.LG},
  url={https://arxiv.org/abs/2410.01680},
}

@InProceedings{Ranzinger_2024_CVPR,
  author    = {Ranzinger, Mike and Heinrich, Greg and Kautz, Jan and Molchanov, Pavlo},
  title     = {AM-RADIO: Agglomerative Vision Foundation Model Reduce All Domains Into One},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  month     = {June},
  year      = {2024},
  pages     = {12490-12500}
}

@misc{rw2019timm,
  author = {Ross Wightman},
  title = {PyTorch Image Models},
  year = {2019},
  publisher = {GitHub},
  journal = {GitHub repository},
  doi = {10.5281/zenodo.4414861},
  howpublished = {\url{https://github.com/huggingface/pytorch-image-models}}
}