Model Overview

Description or Model Summary with Description and License:

C-RADIOv4-H-AV generates embeddings for images and text in a shared space, used for text-to-image and image-to-image retrieval over autonomous vehicle (AV) driving data. C-RADIOv4-H-AV was developed by NVIDIA as a fine-tune of C-RADIOv4-H (SigLIP2-g adaptor), a contrastive CLIP-style model that outputs embeddings and similarity scores; it does not generate images or text. Only the SigLIP2-g adaptor head was trained; the image backbone and the text encoder are unchanged from the base model.

This model is ready for commercial or non-commercial use.

License/Terms of Use:

GOVERNING TERMS: Use of this model is governed by the NVIDIA Open Model License Agreement.

The base model, C-RADIOv4-H, is governed by the NVIDIA Open Model License Agreement.

Deployment Geography:

Global

Use Case:

AV data engineers and developers searching large driving-data collections with text or image queries, for example finding traffic signs, traffic lights, road markings, intersections and other driving scenes.

Release Date:

Hugging Face 11/20/2026 via https://huggingface.co/nvidia/C-RADIOv4-H-AV

References(s):

Model Architecture:

Architecture Type: Transformer

Network Architecture: Other: Vision Transformer (ViT-H) image encoder with an MLP adaptor head (SigLIP2-g adaptor) and a text encoder

This model was developed based on C-RADIOv4-H

** Number of model parameters: C-RADIOv4-H (base) 653M. Only the SigLIP2-g adaptor head (8.9M parameters) was trained.

Input(s):

Input Type(s): Image, Text

Input Format(s):

  • Image: Red, Green, Blue (RGB)
  • Text: String

Input Parameters:

  • Image: Two-Dimensional (2D)
  • Text: One-Dimensional (1D)

Other Properties Related to Input: Images are resized to 384x384 without preserving the aspect ratio.

Output(s)

Output Type(s): Other: Embeddings

Output Format(s): Other: Tensor (embedding vectors)

Output Parameters: One-Dimensional (1D)

Other Properties Related to Output: 512-dimensional vectors (first 512 PCA components of the 1536-dimensional adaptor output). The PCA eigenvalues are near zero beyond about 500 components, so the reduction shows a negligible drop on the AV and generalization benchmarks. Image and text embeddings share one space and are compared with a dot product or cosine similarity. The model outputs embeddings only; it does not generate images or text. A downstream application can use the cosine similarity between a query (text or image) embedding and image embeddings to rank multiple images by relevance.

Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA’s hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.

Usage:

The model is loaded with the RADIO PyTorch code through torch.hub, using the checkpoint downloaded from this repository. The 512-dimensional output is obtained by multiplying the eigenvectors (c-radio_v4-h_half_av-eigenvectors.npy) into the SigLIP2-g adaptor head weights and keeping the first 512 output dimensions. For both images and text, the adaptor returns a vector (summary) of shape (B, 1536) before truncation to (B, 512).

import numpy as np
import torch
from huggingface_hub import hf_hub_download
from PIL import Image
from torchvision.transforms.functional import to_tensor

repo = "nvidia/C-RADIOv4-H-AV"

ckpt = hf_hub_download(repo, "c-radio_v4-h_half_av.pth.tar")
eigenvectors = hf_hub_download(repo, "c-radio_v4-h_half_av-eigenvectors.npy")

model = torch.hub.load("NVlabs/RADIO", "radio_model", version=ckpt,
                       adaptor_names=["siglip2-g"], progress=True)

# Absorb the PCA eigenvectors into the image and text heads of the SigLIP2-g adaptor.
E = torch.from_numpy(np.load(eigenvectors).astype(np.float32))
adaptor = model.adaptors["siglip2-g"]
for layer in (adaptor.head_mlp.final[2], adaptor.text_model.head):
    layer.weight.data = (E.T @ layer.weight.data.float()).to(layer.weight.dtype)
    layer.bias.data = (E.T @ layer.bias.data.float()).to(layer.bias.dtype)
model.eval().cuda()

# Image embedding.
image = Image.open("image.jpg").convert("RGB").resize((384, 384), Image.BICUBIC)
x = to_tensor(image).unsqueeze(0).cuda()
with torch.no_grad():
    summary, _ = model(x)["siglip2-g"]
image_embedding = summary[:, :512].float()

# Text embedding.
tokens = adaptor.tokenizer(["a traffic light"]).to("cuda")
with torch.no_grad():
    text_embedding = adaptor.encode_text(tokens)[:, :512].float()

# Cosine similarity can be used to rank images for a text or image query.
score = torch.nn.functional.cosine_similarity(image_embedding, text_embedding)

Software Integration:

Runtime Engine(s):

  • Not Applicable (N/A) - PyTorch

Supported Hardware Microarchitecture Compatibility:

  • NVIDIA Ampere

Supported Operating System(s):

  • Linux

The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Following the V-model methodology, iterative testing and validation at both unit and system levels are essential to mitigate risks, meet technical and functional requirements, and ensure compliance with safety and ethical standards before deployment.

Model Version(s):

V1. The model can be integrated into an AI system by loading it with the RADIO PyTorch code (torch.hub) and calling it to embed images and text for retrieval.

Training, Testing, and Evaluation Datasets:

Dataset Overview

** Total Size: 3,747,347 image-caption pairs (3,369,193 unique images)
** Total Number of Datasets: 1 (MB AV Data, made of AV data and NVClip data)

** A general description of the data processing involved in transforming the acquired data into the training data for the model: AV images from NVIDIA AV data are paired with captions generated by Kimi K3 and caption templates. NVClip image-caption pairs from NV-CC-Img-Text-Dataset are used with the captions provided. Image paths and captions are stored in a CSV of image_path,caption pairs.

Private Datasets

  • List of parties providing datasets: NVIDIA (AV data and NV-CC-Img-Text-Dataset)
  • Number of data points: 3,747,347 image-caption pairs (about 1.8M AV and 1.8M NVClip)
  • Overall size per modality: about 274 GB of images
  • Nature of content: AV images (road-scene data that may contain personal data), synthetic captions

Self-Sourced Synthetic Data

  • Number of data points: 1,850,979 AV captions
  • Overall size per modality:
  • A description of the methods used to synthetically generate training data: AV captions are generated with Kimi K3 and caption templates.
  • The name(s) of any AI model(s) used: Kimi K3

Training Dataset:

Data Modality:

  • Image
  • Text

Training Data Size:

** Image Training Data Size

  • 1 Million to 1 Billion Images

** Text Training Data Size

  • Less than a Billion Tokens

** Data Collection Method by dataset

  • Hybrid: Automatic/Sensors (AV data), Automated (NVClip data)

** Labeling Method by dataset

  • Hybrid: Synthetic (AV captions), Automated (NVClip captions)

Properties: 3,747,347 image-caption pairs (1,850,979 AV, 1,896,368 NVClip) over 3,369,193 unique images, about 274 GB. The AV images come from a 120° field of view (FOV) car camera and cover road markings, lane markings, traffic signs, traffic lights, intersections and full driving scenes; they may contain personal data from real-world driving scenes. The captions are in English.

Testing Dataset:

Data Collection Method by dataset:

  • AV validation data: Automatic/Sensors
  • NVClip validation data: Automated

Labeling Method by dataset:

  • AV validation data: Synthetic
  • NVClip validation data: Automated

Properties The testing data is the validation set used to select the training epoch: 131,604 image-caption pairs (116,221 AV, 15,383 NVClip), image and text modalities, English captions. The AV images are road scenes from a 120° FOV car camera and may contain personal data from real-world driving scenes; the NVClip images are web images. For the AV data, the training, validation and evaluation sets share no H3 cell for a class.

Evaluation Dataset:

**Benchmark Score

Data Collection Method by dataset:

  • AVObjects: Automatic/Sensors
  • NVImageNet: Automated

Labeling Method by dataset:

  • AVObjects: Synthetic (text queries generated with Kimi K3)
  • NVImageNet: Automated (class label taken from the ImageNet category each image was collected for)

Properties AVObjects has 73,961 gallery images (Intersection 5,466; Road Markings 10,162; Traffic Light 7,209; Traffic Sign 51,124), 843 query images and 93 text queries, all in English; the images are road scenes from a 120° FOV car camera and may contain personal data from real-world driving scenes. NVImageNet (NV-CC-Img-Text-Dataset, default version) is evaluated on 1,000 ImageNet classes with up to 100 images per class; the images are natural photos and the class names are in English. Evaluation covers AVObjects (image-to-image and text-to-image retrieval; Intersection, Road Markings, Traffic Light, Traffic Sign) and NVImageNet (Top-1 and Top-5 accuracy, to check that generalization is retained). The splits share no H3 cell per class with the training data.

Higher is better for all metrics.

NVImageNet:

Model Top-1 (%) Top-5 (%)
C-RADIOv4-H (base, no fine-tuning) 60.8 84.5
C-RADIOv4-H-AV 57.6 82.2

AVObjects image-to-image retrieval:

Dataset Model Pr@100 Recall@100 AUROC PR_AUC
Intersection C-RADIOv4-H (base) 0.875 0.081 0.790 0.619
C-RADIOv4-H-AV 0.902 0.083 0.880 0.736
Road Markings C-RADIOv4-H (base) 0.437 0.035 0.584 0.236
C-RADIOv4-H-AV 0.696 0.055 0.774 0.480
Traffic Light C-RADIOv4-H (base) 0.632 0.036 0.594 0.356
C-RADIOv4-H-AV 0.951 0.057 0.840 0.703
Traffic Sign C-RADIOv4-H (base) 0.574 0.119 0.855 0.304
C-RADIOv4-H-AV 0.762 0.169 0.973 0.623

AVObjects text-to-image retrieval:

Dataset Model Pr@100 Recall@100 AUROC PR_AUC
Intersection C-RADIOv4-H (base) 0.926 0.086 0.881 0.749
C-RADIOv4-H-AV 0.944 0.087 0.937 0.834
Road Markings C-RADIOv4-H (base) 0.469 0.036 0.603 0.274
C-RADIOv4-H-AV 0.577 0.046 0.772 0.464
Traffic Light C-RADIOv4-H (base) 0.968 0.058 0.842 0.702
C-RADIOv4-H-AV 0.943 0.056 0.875 0.726
Traffic Sign C-RADIOv4-H (base) 0.650 0.145 0.913 0.433
C-RADIOv4-H-AV 0.762 0.175 0.977 0.624

Inference:

Acceleration (Engine used to optimize the runtime if applicable) Engine: PyTorch
Test Hardware Set-Up (GPU Architecture, Model):

  • NVIDIA Ampere A100

Ethical Considerations:

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. Developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.

Please make sure you have proper rights and permissions for all input image and video content; if image or video includes people, personal health information, or intellectual property, the image or video generated will not blur or maintain proportions of image subjects included.

For more detailed information on ethical considerations for this model, please see the Model Card++ Bias, Explainability, Safety & Security, and Privacy Subcards below.

Please report model quality, risk, security vulnerabilities or NVIDIA AI Concerns here.

Model Card++ Subcards

Bias

Field Response
Participation considerations from adversely impacted groups protected classes in model design and testing: None
Measures taken to mitigate against unwanted bias: None
Bias Metric (If Measured): Not Applicable

Explainability

Field Response
Intended Task/Domain: Text-to-image and image-to-image retrieval over driving data
Model Type: Transformer
Intended Users: AV data engineers and developers use this model to search large driving-data collections with text or image queries, for example to find traffic signs, traffic lights, road markings and intersections for data curation.
Output: 512-dimensional image and text embeddings
Describe how the model works: The image is resized and passed through a Vision Transformer and the SigLIP2-g adaptor head to give an embedding. The text is passed through the text encoder to give an embedding in the same space. A dot product or cosine similarity between the two gives the match score.
List the groups (or group characteristics) for which this was tested to produce comparable outcomes: Not Applicable
Technical Limitations & Mitigation: Fine-tuned on driving data from a 120° FOV car camera and evaluated on AVObjects and NVImageNet. Other camera setups and domains have not been evaluated; domain-specific evaluation is recommended for the target application.
Verified to have met prescribed NVIDIA quality standards: Yes
Performance Metrics: Precision@100, Recall@100, AUROC, PR-AUC (AVObjects); Top-1 and Top-5 accuracy (NVImageNet)
Potential Known Risks: The model only computes similarity scores between the input images and text; it does not generate images or text. Rankings may place irrelevant items high, or miss relevant ones, for rare or unseen categories.
Licensing: NVIDIA Open Model License

Privacy

The datasets used to train C-RADIOv4-H-AV included data from real-world road driving scenes. NVIDIA collected and used this data in compliance with applicable data protection and privacy laws. The datasets used to train this model were reviewed before release. Please review NVIDIA’s Privacy Policy (https://www.nvidia.com/en-us/about-nvidia/privacy-policy/) for more information.

Safety & Security

Field Response
Model Application Field(s): Autonomous Vehicles
Describe the life critical impact (if present). Not applicable. This model computes similarity scores between the input images and text; it does not generate images or text. Rankings may place irrelevant items high, or miss relevant ones, for rare or unseen categories.
Use Case Restrictions: Abide by NVIDIA Open Model License Agreement
Model and dataset restrictions: The Principle of least privilege (PoLP) is applied limiting access for dataset generation and model development. Restrictions enforce dataset access during training, and dataset license constraints adhered to.
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for nvidia/C-RADIOv4-H-AV

Finetuned
(5)
this model

Paper for nvidia/C-RADIOv4-H-AV