Virchow-SPECTRA

MedARC · Sophont · Paper (coming soon)

Virchow-SPECTRA is Virchow adapted with SPECTRA by MedARC and Sophont. We trained on registered PLISM image tiles, aligning representations of the same tissue across scanners and stains with a contrastive objective. The learned updates are merged into the full model weights.

The model maps RGB histology tiles to 2,560-dimensional embeddings for downstream analysis. All inference weights are included; no separate base checkpoint, adapter, or cluster access is required.

Inference

In a Python environment:

uv pip install torch torchvision "timm==1.0.30" huggingface_hub safetensors pillow

import torch
import timm
from PIL import Image
from timm.data import create_transform, resolve_model_data_config
from timm.layers import SwiGLUPacked

device = "cuda"  # Use "cpu" to run without a GPU.
model = timm.create_model(
    "hf-hub:medarc/Virchow-SPECTRA", pretrained=True, mlp_layer=SwiGLUPacked, act_layer=torch.nn.SiLU,
).eval().to(device)
transform = create_transform(**resolve_model_data_config(model), is_training=False)

image = Image.open("tile.png").convert("RGB")
batch = transform(image).unsqueeze(0).to(device)
with torch.inference_mode():
    tokens = model.forward_features(batch)
    embedding = torch.cat([tokens[:, 0], tokens[:, 1:].mean(dim=1)], dim=-1)
print(embedding.shape)  # torch.Size([1, 2560])

The configuration supplies bicubic resizing to 224 pixels, a 224 × 224 center crop, and ImageNet normalization stored in config.json. The embedding uses concatenated CLS and mean patch tokens. Use tissue tiles sampled at 0.5 µm/pixel, following the base model.

These are tile embeddings. Slide-level predictions require a downstream model.