RendeiroLab/metpredict-vessel-airway-seg

MetPredict Blood vessel and airway Segmentation (DPT)

Dense semantic segmentation for lung H&E pathology (blood vessel and airway).

• Encoder (frozen): H-optimus-0 ViT backbone (pretrained on histopathology data).
• Decoder (trained): custom DPT head with multi-scale feature fusion.

Classes (3): 0 = background, 1 = blood vessel, 2 = airway Input tile: 224x224 @ 1.5 MPP, ImageNet-normalized RGB.

Preprocessing

from torchvision.transforms import ToTensor, Normalize, Resize, Compose

transform = Compose([
    ToTensor(),
    Resize((224, 224)),
    Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])
# pixel_values = transform(pil_rgb_image).unsqueeze(0)  # (1, 3, 224, 224)

Usage

Option A — Transformers (safetensors). Needs transformers with trust_remote_code=True, and access to the gated bioptimus/H-optimus-0 backbone (re-instantiated at load).

import torch
from transformers import AutoModel

model = AutoModel.from_pretrained("RendeiroLab/metpredict-vessel-airway-seg", trust_remote_code=True).eval()
with torch.inference_mode():
    out = model(pixel_values)
logits = out.logits                # (B, 3, H, W)
pred = logits.argmax(dim=1)        # (B, H, W)

Option B — torch.export (model.pt2): torch-only, self-contained. No transformers, no custom code, no gated-backbone download — the weights are baked into the exported program.

import torch
from huggingface_hub import hf_hub_download

path = hf_hub_download("RendeiroLab/metpredict-vessel-airway-seg", "model.pt2")
model = torch.export.load(path).module()
with torch.inference_mode():
    logits = model(pixel_values)   # (B, 3, H, W)
pred = logits.argmax(dim=1)

Validation metrics

Held-out validation split of the 16-PDX reported cohort, all figures from the single exported epoch (epoch 72).

Class	Precision	Recall	F1	IoU
background	0.977	0.935	0.955	0.915
blood vessel	0.605	0.791	0.686	0.522
airway	0.763	0.908	0.829	0.709

• Mean foreground IoU: 0.615 (primary metric)
• Mean IoU incl. background: 0.715
• Mean foreground Dice: 0.430
• Scope: trained on all annotated PDX lines; metrics reported on the 16-PDX reported cohort only.
• clDice (tubular connectivity): 0.724
• Per-PDX foreground IoU (n=16 lines with adequate validation data): min 0.555 / median 0.601 / max 0.738
• PDX-macro foreground IoU, n=16: 0.615 (lines weighted equally, not by tile count)