cloudkites/her2-reader

HER2 reader: a small ViT-S/16 encoder and HER2 slide classifier for routine H&E

Model weights and output data behind the paper "A small HER2 reader for routine H&E slides: real-world deployment" (Thang Tran, Lan Dang, 2026; arXiv link to follow). We publish them so that anyone can check our numbers, rerun our evaluation and build on our encoder.

Research use only. This is not a medical device and must not be used for clinical decisions. Our reader proposes a probability that a breast cancer is HER2-positive from its H&E slide, for a pathologist who reads it beside IHC and ISH, never instead of them.

What is in this repository

Path	What it holds
encoders/	ViT-S/16 tile encoders (22M parameters) at every training stage, with timm parameter names
checkpoints/	the same models exactly as trained, in our original layout (including the self-distillation heads), for audit
reader/her2-jepa2-t2y-s4/	the deployed reader: encoder, five CLAM-SB fold heads, and a manifest with architecture, tile recipe, operating point and validation
data/predictions/	per-slide predictions, by seed: TCGA-BRCA out of fold, Yale-HER2 external, TCGA-BRCA + CPTAC-BRCA under the published protocol
data/paper/	data behind the paper's tables and figures
CHECKSUMS.sha256	SHA-256 of every file

Encoders

File (encoders/)	Stage	TCGA-to-Yale AUC*
stage1_ssl-vits16-final.safetensors	self-supervised pretraining (DINOv2 + iBOT + KoLeo) on 98.6M TCGA tiles	0.589
stage2_distilled-hoptimus0-step25000.safetensors	distilled from H-optimus-0 (CLS + 4x4 patch targets), step 25,000	0.762
stage3a_jepa-round1.safetensors	one JEPA refinement round	0.798
stage3b_jepa-round2_deployed-encoder.safetensors	second JEPA round: the deployed encoder	0.812
r12a_distilled-1um-step25000.safetensors	stage 3b distilled again at 1.0 um/px (not adopted)	0.813**
r12b_distilled-1um-jepa6000.safetensors	r12a plus one JEPA round (not adopted)	0.812**

* CLAM-SB reader trained on 490 TCGA-BRCA slides, tested on 192 Yale-HER2 slides, mean of 10 seeds (our protocol). ** Under the published reader's protocol (TCGA + CPTAC training, all tiles, 40 attention-MIL models); the deployed encoder scores 0.809 there.

Load an encoder (timm)

import timm
from safetensors.torch import load_file

enc = timm.models.vision_transformer.VisionTransformer(
    img_size=224, patch_size=16, embed_dim=384, depth=12, num_heads=6, mlp_ratio=4,
    qkv_bias=True, init_values=1.0, class_token=True, no_embed_class=False, reg_tokens=0,
    global_pool="token", fc_norm=False, num_classes=0)
enc.load_state_dict(load_file("encoders/stage3b_jepa-round2_deployed-encoder.safetensors"), strict=True)
enc.eval()
# input: float32 [N, 3, 224, 224], RGB, ImageNet mean (0.485, 0.456, 0.406) and std (0.229, 0.224, 0.225)
# output: [N, 384], the class token after the final LayerNorm

Tiles for our reader: 128 um squares read at 0.5 um/px (the coarsest pyramid level at or below 0.5 um/px), resized to 224 px by nearest neighbour, tissue only; up to 400 tiles per slide, taken at an even stride across the section. The manifest in reader/ gives every step.

The deployed reader

Five CLAM-SB heads (5-fold cross-validation on 490 TCGA-BRCA slides, seed 4) read one bag of 400 tile vectors. The reader's probability is the mean of the five heads' softmax probabilities; at or above 0.27228 it proposes "positive". That threshold comes from TCGA-BRCA's out-of-fold predictions, and a laboratory should refit it on its own slides, because scores shift between hospitals.

Test set (never trained on)	AUC
Yale-HER2, 192 slides (5-model ensemble, seed 4)	0.815 [0.753, 0.876]
BCNB, 1,058 core biopsies (mean of 10 seeds)	0.628
HEROHE test split, 150 slides (mean of 10 seeds)	0.746
HEROHE IHC 2+ cases, 85 slides (amplified vs not)	0.761

At its threshold on Yale-HER2: sensitivity 0.72, specificity 0.79. On a 12-core CPU server it reads a slide in 5.2 to 5.6 s.

Limitations

• Its 490 TCGA-BRCA training slides were size-biased: negatives carry less tissue than positives. This inflates the Yale AUC by about 0.04; on representative training sets our readers reach about 0.81.
• Probabilities run low at hospitals other than TCGA's; set the threshold per laboratory.
• On equivocal IHC 2+ cases (AUC 0.76 on HEROHE) it can help order and prioritise ISH. It does not replace ISH.
• The best published reader (H-optimus-0, 1.1B parameters, Yale AUC 0.907) remains well ahead. The gap lies in the encoder.

Training data and attributions

• Encoder: TCGA diagnostic slides (NCI Genomic Data Commons, open access). Distillation teacher: H-optimus-0 (Bioptimus), Apache-2.0. See NOTICE.
• Reader heads: 490 TCGA-BRCA slides with HER2 status from GDC clinical data.
• Evaluation cohorts: Yale-HER2 (TCIA HER2-TUMOR-ROIS, CC BY 4.0), CPTAC-BRCA (open), HEROHE (CC BY-NC-ND 3.0, evaluation only) and BCNB (non-commercial, evaluation only). Per-slide data for HEROHE and BCNB are not published here; only aggregate numbers appear in the paper.

Licences

• Model weights (encoders/, checkpoints/, reader/): CC BY-NC 4.0 (LICENSE-weights.md).
• Data (data/): CC BY 4.0 (LICENSE-data.md).
• The distillation teacher's Apache-2.0 notice: NOTICE.

Citation

@misc{tran2026her2reader,
  title  = {A small HER2 reader for routine H\&E slides: real-world deployment},
  author = {Tran, Thang and Dang, Lan},
  year   = {2026},
  note   = {Model weights and data: https://huggingface.co/cloudkites/her2-reader}
}