BurnyCoder/pcam-hibou-hillclimb

PCam hill-climbing of Hibou-L: heads and adapters

Classification heads and LoRA adapters for PatchCamelyon metastasis detection, trained on top of the Hibou-L pathology foundation model (Nechaev et al. 2024); the control heads use the other frozen backbones listed per arm and under "Licences and attribution". The data are the official PCam splits from Zenodo record 2546921 (CC0 per the PCam README; the Zenodo record lists MIT). They come from the pre-registered study at https://github.com/BurnyCoder/better-cancer-classification-with-machine-learning, where you will find the pre-registration, the code to rebuild every artifact, and all results with slide-level confidence intervals. The current paper release is included as paper.pdf.

Research use only. Not a medical device. Not for clinical decision-making.

Artifacts

Arm	Backbone(s)	Valid BA (seed-averaged)	Train OOF BA
control_color_histogram	—	0.8098	0.7870 (all train)
control_shuffled_labels	hibou_l	0.4731	0.5204 (all train)
H4_aug_control	hibou_l	0.9344	0.9731 (fold0)
S0_eva_exact_hibou_l	hibou_l	0.9396	—
S0_hibou_l_cls_linear	hibou_l	0.9367	0.9498 (all train)
S0_scan_dinov2_reg_l	dinov2_reg_l	0.8682	0.8797 (all train)
S0_scan_random_vitl_reg4	random_vitl_reg4	0.7902	0.7973 (all train)
S1_b_cls24_mean24	hibou_l	0.9353	0.9491 (all train)
S1_c_dinov2_last4	hibou_l	0.9316	0.9457 (all train)
S1_d_spread_6_12_18_24	hibou_l	0.9326	0.9469 (all train)
S1_e_best_plus_centre_tokens	hibou_l	0.9352	0.9496 (all train)
S1_f_best_plus_centre_view	hibou_l	0.9354	0.9497 (all train)
S2_logreg	hibou_l	0.9365	0.9510 (all train)
S2_mlp	hibou_l	0.9054	0.9312 (all train)
S3_d4_tta	hibou_l	0.9343	0.9522 (train64k)
S3b_oof_label_cleaning	hibou_l	0.9368	0.9525 (all train)
S5_lora	hibou_l	0.9444	0.9798 (fold0)

Each heads/<arm>/seed<k>.pt holds a torch state dict (module), the z-scoring mu/sd (or None), and the head kind. The input is the representation documented in the arm's JSON record (arms/<arm>.json). Train OOF BA is out-of-fold over the patch set in brackets: all training slides, the train-64k subset (S3), or train fold 0 (the fine-tuned model and its H4 control).

Fine-tuned model (lora/S5_lora/)

The adopted LoRA model (LoRA r=16 lr=0.0001, all 24 blocks, LP-FT, D4+HED, drop-path 0.1, EMA). Files: seed*.safetensors hold the trainable weights (LoRA A/B matrices of the attention projections in all 24 blocks plus the linear head; best EMA checkpoint), lp_head.safetensors holds the LP-FT start head, and config.json holds the pinned Hibou-L revision, PEFT settings and preprocessing. Rebuild it with the repository's pcam_hc.stage_c.new_model and load_state_dict(..., strict=False). Hibou-L itself is not redistributed: accept its licence on Hugging Face first.

Known issues of this run (found by code review after training): its training-set cleaning mask used predictions that saw fold-0 labels, its stain augmentation is stronger than "HED-light" suggests, and its lr/rank choice is close to arbitrary. Its validation result is unaffected by the first issue. Details are in the repository's reports/experiments/02_ladder_hibou_l.md.

Licences and attribution

• The base model weights are not included. Accept the Hibou-L licence on its Hugging Face page and download it yourself.
• dinov2_reg_l: facebook/dinov2-with-registers-large at e4c89a4 (frozen weights, not included; Apache-2.0). Licensed under the Apache License 2.0; derived artifacts keep attribution and the upstream notices.
• hibou_l: histai/hibou-L at 31a3b9f (frozen weights, not included; Apache-2.0). Licensed under the Apache License 2.0; derived artifacts keep attribution and the upstream notices.
• random_vitl_reg4: histai/hibou-L at 31a3b9f (architecture config only, random weights; Apache-2.0). Random weights; only the architecture config is used.
• These derived artifacts are released under Apache-2.0 (LICENSE, NOTICE). Changes: newly trained classification heads / LoRA adapters.