albtad01/STHELAR-Adapt-CellViT-256-x40

STHELAR-Adapt for CellViT-256-x40

This repository contains verified, architecture-specific STHELAR-Adapt states for the CellViT-256-x40 backbone. STHELAR-Adapt specializes CellViT for nuclei instance segmentation and spatial-transcriptomics-informed cell typing in STHELAR 40× H&E images.

The official CellViT-256-x40 base checkpoint is required separately. It is not redistributed here.

Q/V LoRA and AdaptFormer specialize the pretrained encoder while the decoder body remains frozen and only the final nucleus heads are trained.

Model and target taxonomy

The adapter state uses the selected MSc-thesis PEFT configuration:

• Q/V LoRA with rank 8, alpha 8, and dropout 0;
• AdaptFormer with reduction 16 and GELU;
• trainable final NP, HV, and NT 1×1 heads;
• frozen pretrained encoder weights outside the inserted PEFT modules;
• frozen decoder body;
• frozen CellViT-256 auxiliary tissue classifier;
• required changed mutable normalization buffers included in the state.

The state contains 374,198 trainable parameters, or 0.7942% of the instantiated CellViT-256 PEFT model. This is a custom composite adapter format, not a drop-in standard PEFT/LoRA package.

STHELAR-derived foreground identities are grouped into five classes: Immune, Stromal, Epithelial, Melanocyte, and Other. Background is retained as the non-foreground class.

Required base checkpoint

Obtain CellViT-256-x40.pth from the official CellViT project. The expected SHA-256 is:

ee3986922fc500353db3d7692c566e19e1c694c8f10e47cf49dfd90160fc3b2b

The official base weights, FullFT weights, training checkpoints, optimizer state, datasets, and run logs are not part of this release.

Adapter cohorts

The manifest contains 32 unique verified payloads:

Cohort	Count	Scientific role
Complete-slide evaluation	20	KLT plus nine tissues, reciprocal Fold A/B, seed 42
All-slides deployment	10	KLT plus all nine tissues, seed 42
Supplemental verified KLT	2	Fold A/B seed-44 evaluation states

Evaluation adapters were trained without the reciprocal held-out complete slide. All-slides adapters use all designated slides with the established within-slide training/validation construction; they are deployment states and carry no held-out-slide performance claim. Supplemental states are kept separate from the seed-42 core.

The KLT headline aggregate uses all three training seeds. Two seed-43 CellViT-256 states remain excluded from this publication set because their historical source-config hash provenance is unresolved, even though their result rows remain part of the audited thesis aggregate.

Directory conventions:

adapters/complete_slide_v1/klt/{fold_a,fold_b}/seed42/
adapters/complete_slide_v1/klt/all_slides/seed42/
adapters/complete_slide_v1/tissue_specific/<tissue>/{fold_a,fold_b}/seed42/
adapters/complete_slide_v1/tissue_specific/<tissue>/all_slides/seed42/
adapters/supplemental_klt_replicates/{fold_a,fold_b}/seed44/

Each directory contains adapter_model.safetensors, adapter_config.json, checksums.json, and verification.json. The corresponding training config is recorded in adapter_manifest.csv and included under configs/.

One official architecture-matched base can be combined with one verified compact state at a time. CellViT-256 and SAM-H states are not interchangeable.

MSc thesis evaluation

The final predictive evaluation uses reciprocal complete-slide holdout. KLT values are equal-weight reciprocal-fold means after averaging each fold over seeds 42, 43, and 44. Nine-tissue values use seed 42 and equal weighting over the nine tissue-level reciprocal-fold means.

Evaluation	Method	bPQ	mPQ	F1 detection	F1 type
KLT complete-slide	PEFT	0.517	0.187	0.847	0.471
KLT complete-slide	FullFT	0.552	0.200	0.849	0.482
Nine-tissue complete-slide	PEFT	0.420	0.151	0.791	0.348
Nine-tissue complete-slide	FullFT	0.433	0.168	0.790	0.388

Nucleus detection is preserved particularly closely, while CellViT-256 shows a clearer PEFT bPQ deficit and ST-informed typing remains more difficult.

Systems and storage

Method	Trainable	Peak allocated VRAM	Serialized state
PEFT	0.7942%	2.040 GiB	1.548 MiB
FullFT	100%	2.667 GiB	0.174 GiB

For nine architecture-compatible domains, independent FullFT states require 1.569 GiB, compared with 0.188 GiB for one shared base plus nine PEFT states. Parameter efficiency does not imply universal runtime acceleration; timings depend on hardware, data movement, validation, checkpointing, and postprocessing.

Load a verified package

Install a compatible checkout of STHELAR-Adapt and its dependencies, then use the repository's verified release loader:

from utils.sthelar_release_loader import load_verified_adapter

model, evidence = load_verified_adapter(
    base_checkpoint="/path/to/CellViT-256-x40.pth",
    package_dir=(
        "/path/to/STHELAR-Adapt-CellViT-256-x40/"
        "adapters/complete_slide_v1/klt/fold_a/seed42"
    ),
    device="cpu",
)

The loader validates the base hash, payload hash, tensor inventory, expected trainable and mutable-buffer keys, and verification metadata before returning an eval-mode model.

To repeat the package-level checks and deterministic forward sanity test:

PYTHONPATH=/path/to/STHELAR-Adapt \
python scripts/verify_public_release_package.py \
  --base-checkpoint /path/to/CellViT-256-x40.pth \
  --package adapters/complete_slide_v1/klt/fold_a/seed42

The original finalization standard required state_reconstruction: exact_all_tensors and forward_verification: exact_all_output_tensors. See adapter_manifest.csv and verification_summary.json for the release-wide inventory.

Intended use and limitations

This release is intended for research, reproducibility, and evaluation of STHELAR-specific CellViT adaptation. It is not clinically validated and must not be used as a medical device.

• STHELAR contains few complete slides per tissue; reciprocal direction can materially affect results.
• Tissue-specific results use one seed per direction; KLT uses three.
• No patient-, site-, scanner-, or cohort-level generalization claim is made.
• ST-derived identities contain assignment uncertainty and are not interchangeable with morphology-only labels.
• Performance may change with stain, scanner, magnification, preprocessing, taxonomy, or base-checkpoint version.
• Comparisons with SAM-H are practical backbone comparisons, not a controlled scaling experiment.

Citation, provenance, and terms

Prepared STHELAR revision: e32a8cdd50eff2d38e237f3729e9ac85bbb5203b. The adapter manifest records payload hashes, roles, configs, and held-out metrics. No unpublished paper citation is asserted.

See CITATION.cff, LICENSE, and NOTICE. Applicable CellViT/HIPT-derived components include the upstream Apache 2.0 and Commons Clause conditions reproduced in LICENSE. Cite the upstream CellViT, STHELAR, Segment Anything, HIPT, and PanNuke resources as applicable. STHELAR data and the official base checkpoint retain their own terms and are not redistributed here.