VOICE-23M

Predict spatial single-cell gene expression from H&E images using a LoRA-adapted image encoder and a spatial decoder. This model predicts 6,029 genes and can also extract 1,536-dimensional cell embeddings.

Resources

• Code: GitHub
• Paper: arXiv:2608.08366

Installation

git clone https://github.com/Luosanj/VOICE_HE_ST_Pred.git
cd VOICE_HE_ST_Pred
pip install -r requirements.txt

For automatic segmentation, also install cellpose>=4.0 and opencv-python-headless. SVS, NDPI, and MRXS images require the OpenSlide system library and openslide-python.

Download the model

Download the model files from this repository. Obtain access to the image encoder separately through MahmoodLab/UNI2-h and authenticate with your own Hugging Face account.

Choose your own Hugging Face cache location before downloading. Use the same VOICE_HF_HOME setting when running prediction:

export VOICE_HF_HOME=/path/to/huggingface/cache
export HF_HOME="$VOICE_HF_HOME"
hf auth login

python - <<'PYTHON'
from huggingface_hub import snapshot_download

snapshot_download("zchx/VOICE-23M", local_dir="models/voice23m",
                  allow_patterns=["stage*.safetensors", "config.json", "genes.tsv", "LICENSE"])
snapshot_download("MahmoodLab/UNI2-h")
PYTHON

File	Contents
stage1.safetensors	Image adapters, image–RNA projection towers, and RNA feature normalization
stage2.safetensors	Expression-trained image adapters, spatial decoder, and expression head
config.json	Model architecture and image geometry
genes.tsv	Ordered output gene symbols and indices
bank/TISSUE/	Tissue-pooled reference cells in compressed HDF5 shards
beta/TISSUE.json	Existing gene-wise fusion weights for supported tissues

The UNI2-h base weights are downloaded separately under their own terms. Prediction loads the Stage-1 adapters and overlays the trained Stage-2 adapters.

Prediction

Use your own image, model, and output paths. For a whole-slide image, first segment the cells or provide existing cell boundaries:

python predict/segment.py --image /path/to/slide.svs --mpp 0.25 \
  --out /path/to/cells.npz
python predict/predict.py --image /path/to/slide.svs \
  --cells /path/to/cells.npz --mpp 0.25 \
  --release models/voice23m --out /path/to/prediction.h5ad \
  --save_embeddings

0.25 is an example resolution in microns per pixel; supply the actual resolution of your image. Cell crops cover 55 µm and are resized to 224×224 pixels. Existing cell boundaries must use the source image's pixel coordinates.

For a prepared slide containing cell crops and boundaries:

python predict/predict.py --prepared /path/to/prepared_slide \
  --release models/voice23m --out /path/to/prediction.h5ad \
  --save_embeddings

Prediction requires no measured expression for the target slide. CUDA is the default device. Use --workers to control image-loading workers and --embeddings_only to extract cell embeddings without expression predictions.

Reference retrieval and fusion

The bank contains all 23,232,915 training reference cells, pooled and shuffled within tissue types. Features and expression remain paired. Features are L2-normalized and expression was standardized within each reference before pooling; no additional z-score is applied at prediction. Original slide names, cell identifiers, coordinates and source paths are omitted.

Tissue	Cells	Bank size (GiB)
bone	344,181	1.05
brain	816,519	2.60
breast	7,656,449	24.61
cervix	836,989	3.18
colon	2,169,152	6.86
heart	26,364	0.08
kidney	154,068	0.48
liver	401,890	1.26
lung	6,188,966	23.67
lymph_node	708,981	3.35
ovary	813,009	3.90
pancreas	331,666	1.04
prostate	193,000	0.84
skin	377,692	1.38
tonsil	2,213,989	6.88

Download only the tissue you need, for example lung:

python - <<'PYTHON'
from huggingface_hub import snapshot_download

snapshot_download("zchx/VOICE-23M", local_dir="models/voice23m",
                  allow_patterns=["bank/lung/*", "beta/lung.json"])
PYTHON
python predict/predict.py --image /path/to/slide.svs \
  --cells /path/to/cells.npz --mpp 0.25 --release models/voice23m \
  --bank models/voice23m/bank/lung --gate models/voice23m/beta/lung.json \
  --out /path/to/prediction.h5ad --save_embeddings

Existing fusion weights are supplied for breast, kidney, lung, ovary and pancreas. Genes without fitted fusion weights use the direct prediction. Other tissues can be used with --bank alone to save retrieval scores; X then contains direct predictions. Downloading the whole repository includes all tissues, so use allow_patterns for smaller downloads.

Use the released banks with independent query slides. Source-level exclusions are unavailable after pooling. To fit fusion weights or exclude particular reference slides, build a separate slide-resolved bank from your own references using the code repository.

Output

import anndata as ad

prediction = ad.read_h5ad("/path/to/prediction.h5ad")

Field	Contents
X	Direct expression predictions in log1p(mu) scale
var_names	Gene symbols in model-head order
obsm['spatial']	Cell centroids as x,y coordinates in source-image pixels
obsm['X_voice']	Cell embeddings, when --save_embeddings is supplied

With a bank, layers['R'] contains retrieved standardized reference expression. With both a bank and fusion weights, layers['A'] contains direct predictions, var['beta'] contains applied gene weights, and X contains beta*A + (1-beta)*R. These fused values are gene-wise scores.

See the code repository for input formats, training, and optional reference retrieval and fusion.

License

See LICENSE for code terms and BANK_DATA_TERMS.md for reference-bank terms and source credits. UNI2-h is subject to the separate terms of its authors.