PhenoSeq: Image-Conditioned Diffusion for Single-Cell Transcriptomics

PhenoSeq is a Gaussian diffusion model that generates scGPT RNA-seq embeddings conditioned on ViT-L microscopy imaging features. Given fluorescence microscopy images of a cell or well, it predicts a 512-dimensional scGPT embedding representing the transcriptomic state of individual cells — enabling image-to-transcriptome translation at single-cell resolution.

Model summary

Property	Value
Input	ViT-L imaging features — (N, 5120) per sample (5 fluorescence channels × 1,024 dims)
Output	scGPT embeddings — (n_cells, 512)
Architecture	Cross-attention diffusion denoiser
Diffusion steps	1,000 (cosine schedule)
Inference steps	50 (DDIM, default)
Model parameters	~168 M
Training dataset	scGeneScope
Best val MSE loss	0.1683 (epoch 87)

Architecture

The denoiser uses a cross-attention transformer stack:

1. Imaging encoder — 2-layer self-attention transformer projects (B, N, 5120) → (B, N, 1024) context.
2. RNA + time input — noisy scGPT embedding and sinusoidal time embedding are projected to model_dim.
3. Cross-attention blocks (×6) — RNA queries attend to imaging context, with self-attention and adaptive layer normalization conditioned on the timestep.
4. Output projection — predicts noise ε ∈ ℝ^{512} for the denoising objective.

The diffusion process uses a cosine beta schedule over T=1,000 steps with EMA weight averaging (decay 0.9999).

Quick start

pip install torch numpy huggingface_hub
python example.py

import numpy as np
from pipeline import PhenoSeqPipeline

pipe = PhenoSeqPipeline.from_pretrained("Sentinal4D/PhenoSeq")

# img_features: ViT-L embeddings — (n_cells, n_imaging_cells=16, 5120)
img_features = np.random.randn(8, 16, 5120).astype(np.float32)
rna_predictions = pipe(img_features)   # → (8, 512)

Inputs and outputs

Input: imaging features

img_features must be ViT-L image embeddings extracted from 5 fluorescence channels using a ViT-L/14 backbone, resulting in 5 × 1,024 = 5,120 dimensions per imaging cell. Each sample/well typically provides N = 16 imaging cells (evenly spaced from the available pool) that form the conditioning context.

Shape: (B, N, 5120) where B is the number of target RNA cells to predict.

Imaging normalisation — img_norm.npz contains the per-feature mean and std computed from the training split of scGeneScope. These are applied automatically by PhenoSeqPipeline. If you work with a different dataset you will need to recompute and supply your own normalisation stats.

Output: RNA embeddings

Shape: (B, 512) — scGPT-space embeddings un-normalized back to the original scGPT embedding scale. These can be used directly for downstream tasks such as cell-type classification (see classify_improved.py), clustering, or trajectory inference.

Imaging normalisation stats

The pipeline requires img_norm.npz (per-feature mean and std from the training split). This file is distributed alongside best_model.pt in this repo. If you retrain or use different data, regenerate it:

python save_img_norm.py --config config.yaml --output img_norm.npz

Full inference on scGeneScope data

For large-scale inference over the cached scGeneScope data (.npz per sample):

# Fast (DDIM, 50 steps)
python infer.py --checkpoint best_model.pt --ddim_steps 50

# Val split only
python infer.py --checkpoint best_model.pt --split val --output_dir results/predictions

# Full DDPM sampling (slower, ~1000 steps)
python infer.py --checkpoint best_model.pt --ddim_steps 0

Output: one {Sample_ID}.npz per sample under results/predictions/, with key X of shape (n_cells, 512).

Downstream: cell-type classification

Predicted RNA embeddings can be evaluated with the included classifier:

python classify_improved.py

See PERFORMANCE_ANALYSIS.md for benchmark results.

Training

The model was trained from scratch on scGeneScope using:

python train.py --config config.yaml

Data

Training and evaluation data come from the scGeneScope dataset (Altos Labs):

• Imaging features: ViT-L/14 embeddings extracted from 5-channel fluorescence microscopy images, stored as .h5ad files.
• RNA-seq features: scGPT cell embeddings (512-dim) from paired single-cell RNA-seq, stored as .h5ad files.
• Samples are matched by Sample_ID at well level.

Prepare the local cache before training:

python prepare_data.py

Repository structure

PhenoSeq/
├── pipeline.py          ← self-contained inference pipeline (start here)
├── best_model.pt        ← trained checkpoint with EMA weights & RNA norm stats
├── img_norm.npz         ← imaging normalisation stats (mean/std, training split)
├── config.yaml          ← full training configuration
├── infer.py             ← batch inference over cached scGeneScope data
├── save_img_norm.py     ← helper to (re)generate img_norm.npz
├── train.py             ← training entry point
├── prepare_data.py      ← extract imaging/RNA features → .npz cache
├── models/
│   ├── denoiser.py      ← cross-attention denoiser
│   ├── diffusion.py     ← Gaussian diffusion process (forward + reverse)
│   └── lit_module.py    ← PyTorch Lightning training wrapper
└── data/
    └── dataset.py       ← paired imaging-RNA dataset & dataloader

Requirements

torch>=2.0
anndata>=0.10
numpy>=1.24
scipy>=1.10
PyYAML>=6.0
tqdm>=4.65
# Optional: huggingface_hub (for from_pretrained with a Hub repo id)

Install:

pip install -r requirements.txt

License

Apache 2.0 — see LICENSE.

Citation

If you use PhenoSeq, please cite our paper:

@inproceedings{naidoo2026phenoseq,
  title     = {Cell Painting Generates Single-Cell Transcriptomics via Conditional Diffusion},
  author    = {Naidoo, Reed and Hu, Jingyu and Tripodi, Giuseppe and Bakal, Chris and Chakraborti, Tapabrata},
  booktitle = {ICML 2026 Workshop on Multi-modal Foundation Models and Large Language Models for Life Sciences (FM4LS)},
  year      = {2026},
  url       = {https://openreview.net/forum?id=ACHMa1e8J1}
}

Please also cite the training dataset:

@dataset{scgenescope,
  author    = {Altos Labs},
  title     = {scGeneScope},
  year      = {2024},
  publisher = {HuggingFace},
  url       = {https://huggingface.co/datasets/altoslabs/scGeneScope}
}