scRM S-96

scRM is a single-cell foundation model trained directly on sequencing reads. Its input is raw per-(cell barcode, UMI) sequence: there is no aligner, no reference genome, no gene annotation and no count matrix anywhere on the input side. A cell embedding is read out through a hierarchy of attention pools over bytes, contigs, UMIs and windows. The model holds 1,182,720 parameters.

Preprint: He et al., scRM: A single-cell foundation model trained on scRNA-seq reads, bioRxiv 2026. Code: github.com/altoslabs/scRM.

Checkpoints

name	trained on	steps	parameters
scRM-S96	3,000 cells from four public mouse 10x 3′ datasets	8,947	1,182,720
scRM-S96-sc_mixology	sc_mixology's own cells at the same 3,000-cell cap	8,947	1,182,720

Both are the same architecture at the same recipe and differ only in what they read. scRM-S96 is the reported model, benchmarked zero-shot. scRM-S96-sc_mixology is the dataset-specific arm, used only for the gene-detection comparison on sc_mixology.

Files

Four files per checkpoint:

• <name>.safetensors — 103 weight tensors.
• <name>.pt — the same weights as a torch state_dict.
• <name>.eval.pt — that state_dict together with the arch, readout and arch_sha fields the evaluation loader requires. This is the file to load.
• <name>.config.json — the arch and readout dicts, the step, the seed and the parameter count.

Loading

embed() takes no architecture arguments; the architecture comes from the checkpoint, and Checkpoint.load recomputes arch_sha over it and refuses a checkpoint whose weights and declared shape disagree.

from scrm_eval.embed import Checkpoint
from scrm_model.evalio import model_factory

ckpt = Checkpoint.load("scRM-S96.eval.pt", model_factory=model_factory)

Run with PYTHONPATH=model:eval:target/py from a checkout of the code repository.

Architecture

A local trunk of 4 layers and a global trunk of 6 layers, both at width 96 with 4 heads, rotary position embeddings at theta = 10000 over a maximum of 1,600 positions, and a projector of hidden width 256 onto 96 dimensions. The readout pools bytes into contigs, contigs into UMIs, UMIs into windows, and two views into a cell, each pool a masked seed-attention pool. Both checkpoints carry arch_sha = 7c4e4ffc9d304902eaaaeacc4ebac858edb48d5f5078d7f7245d71e7f7359bba, which gates evaluation compatibility.

Training

Trained as a JEPA. The pretraining corpora are four public mouse 10x 3′ datasets — mouse_brain_5k, neuron_1k_v3, neurons_5k and sod1_spinal_cord — all downloadable without registration and released under CC BY 4.0. The reported model draws its 8,947 steps from 3,000 of those cells. Seed 20260903, torch 2.11.0+cu128.

Scope of the evidence

The reported numbers are zero-shot on held-out human and fly data; none of the four pretraining datasets appears in any evaluation. A read-level representation retains nucleotide content and can therefore carry germline variation, so these embeddings are better governed as genotype data than as expression matrices.

Citation

@article{he2026scrm,
  title    = {scRM: A single-cell foundation model trained on scRNA-seq reads},
  author   = {He, D. and Patro, R. and Nassar, M. and Wu, H. and Yan, Q. and
              Xu, G. and Torkenczy, K. and Liu, J. and Tyanova, S.},
  journal  = {bioRxiv},
  year     = {2026},
  doi      = {10.64898/2026.10.24.761590},
  url      = {https://www.biorxiv.org/content/10.64898/2026.10.24.761590v1}
}