MetacellFormer-SC

MetacellFormer-SC is a comparison model pretrained on individual-cell profiles using the same architecture and training objectives as MetacellFormer.

Usage

Install the companion MetacellFormer GitHub package with pip install . from its repository root. Load this downloaded model directory with the supplied API:

import torch
from metacellformer import GeneRankTokenizer, load_encoder, cell_embeddings

model_dir = "MODEL_DIRECTORY"
tokenizer = GeneRankTokenizer(model_dir)
model = load_encoder(model_dir, device="cpu")
# Small illustrative profile; replace with your gene IDs and raw counts.
gene_ids = tokenizer.genes[:4]
encoded = tokenizer.encode(gene_ids, [10, 5, 2, 1])
ids = torch.as_tensor(encoded["input_ids"]).unsqueeze(0)
mask = torch.as_tensor(encoded["attention_mask"]).unsqueeze(0)
embedding = cell_embeddings(model, ids, mask)
print(embedding.shape)  # torch.Size([1, 1024])

Provide all measured genes and their raw counts so normalization uses the whole profile. When supplying a feature subset, pass the full-profile total as total_counts_all_features to encode. Paper evaluation uses token type 0 and non-padding final-layer mean pooling, including CLS. The GitHub examples/02_embeddings.ipynb demonstrates inference on an evaluation matrix.

Files

model.safetensors, config.json, gene_vocabulary.json and gene_reference_medians.npy provide the model and inference resources. gene_reference_support.npy and training_resources/ support pretraining reproduction. Evaluation inputs and fixed contracts are supplied in MetacellFormer-evaluation. Checkpoint details are described in the manuscript Methods.

License

Model materials: CC BY 4.0. The GitHub code is licensed separately under Apache License 2.0.