Varformer
Varformer is a multimodal transformer for population-scale drug target and variant prioritization. It learns contextual representations of the missense variants observed in a population and integrates them with gene-level biological features through cross-modal attention. It is trained with a non-negative positive-unlabeled objective for the gene score, and an allele-frequency objective for a per-variant score.
Code, documentation and installation: https://github.com/aaronwtr/Varformer
Models in this repository
The 25 models of the paper. Each was trained with five seeds (13, 23, 42, 85, 482), with the variant head (variant_loss_weight 0.1, pooled binary cross-entropy) and 32-bit precision.
| Folder | Cohort | Variant source | Gene inputs | Use in the paper |
|---|---|---|---|---|
afr/ |
African / African American | gnomAD v4.1 | without geneticConstraint |
gene prioritization |
amr/ |
Admixed American | gnomAD v4.1 | without geneticConstraint |
gene prioritization |
nfe/ |
Non-Finnish European | gnomAD v4.1 | without geneticConstraint |
gene prioritization |
sas/ |
South Asian | Genes & Health | without geneticConstraint |
gene prioritization |
sas-gnh/ |
South Asian | Genes & Health | with geneticConstraint |
variant scores in the Genes & Health analyses |
Each model has two files:
seed<N>.safetensors: the model weights only. The files contain no training data, no genotypes and no individual-level information.seed<N>.json: the training settings, the input dimensions, the class prior, the decision threshold, the selected epoch, the validation Spearman correlation, and SHA-256 checksums of the weights and of the source checkpoint.
index.json lists all 25 models.
Use
from varformer import Varformer
model = Varformer.from_pretrained("nfe", seed="best")
predictions = model.predict(genes=["ENSG00000141510"])
variant_scores = model.score_variants(genes=["ENSG00000141510"])
from_pretrained checks the weights against their checksum. The model inputs are built from local copies of the input data (gnomAD or Genes & Health variants, AlphaMissense scores, Open Targets, Human Protein Atlas and GTEx features); see the data sources in the code repository. Genes & Health data are available only to approved researchers.
The gene score ranks genes by target-prioritization evidence; it is not a calibrated probability of clinical success. The variant score ranks variants for research prioritization; it is not validated for clinical variant classification.
License
The weights are released under CC BY-NC-SA 4.0: attribution, non-commercial use only, and the same license for derived works. The models take AlphaMissense scores as input, and AlphaMissense is released under the same license. The code is MIT-licensed. For commercial use, train a model with another pathogenicity predictor; the code repository describes how.
Citation
The paper citation will be added when the preprint is available. Until then, please cite the software release described in the code repository.