BiGamba (MLM + MEM, step 44000)
Repository: micanonsens/bigamba-dual-step44000
The Gamba models are a family of DNA language models from the ArGamba paper that jointly model DNA sequence and evolutionary rate information.
This repository includes the model weights and code for the Caduceus-based bidirectional variant model (BiGamba) trained to predict both the human genome sequence and the Zoonomia 241-mammalian alignment-derived PhyloP scores using both Masked Language Modelling (MLM) and Masked Evolutionary rate Modelling (MEM) tasks. For more details, see the GitHub repo.
Model family
All Gamba family models have checkpoints available at 44,000 steps:
| Checkpoint name | Architecture | Training task |
|---|---|---|
| ArGamba-dual | ArGamba (Jamba autoregressive) | NTP + CEP |
| ArGamba-seq_only | ArGamba (Jamba autoregressive) | NTP |
| ArGamba-cons_only | ArGamba (Jamba autoregressive) | CEP |
| BiGamba-dual | BiGamba (Mamba bidirectional) | MLM + MEM |
| BiGamba-seq_only | BiGamba (Mamba bidirectional) | MLM |
| BiGamba-cons_only | BiGamba (Mamba bidirectional) | MEM |
Load
from transformers import AutoModel
model = AutoModel.from_pretrained(
"micanonsens/bigamba-dual-step44000",
trust_remote_code=True
)
Notes
- This repository includes custom modeling code;
trust_remote_code=Trueis required. - Ensure your environment has the necessary project dependencies installed (see GitHub).
- Downloads last month
- 28