BiGamba (MEM, step 44000)
Repository: micanonsens/bigamba-cons_only-step44000
The Gamba models are a family of DNA language models from the ArGamba paper that jointly model DNA sequence and evolutionary rate information.
This repository includes the model weights and code for the Caduceus-based bidirectional variant model (BiGamba) trained to predict the Zoonomia 241-mammalian alignment-derived PhyloP scores using the Masked Evolutionary rate Modelling (MEM) task. For more details, see the GitHub repo.
Model family
All Gamba family models have checkpoints available at 44,000 steps:
| Checkpoint name | Architecture | Training task |
|---|---|---|
| ArGamba-dual | ArGamba (Jamba autoregressive) | NTP + CEP |
| ArGamba-seq_only | ArGamba (Jamba autoregressive) | NTP |
| ArGamba-cons_only | ArGamba (Jamba autoregressive) | CEP |
| BiGamba-dual | BiGamba (Mamba bidirectional) | MLM + MEM |
| BiGamba-seq_only | BiGamba (Mamba bidirectional) | MLM |
| BiGamba-cons_only | BiGamba (Mamba bidirectional) | MEM |
Load
from transformers import AutoModel
model = AutoModel.from_pretrained(
"micanonsens/bigamba-cons_only-step44000",
trust_remote_code=True
)
Notes
- This repository includes custom modeling code;
trust_remote_code=Trueis required. - Ensure your environment has the necessary project dependencies installed (see GitHub).
- Downloads last month
- 12