micanonsens/argamba-seq_only-step44000

ArGamba (NTP, step 44000)

Repository: micanonsens/argamba-seq_only-step44000

The Gamba models are a family of DNA language models from the ArGamba paper that jointly model DNA sequence and evolutionary rate information.

This repository includes the model weights and code for the Jamba autoregressive variant model (ArGamba) trained to predict the human genome sequence using the Next Token Prediction (NTP) task. For more details, see the GitHub repo.

Model family

All Gamba family models have checkpoints available at 44,000 steps:

Checkpoint name	Architecture	Training task
ArGamba-dual	ArGamba (Jamba autoregressive)	NTP + CEP
ArGamba-seq_only	ArGamba (Jamba autoregressive)	NTP
ArGamba-cons_only	ArGamba (Jamba autoregressive)	CEP
BiGamba-dual	BiGamba (Mamba bidirectional)	MLM + MEM
BiGamba-seq_only	BiGamba (Mamba bidirectional)	MLM
BiGamba-cons_only	BiGamba (Mamba bidirectional)	MEM

Load

from transformers import AutoModel

model = AutoModel.from_pretrained(
    "micanonsens/argamba-seq_only-step44000",
    trust_remote_code=True
)

Notes

• This repository includes custom modeling code; trust_remote_code=True is required.
• Ensure your environment has the necessary project dependencies installed (see GitHub).