Evo2-1B-Base (Transformers port)

Unofficial Transformers port of the official Evo 2 1B base checkpoint (evo2_1b_base). All model, data, and research credit goes to the Evo 2 team at Arc Institute and collaborators. Original project: https://github.com/ArcInstitute/evo2

Weights here were converted from the official Vortex checkpoint with no retraining. The architecture is reimplemented in plain PyTorch so the model loads through AutoModelForCausalLM without Vortex, Transformer Engine, or custom kernels. Conversion code: https://github.com/Aquiles-ai/Evo2-transformers

Model details

Evo 2 models DNA at single nucleotide resolution with a byte level tokenizer (vocab 512, one token per nucleotide). This variant mixes Hyena convolutions (short, medium, and implicit long filters) with grouped query attention plus RoPE.

Item	Value
Parameters	1B
Layers	25 (4 attention, 21 Hyena)
Hidden size	1920
Attention heads	15
MLP size	5120
Context length	8,192 tokens
Tokenizer	Byte level, vocab 512
Weight dtype	bf16, with poles, residues, and RoPE buffers in fp32

Usage

Requires trust_remote_code=True (modeling files are vendored in this repo).

from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("Aquiles-ai/Evo2-1B-Base", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("Aquiles-ai/Evo2-1B-Base", trust_remote_code=True)
model.eval()

Score a sequence:

import torch

ids = torch.tensor([tok.vortex_tokenize("ACGT")])
with torch.inference_mode():
    out = model(ids)
print(out.logits.shape)

Generate:

import torch

ids = torch.tensor([tok.vortex_tokenize("ACGT")])
with torch.inference_mode():
    gen = model.generate(ids, max_new_tokens=400, do_sample=True,
                         temperature=1.0, top_k=4, use_cache=False)
print(tok.vortex_detokenize(gen[0].tolist()))

Limitations

This port is less efficient than the original Vortex implementation. It has no FlashAttention, no FP8 path, no fused kernels, and no recurrent state decoding, so generation recomputes the full prefix at each step and runs slowly past a few hundred tokens. For large scale inference, use the original stack or NVIDIA NIM.

Evaluation

No new evaluation was run for this upload. For reference, the original paper reports 64.9 mean amino acid recovery for the 1B base model on the prokaryote gene completion panel (50 generations per gene). The port repo includes a script that reproduces that panel: https://github.com/Aquiles-ai/Evo2-transformers

Citation

@article{Brixi2026,
  author  = {Brixi, Garyk and Durrant, Matthew G. and Ku, Jerome and Naghipourfar, Mohsen and Poli, Michael and Sun, Gwanggyu and Brockman, Greg and Chang, Daniel and Fanton, Alison and Gonzalez, Gabriel A. and King, Samuel H. and Li, David B. and Merchant, Aditi T. and Nguyen, Eric and Ricci-Tam, Chiara and Romero, David W. and Schmok, Jonathan C. and Taghibakhshi, Ali and Vorontsov, Anton and Yang, Brandon and Deng, Myra and Gorton, Liv and Nguyen, Nam and Wang, Nicholas K. and Pearce, Michael T. and Simon, Elana and Adams, Etowah and Amador, Zachary J. and Ashley, Euan A. and Baccus, Stephen A. and Dai, Haoyu and Dillmann, Steven and Ermon, Stefano and Guo, Daniel and Herschl, Michael H. and Ilango, Rajesh and Janik, Ken and Lu, Amy X. and Mehta, Reshma and Mofrad, Mohammad R. K. and Ng, Madelena Y. and Pannu, Jaspreet and R{\'e}, Christopher and St. John, John and Sullivan, Jeremy and Tey, Joseph and Viggiano, Ben and Zhu, Kevin and Zynda, Greg and Balsam, Daniel and Collison, Patrick and Costa, Anthony B. and Hernandez-Boussard, Tina and Ho, Eric and Liu, Ming-Yu and McGrath, Thomas and Powell, Kimberly and Pinglay, Sudarshan and Burke, Dave P. and Goodarzi, Hani and Hsu, Patrick D. and Hie, Brian L.},
  title   = {Genome modelling and design across all domains of life with Evo 2},
  journal = {Nature},
  year    = {2026},
  doi     = {10.1038/s41586-026-10176-5},
  url     = {https://doi.org/10.1038/s41586-026-10176-5}
}

License

Apache-2.0. Original Evo 2 weights and code remain property of their owners under their original terms.