Evo2-7B (Transformers port)

Unofficial Transformers port of the official Evo 2 7B checkpoint (evo2_7b, 1M context). All model, data, and research credit goes to the Evo 2 team at Arc Institute and collaborators. Original project: https://github.com/ArcInstitute/evo2

Weights here were converted from the official Vortex checkpoint with no retraining. The architecture is reimplemented in plain PyTorch so the model loads through AutoModelForCausalLM without Vortex, Transformer Engine, or custom kernels. Conversion code: https://github.com/Aquiles-ai/Evo2-transformers

Model details

Evo 2 models DNA at single nucleotide resolution with a byte level tokenizer (vocab 512, one token per nucleotide). This variant mixes Hyena convolutions (short, medium, and implicit long filters) with grouped query attention plus RoPE, using interpolated rotary positions to reach 1M context.

Item	Value
Parameters	7B
Layers	32 (5 attention, 27 Hyena)
Hidden size	4096
Attention heads	32
MLP size	11264
Context length	1,048,576 tokens
Tokenizer	Byte level, vocab 512
Weight dtype	bf16, with poles, residues, and RoPE buffers in fp32

The model needs about 14 GB of GPU memory in bf16 before KV and activations. The 1M context is supported by config, but in this port long inputs are slow and memory heavy (see Limitations).

Usage

Requires trust_remote_code=True (modeling files are vendored in this repo).

from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("Aquiles-ai/Evo2-7B", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("Aquiles-ai/Evo2-7B", trust_remote_code=True)
model.eval()

Score a sequence:

import torch

ids = torch.tensor([tok.vortex_tokenize("ACGT")])
with torch.inference_mode():
    out = model(ids)
print(out.logits.shape)

Generate:

import torch

ids = torch.tensor([tok.vortex_tokenize("ACGT")])
with torch.inference_mode():
    gen = model.generate(ids, max_new_tokens=400, do_sample=True,
                         temperature=1.0, top_k=4, use_cache=True)
print(tok.vortex_detokenize(gen[0].tolist()))

generate uses the decoding cache by default (needs transformers>=5). With use_cache=False it recomputes the full prefix at each step, which is slow past a few hundred tokens.

Limitations

This port is less efficient than the original Vortex implementation. It has no FlashAttention, no FP8 path, and no fused kernels. Generation uses a decoding cache (needs transformers>=5); with use_cache=False it recomputes the full prefix at each step and runs slowly past a few hundred tokens. Long context forward passes run as full precision FFTs per Hyena channel, which is correct but heavy. For large scale or long context inference, use the original stack or NVIDIA NIM.

Evaluation

No new evaluation was run for this upload. See the original paper for reported 7B results: https://www.nature.com/articles/s41586-026-10176-5

Citation

@article{Brixi2026,
  author  = {Brixi, Garyk and Durrant, Matthew G. and Ku, Jerome and Naghipourfar, Mohsen and Poli, Michael and Sun, Gwanggyu and Brockman, Greg and Chang, Daniel and Fanton, Alison and Gonzalez, Gabriel A. and King, Samuel H. and Li, David B. and Merchant, Aditi T. and Nguyen, Eric and Ricci-Tam, Chiara and Romero, David W. and Schmok, Jonathan C. and Taghibakhshi, Ali and Vorontsov, Anton and Yang, Brandon and Deng, Myra and Gorton, Liv and Nguyen, Nam and Wang, Nicholas K. and Pearce, Michael T. and Simon, Elana and Adams, Etowah and Amador, Zachary J. and Ashley, Euan A. and Baccus, Stephen A. and Dai, Haoyu and Dillmann, Steven and Ermon, Stefano and Guo, Daniel and Herschl, Michael H. and Ilango, Rajesh and Janik, Ken and Lu, Amy X. and Mehta, Reshma and Mofrad, Mohammad R. K. and Ng, Madelena Y. and Pannu, Jaspreet and R{\'e}, Christopher and St. John, John and Sullivan, Jeremy and Tey, Joseph and Viggiano, Ben and Zhu, Kevin and Zynda, Greg and Balsam, Daniel and Collison, Patrick and Costa, Anthony B. and Hernandez-Boussard, Tina and Ho, Eric and Liu, Ming-Yu and McGrath, Thomas and Powell, Kimberly and Pinglay, Sudarshan and Burke, Dave P. and Goodarzi, Hani and Hsu, Patrick D. and Hie, Brian L.},
  title   = {Genome modelling and design across all domains of life with Evo 2},
  journal = {Nature},
  year    = {2026},
  doi     = {10.1038/s41586-026-10176-5},
  url     = {https://doi.org/10.1038/s41586-026-10176-5}
}

License

Apache-2.0. Original Evo 2 weights and code remain property of their owners under their original terms.