Package: evo-model
Evo: DNA foundation modeling from molecular to genome scaleEvo is a biological foundation model capable of long-context modeling and design.
Evo uses theStripedHyena architectureto enable modeling of sequences at a single-nucleotide, byte-level resolution with near-linear scaling of compute and memory relative to context length.
Evo has 7 billion parameters and is trained on OpenGenome, a prokaryotic whole-genome dataset containing ~300 billion tokens.We describe Evo in the the paper“Sequence modeling and design from molecular to genome scale with Evo”and in theaccompanying blog post.We provide the following model checkpoints:Checkpoint NameDescriptionevo-1-8k-baseA model pretrained with 8,192 context. We use this model as the base model for molecular-scale finetuning tasks.evo-1-131k-baseA model pretrained with 131,072 context usingevo-1-8k-baseas the base model. We use this model to reason about and generate sequences at the genome scale.ContentsSetupRequirementsInstallationUsageHuggingFaceTogether APICitationSetupRequirementsEvo is based onStripedHyena.Evo usesFlashAttention-2, which may not work on all GPU architectures.
Please consult theFlashAttention GitHub repositoryfor the current list of supported GPUs.Make sure to install the correctPyTorch version is installedon your system.InstallationYou can install Evo usingpippipinstallevo-modelor directly from the GitHub sourcegitclonehttps://github.com/evo-design/evo.gitcdevo/
pipinstall.We recommend that you install the PyTorch library first, before installing all other dependencies (due to dependency issues of theflash-attnlibrary; see, e.g., thisissue).One of ourexample scripts, demonstrating how to go from generating sequences with Evo to folding proteins (scripts/generation_to_folding.py), further requires the installation ofprodigal. We have created anenvironment.ymlfile for this:condaenvcreate-fenvironment.yml
condaactivateevo-designUsageBelow is an example of how to download Evo and use it locally through the Python API.fromevoimportEvoimporttorchdevice='cuda:0'evo_model=Evo('evo-1-131k-base')model,tokenizer=evo_model.model,evo_model.tokenizermodel.to(device)model.eval()sequence='ACGT'input_ids=torch.tensor(tokenizer.tokenize(sequence),dtype=torch.int,).to(device).unsqueeze(0)logits,_=model(input_ids)# (batch, length, vocab)print('Logits: ',logits)print('Shape (batch, length, vocab): ',logits.shape)An example of batched inference can be found inscripts/example_inference.py.We provide anexample scriptfor how to prompt the model and sample a set of sequences given the prompt.python-mscripts.generate\--model-name'evo-1-131k-base'\--promptACGT\--n-samples10\--n-tokens100\--temperature1.\--top-k4\--devicecuda:0We also provide anexample scriptfor using the model to score the log-likelihoods of a set of sequences.python-mscripts.score\--input-fastaexamples/example_seqs.fasta\--output-tsvscores.tsv\--model-name'evo-1-131k-base'\--devicecuda:0HuggingFaceEvo is integrated withHuggingFace.fromtransformersimportAutoConfig,AutoModelForCausalLMmodel_name='togethercomputer/evo-1-8k-base'model_config=AutoConfig.from_pretrained(model_name,trust_remote_code=True)model_config.use_cache=Truemodel=AutoModelForCausalLM.from_pretrained(model_name,config=model_config,trust_remote_code=True,)Together APIEvo is also available via an API byTogetherAI.importopenaiimportos# Fill in your API information here.client=openai.OpenAI(api_key=TOGETHER_API_KEY,base_url='https://api.together.xyz',)chat_completion=client.chat.completions.create(messages=[{"role":"system","content":""},{"role":"user","content":"ACGT",# Prompt the model with a sequence.}],model="togethercomputer/evo-1-131k-base",max_tokens=128,# Sample some number of new tokens.logprobs=True)print(chat_completion.choices[0].logprobs.token_logprobs,chat_completion.choices[0].message.content)CitationPlease cite the following preprint when referencing Evo.@article{nguyen2024sequence,
   author = {Eric Nguyen and Michael Poli and Matthew G. Durrant and Armin W. Thomas and Brian Kang and Jeremy Sullivan and Madelena Y. Ng and Ashley Lewis and Aman Patel and Aaron Lou and Stefano Ermon and Stephen A. Baccus and Tina Hernandez-Boussard and Christopher Ré and Patrick D. Hsu and Brian L. Hie},
   journal = {Arc Institute manuscripts},
   title = {Sequence modeling and design from molecular to genome scale with Evo},
   url = {https://arcinstitute.org/manuscripts/Evo},
   year = {2024},
}