AIDO.RNA 1.6B

AIDO.RNA is a 1.6B parameter RNA foundation model trained on 42 million non-coding RNA sequences at single-nucleotide resolution. It achieves state-of-the-art performance on a comprehensive set of tasks, including RNA secondary structure prediction, mRNA-related tasks, RNA function prediction tasks, and RNA inverse folding.

Model architectural details

TODO

Pre-training data

TODO

Downstream evaluation

TODO

How to Use

Build any downstream models from this backbone

Get RNA sequence embedding

from genbio_finetune.tasks import Embed
model = Embed.from_config({"model.backbone": "rnafm"})
collated_batch = model.collate({"sequences": ["ACGT", "ACGT"]})
embedding = model(collated_batch)
print(embedding.shape)
print(embedding)

Sequence-level classification

import torch
from genbio_finetune.tasks import SequenceClassification
model = SequenceClassification.from_config({"model.backbone": "rnafm", "model.n_classes": 2})
collated_batch = model.collate({"sequences": ["ACGT", "ACGT"]})
logits = model(collated_batch)
print(logits)
print(torch.argmax(logits, dim=-1))

Token-level classification

import torch
from genbio_finetune.tasks import TokenClassification
model = TokenClassification.from_config({"model.backbone": "rnafm", "model.n_classes": 3})
collated_batch = model.collate({"sequences": ["ACGT", "ACGT"]})
logits = model(collated_batch)
print(logits)
print(torch.argmax(logits, dim=-1))

Pairwise token-level classification

@Sazan TODO

Sequence-level regression

from genbio_finetune.tasks import SequenceRegression
model = SequenceRegression.from_config({"model.backbone": "rnafm"})
collated_batch = model.collate({"sequences": ["ACGT", "ACGT"]})
logits = model(collated_batch)
print(logits)

Or use our one-liner CLI to finetune or evaluate any of the above!

gbft fit --model SequenceClassification --model.backbone rnafm --data SequenceClassification --data.path <hf_or_local_path_to_your_dataset>
gbft test --model SequenceClassification --model.backbone rnafm --data SequenceClassification --data.path <hf_or_local_path_to_your_dataset>

For more information, visit: Model Generator

Citation

Please cite AIDO.RNA using the following BibTeX code:

@inproceedings{ellington2024accurate, title={Accurate and General {DNA} Representations Emerge from Genome Foundation Models at Scale}, author={Caleb Ellington, Ning Sun, Nicholas Ho, Tianhua Tao, Sazan Mahbub, Yonghao Zhuang, Hongyi Wang, Eric P. Xing, Le Song}, booktitle={NeurIPS 2024 Workshop on AI for New Drug Modalities}, year={2024} }

License

@Hongyi TODO