# AIDO.RNA 1.6B AIDO.RNA is a 1.6B parameter RNA foundation model trained on 42 million non-coding RNA sequences at single-nucleotide resolution. It achieves state-of-the-art performance on a comprehensive set of tasks, including RNA secondary structure prediction, mRNA-related tasks, RNA function prediction tasks, and RNA inverse folding. description ## Model architectural details TODO ## Pre-training data TODO ## Downstream evaluation TODO ## How to Use Build any downstream models from this backbone ### Get RNA sequence embedding ``` from genbio_finetune.tasks import Embed model = Embed.from_config({"model.backbone": "rnafm"}) collated_batch = model.collate({"sequences": ["ACGT", "ACGT"]}) embedding = model(collated_batch) print(embedding.shape) print(embedding) ``` ### Sequence-level classification ``` import torch from genbio_finetune.tasks import SequenceClassification model = SequenceClassification.from_config({"model.backbone": "rnafm", "model.n_classes": 2}) collated_batch = model.collate({"sequences": ["ACGT", "ACGT"]}) logits = model(collated_batch) print(logits) print(torch.argmax(logits, dim=-1)) ``` ### Token-level classification ``` import torch from genbio_finetune.tasks import TokenClassification model = TokenClassification.from_config({"model.backbone": "rnafm", "model.n_classes": 3}) collated_batch = model.collate({"sequences": ["ACGT", "ACGT"]}) logits = model(collated_batch) print(logits) print(torch.argmax(logits, dim=-1)) ``` ### Pairwise token-level classification @Sazan TODO ### Sequence-level regression ``` from genbio_finetune.tasks import SequenceRegression model = SequenceRegression.from_config({"model.backbone": "rnafm"}) collated_batch = model.collate({"sequences": ["ACGT", "ACGT"]}) logits = model(collated_batch) print(logits) ``` Or use our one-liner CLI to finetune or evaluate any of the above! ``` gbft fit --model SequenceClassification --model.backbone rnafm --data SequenceClassification --data.path gbft test --model SequenceClassification --model.backbone rnafm --data SequenceClassification --data.path ``` For more information, visit: [Model Generator](https://github.com/genbio-ai/test) ## Citation Please cite AIDO.RNA using the following BibTeX code: @inproceedings{ellington2024accurate, title={Accurate and General {DNA} Representations Emerge from Genome Foundation Models at Scale}, author={Caleb Ellington, Ning Sun, Nicholas Ho, Tianhua Tao, Sazan Mahbub, Yonghao Zhuang, Hongyi Wang, Eric P. Xing, Le Song}, booktitle={NeurIPS 2024 Workshop on AI for New Drug Modalities}, year={2024} } ## License @Hongyi TODO