| # AIDO.RNA 1.6B | |
| AIDO.RNA is a 1.6B parameter RNA foundation model trained on 42 million non-coding RNA sequences at single-nucleotide resolution. It achieves state-of-the-art performance on a comprehensive set of tasks, including RNA secondary structure prediction, mRNA-related tasks, RNA function prediction tasks, and RNA inverse folding. | |
| <img src="https://cdn-uploads.huggingface.co/production/uploads/63008d4bc1e149ceaff724a3/mNqn5SKQFHxSby3E2dosE.png" alt="description" style="width:80%; height:auto;"> | |
| ## Model architectural details | |
| TODO | |
| ## Pre-training data | |
| TODO | |
| ## Downstream evaluation | |
| TODO | |
| ## How to Use | |
| Build any downstream models from this backbone | |
| ### Get RNA sequence embedding | |
| ``` | |
| from genbio_finetune.tasks import Embed | |
| model = Embed.from_config({"model.backbone": "rnafm"}) | |
| collated_batch = model.collate({"sequences": ["ACGT", "ACGT"]}) | |
| embedding = model(collated_batch) | |
| print(embedding.shape) | |
| print(embedding) | |
| ``` | |
| ### Sequence-level classification | |
| ``` | |
| import torch | |
| from genbio_finetune.tasks import SequenceClassification | |
| model = SequenceClassification.from_config({"model.backbone": "rnafm", "model.n_classes": 2}) | |
| collated_batch = model.collate({"sequences": ["ACGT", "ACGT"]}) | |
| logits = model(collated_batch) | |
| print(logits) | |
| print(torch.argmax(logits, dim=-1)) | |
| ``` | |
| ### Token-level classification | |
| ``` | |
| import torch | |
| from genbio_finetune.tasks import TokenClassification | |
| model = TokenClassification.from_config({"model.backbone": "rnafm", "model.n_classes": 3}) | |
| collated_batch = model.collate({"sequences": ["ACGT", "ACGT"]}) | |
| logits = model(collated_batch) | |
| print(logits) | |
| print(torch.argmax(logits, dim=-1)) | |
| ``` | |
| ### Pairwise token-level classification | |
| @Sazan TODO | |
| ### Sequence-level regression | |
| ``` | |
| from genbio_finetune.tasks import SequenceRegression | |
| model = SequenceRegression.from_config({"model.backbone": "rnafm"}) | |
| collated_batch = model.collate({"sequences": ["ACGT", "ACGT"]}) | |
| logits = model(collated_batch) | |
| print(logits) | |
| ``` | |
| Or use our one-liner CLI to finetune or evaluate any of the above! | |
| ``` | |
| gbft fit --model SequenceClassification --model.backbone rnafm --data SequenceClassification --data.path <hf_or_local_path_to_your_dataset> | |
| gbft test --model SequenceClassification --model.backbone rnafm --data SequenceClassification --data.path <hf_or_local_path_to_your_dataset> | |
| ``` | |
| For more information, visit: [Model Generator](https://github.com/genbio-ai/test) | |
| ## Citation | |
| Please cite AIDO.RNA using the following BibTeX code: | |
| @inproceedings{ellington2024accurate, | |
| title={Accurate and General {DNA} Representations Emerge from Genome Foundation Models at Scale}, | |
| author={Caleb Ellington, Ning Sun, Nicholas Ho, Tianhua Tao, Sazan Mahbub, Yonghao Zhuang, Hongyi Wang, Eric P. Xing, Le Song}, | |
| booktitle={NeurIPS 2024 Workshop on AI for New Drug Modalities}, | |
| year={2024} | |
| } | |
| ## License | |
| @Hongyi TODO | |