Package: rna-fm
RNA-FMThis repository contains codes and pre-trained models forRNA foundation model (RNA-FM).RNA-FM outperforms all tested single-sequence RNA language models across a variety of structure prediction tasks as well as several function-related tasks.You can find more details aboutRNA-FMin our paper,"Interpretable RNA Foundation Model from Unannotated Data for Highly Accurate RNA Structure and Function Predictions" (Chen et al., 2022).Citation@article{chen2022interpretable,title={Interpretable rna foundation model from unannotated data for highly accurate rna structure and function predictions},author={Chen, Jiayang and Hu, Zhihang and Sun, Siqi and Tan, Qingxiong and Wang, Yixuan and Yu, Qinze and Zong, Licheng and Hong, Liang and Xiao, Jin and King, Irwin and others},journal={arXiv preprint arXiv:2204.00300},year={2022}}Table of contentsSetup EnvironmentPre-trained ModelsUsageRNA-FM Embedding GenerationRNA Secondary Structure PredictionServerQuick StartCitationsLicenseCreate Environment with CondaFirst, download the repository and create the environment.git clone https://github.com/mydkzgj/RNA-FM.git
cd ./RNA-FM
conda env create -f environment.ymlThen, activate the "RNA-FM" environment and enter into the workspace.conda activate RNA-FM
cd ./redevelopAccess pre-trained models.Download pre-trained models fromthis gdrive linkand place the pth files into thepretrainedfolder.Apply RNA-FM with Existing Scripts.1. Embedding Extraction.python launch/predict.py --config="pretrained/extract_embedding.yml" \
--data_path="./data/examples/example.fasta" --save_dir="./resuts" \
--save_frequency 1 --save_embeddingsRNA-FM embeddings with shape of (L,640) will be saved in the$save_dir/representations.2. Downstream Prediction - RNA secondary structure.python launch/predict.py --config="pretrained/ss_prediction.yml" \
--data_path="./data/examples/example.fasta" --save_dir="./resuts" \
--save_frequency 1The predicted probability maps will be saved in form of.npyfiles, and the post-processed binary predictions will be saved in form of.ctfiles. You can find them in the$save_dir/r-ss.3. Online Version - RNA-FM server.If you have any trouble with the deployment of the local version of RNA-FM, you can access its online version from this link,RNA-FM server.
You can easily submit jobs on the server and download results from it afterwards, without setting up environment and occupying any computational resources.Quick Start for Further Development.PyTorch is the prerequisite package which you must have installed to use this repository.
You can installrna-fmin your own environment with the following pip command if you just want to
use the pre-trained language model.
you can either install rna-fm from PIPY:pip install rna-fmor installrna-fmfrom github:cd ./RNA-FM
pip install .After installation, you can load the RNA-FM and extract its embeddings with the following code:import torch
import fm

# Load RNA-FM model
model, alphabet = fm.pretrained.rna_fm_t12()
batch_converter = alphabet.get_batch_converter()
model.eval()  # disables dropout for deterministic results

# Prepare data
data = [
    ("RNA1", "GGGUGCGAUCAUACCAGCACUAAUGCCCUCCUGGGAAGUCCUCGUGUUGCACCCCU"),
    ("RNA2", "GGGUGUCGCUCAGUUGGUAGAGUGCUUGCCUGGCAUGCAAGAAACCUUGGUUCAAUCCCCAGCACUGCA"),
    ("RNA3", "CGAUUCNCGUUCCC--CCGCCUCCA"),
]
batch_labels, batch_strs, batch_tokens = batch_converter(data)

# Extract embeddings (on CPU)
with torch.no_grad():
    results = model(batch_tokens, repr_layers=[12])
token_embeddings = results["representations"][12]More tutorials can be found fromhttps://ml4bio.github.io/RNA-FM/. The related notebooks are stored in thetutorialsfolder.CitationsIf you find the models useful in your research, we ask that you cite the relevant paper:For RNA-FM:@article{chen2022interpretable,title={Interpretable rna foundation model from unannotated data for highly accurate rna structure and function predictions},author={Chen, Jiayang and Hu, Zhihang and Sun, Siqi and Tan, Qingxiong and Wang, Yixuan and Yu, Qinze and Zong, Licheng and Hong, Liang and Xiao, Jin and King, Irwin and others},journal={arXiv preprint arXiv:2204.00300},year={2022}}The model of this code builds on theesmsequence modeling framework.
And we usefairseqsequence modeling framework to train our RNA language modeling.
We very appreciate these two excellent works!LicenseThis source code is licensed under the MIT license found in theLICENSEfile
in the root directory of this source tree.