DOI: 10.5281/zenodo.17130429
title: UltraSynBERT_All_Archives

description: Decoding the RNA interactome by UltraSynBERT

The official repository for the article "Decoding the RNA interactome by UltraSynBERT".

Overview

We introduce UltraSynBERT, an RNA language model designed to capture RNA binding properties. Utilizing fine-grained self-learning, UltraSynBERT identifies RNA aptamers for a wide range of target sizes, including small molecules, proteins, cells, and tissue. Additionally, UltraSynBERT discerns tissue specificity for millions RNA species across 22 human organs based on their 3’-UTR sequences, predicts the tropism of human-pathogenic RNA viruses, and characterizes SARS-CoV-2 replicase RNA binding at single-base resolution.

 

UltraSynBERT Archives

1. Pre-training datasets and models

In this repository, you will find the following pre-training datasets and pre-trained model checkpoints:




Model Name
Parameters
Pre-training Data
RNA Species Number
Description
Checkpoint




UltraSynBERT
33.5M
UltraSelex SiR-B
10 million
pre-training from scratch
UltraSynBERT.pkl


UltraSynBERTsource_SiR
33.5M
SELEX SiR
10 million
pre-training from scratch
UltraSynBERT_source_SiR.pkl


UltraSynBERTsource-nsp12
33.5M
SELEX nsp12
10 million
pre-training from scratch
UltraSynBERT_source_nsp12.pkl


UltraSynBERTmolecules
33.5M
12 SELEX targets’ training sets
3.24 million
continued pre-training the basic UltraSynBERT model
UltraSynBERT_molecule.pkl


UltraSynBERT3UTR
33.5M
The preliminary 3’-UTR full training datasets
1.69 million
continued pre-training the basic UltraSynBERT model
UltraSynBERT_3UTR.pkl


UltraSynBERTplus
33.5M
12 SELEX targets’ training sets and the preliminary 3’-UTR full training datasets
4.93 million
continued pre-training the basic UltraSynBERT model
UltraSynBERT_plus.pkl


UltraSynBERTRIP
33.5M
Human RIP-Seq in ENCODE
6.88 million
continued pre-training the basic UltraSynBERT model
UltraSynBERT_RIP.pkl




NOTE:





The pretraining data is located in the Dataset Availability\1. Dataset for pre-training folder.




The pretrained models can be found in the Model Availability\1. Pre-training models and Model Availability\2. Continued pre-training models folders.



 

2. Downstream task datasets and fine-tuned models

Additionally, this official repository also provides all datasets for downstream tasks and supervised fine-tuned models based on the base UltraSynBERT for downstream applications, including: RNA aptamer identification models targeting small molecules, proteins, cells, and tissues (12 specialized datasets), and tissue-specificity prediction for human mRNA 3'-UTR regions.




Task Type
Data Type
Dataset Number
Dataset




SiR-binding prediction
In vitro
3
(1) UltraSelex-SiR-B.csv (2) SELEX-SiR-whole.csv (3) SELEX-SiR-exclusive.csv


In vitro RNA-target binding prediction
In vitro
12
(1) SELEX-small-molecule-DAse.csv; (2) SELEX-small-molecule-BC.csv: (3) SELEX-small-molecule-PR.csv (4) SELEX-small-molecule-MI.csv: (5) SELEX-protein-TARDBP.csv: (6) SELEX-protein-RT.csv (7) SELEX-protein-RBM24.csv; (8) SELEX-protein-S15.csv; (9) SELEX-(multi)cellular-ISLETS.csv (10) SELEX-(multi)cellular-MDSC.csv; (11) SELEX-(multi)cellular-CHO-K1.csv;(12) SELEX-(multi)cellular-TNBC.csv


In vivo RNA-protein interactions prediction (human source)
In vivo
11
(1) 16_ICLIP_hnRNPC_Hela_iCLIP_all_clusters_sequences.csv; (2) 17_ICLIP_HNRNPC_hg19_sequences.csv;(3) 18_ICLIP_hnRNPL_Hela_group_3975_all-hnRNPL-Hela-hg19_sum_G_hg19--ensembl59_from_2337-2339-741_bedGraph-cDNA-hits-in-genome_sequences.csv (4) 19_ICLIP_hnRNPL_U266_group_3986_all-hnRNPL-U266-hg19_sum_G_hg19--ensembl59_from_2485_bedGraph-cDNA-hits-in-genome_sequences.csv; (5) 20_ICLIP_hnRNPlike_U266_group_4000_all-hnRNPLlike-U266-hg19_sum_G_hg19--ensembl59_from_2342-2486_bedGraph-cDNA-hits-in-genome_sequences.csv; (6) 22_ICLIP_NSUN2_293_group_4007_all-NSUN2-293-hg19_sum_G_hg19--ensembl59_from_3137-3202_bedGraph-cDNA-hits-in-genome_sequences.csv (7) 27_ICLIP_TDP43_hg19_sequences.csv; (8) 28_ICLIP_TIA1_hg19_sequences.csv; (9) 29_ICLIP_TIAL1_hg19_sequences.csv (10) 30_ICLIP_U2AF65_Hela_iCLIP_ctrl_all_clusters_sequences.csv; (11) 31_ICLIP_U2AF65_Hela_iCLIP_ctrl+kd_all_clusters_sequences.csv


In vivo RNA-protein interactions prediction (mouse source)
In vivo
11
(1) CLIP-mouse-EZH2-sequences.csv; (2) CLIP-mouse-FUS-sequences.csv; (3) CLIP-mouse-HNRNPR-sequences.csv (4) CLIP-mouse-LIN28A-sequences.csv; (5) CLIP-mouse-RBFOX2-sequences.csv; (6) CLIP-mouse-RBM10-sequences.csv (7) CLIP-mouse-SRSF2-sequences.csv; (8) CLIP-mouse-SRSF3-sequences.csv; (9) CLIP-mouse-TARDBP-sequences.csv (10) CLIP-mouse-U2AF2-sequences.csv; (11) CLIP-mouse-YTHDC2-sequences.csv


modification 
In vivo
13


m6A: (1) m6A-A549.csv; (2) m6A-CD8T.csv; (3) m6A-ESC.csv (4) m6A-HCT116.csv; (5) m6A-HEK293.csv; (6) m6A-HEK293T.csv (7) m6A-Hela.csv; (8) m6A-HepG2.csv; (9) m6A-MOLM13.csv

m1A: m1A_dataset_with_split.csv

m5C: m5c_dataset_with_split.csv

m6Am: m6Am_dataset_with_split.csv

pseudouridine: pseudouridine_dataset_with_split.csv



3'UTR tissue-specific recognition
In vivo
1
human_22_tissue_three_terminal_UTR.csv


Zero-shot RNA mutation effect prediction targeting SARs-CoV-2 replicase nsp12
In vitro
The wild-type sequence and seven mutated sequences
RNA-mutation.fasta


RNA post-transcriptional regulation–related tasks 
in vitro/In vivo
4


(1) miRNA _interactions.csv; (2) mRNA_subcellular_localization.csv; (3) Polyadenylation_signal.csv; (4) RNA_stability.csv

 



RNA modification on human tumors
In vivo
2


(1) LUAD_human_tumors.csv; (2) Bladder_cancer_human_tumors.csv



human-pathogenic RNA viruses
In vivo
1


virus.csv





NOTE:





The datasets for downstream tasks are in the Dataset Availability\2. Dataset for fine-tuning folder.




The fine-tuned models for downstream tasks are available in the Model Availability\3. Fine-tuned models folder.