Neural representation and generation for RNA secondary structures
Authors: Zichao Yan
William L. Hamilton
Mathieu Blanchette
Venue: ICLR 2021 Poster (2021)
Keywords: Graph neural network
Deep generative modeling
Machine learning
Drug discovery
RNA structure
RNA structure embedding
RNA-protein interaction prediction

Abstract: Our work is concerned with the generation and targeted design of RNA, a type of genetic macromolecule that can adopt complex structures which influence their cellular activities and functions. The design of large scale and complex biological structures spurs dedicated graph-based deep generative modeling techniques, which represents a key but underappreciated aspect of computational drug discovery. In this work, we investigate the principles behind representing and generating different RNA structural modalities, and propose a flexible framework to jointly embed and generate these molecular structures along with their sequence in a meaningful latent space. Equipped with a deep understanding of RNA molecular structures, our most sophisticated encoding and decoding methods operate on the molecular graph as well as the junction tree hierarchy, integrating strong inductive bias about RNA structural regularity and folding mechanism such that high structural validity, stability and diversity of generated RNAs are achieved. Also, we seek to adequately organize the latent space of RNA molecular embeddings with regard to the interaction with proteins, and targeted optimization is used to navigate in this latent space to search for desired novel RNA molecules.

Published as a conference paper at ICLR 2021
NEURAL REPRESENTATION AND GENERATION FOR RNA SECONDARY STRUCTURES
Zichao Yan
William L. Hamilton
School of Computer Science McGill University, Mila
School of Computer Science
McGill University, Mila w @ cs mcg i 11 1 ca
z chao. yanemayyn@maaiiam i megill ca
Mathieu Blanchette
School of Computer Science McGill University
b slanchem@cs macgill.c. ca
ABSTRACT
Our work is concerned with the generation and targeted design of RNA, type of genetic macromolecule that can adopt complex structures which influence their cellular activities and functions. The design of large scale and complex biological structures spurs dedicated graph-based deep generative modeling techniques, which represents a key but underappreciated aspect of computational drug discovery. this work, we investigate the principles behind representing and generating different RNA structural modalities, and propose a flexible framework to jointly embed and generate these molecular structures along with their sequence in meaningful latent space. Equipped with a deep understanding of RNA molecular structures, most sophisticated encoding and decoding methods operate the molecular graph as well as the junction tree hierarchy, integrating strong inductive bias about RNA structural regularity and folding mechanism such that high structural validity, stability and diversity of generated RNAs are achieved. Also, we seek to adequately organize the latent space of RNA molecular embeddings with regard the interaction with proteins, and targeted optimization is used to navigate in this latent space to search for desired novel RNA molecules.
INTRODUCTION
There is an increasing interest in developing deep generative models for biochemical data, especially in the context of generating drug-like molecules. Learning generative models of biochemical molecules can facilitate the development and discovery of novel treatments for various diseases, reducing the lead time for discovering promising new therapies and potentially translating in reduced costs for drug development (Stokes et al., 2020). Indeed, the study of generative models for molecules has become rich and active subfield within machine learning, with standard benchmarks (Sterling
Irwin 2015). set of well-known baseline approaches (Gómez-Bombaree et al., 2018 Kusner et al. 2017
Liu et al. 2018 Jin al. 2018), and high-profile cases of real-world impact
Prior work in this space has focused primarily on the generation of small molecules (with less than 100 atoms), leaving the development generative models for larger and more complicated biologics and biosimilar drugs (e.g., RNA and protein peptides) an open area for research. Developing generative models for larger biochemicals critical in order to expand the frontiers of automated treatment design. More generally, developing effective representation learning for such complex biochemicals will allow machine learning systems to integrate knowledge and interactions involving these biologically-rich structures.
In this work, we take a first step towards the development of deep generative models for complex biomolecules, focusing the representation and generation of RNA structures. RNA plays crucial
c.g. BoabhdaaZem/projcer for exascale search of drug-like molecules.

---

Published as a conference paper at ICLR 2021
role in protein transcription and various regulatory processes within cells which can be influenced by its structure (Crick] 1970; Stefl et al. ).05). and RNA-based therapies are an increasingly active area of research (Pardi et al.] 2018 (Schlake et al. 2012), making it a natural focus for the development of deep generative models. The key challenge in generating RNA moleecuess-comparee generation of small molecules-it that RNA involves hierarchical, multi-scale structure, including a primary sequential structure based on the sequence of nucleic acids as well as more complex secondary and tertiary structures based on the way that the RNA strand folds onto itself. An effective generative model for RNA must be able to generate sequences that give rise to these more complex emergent structures.
to the
There have been prior works on optimizing or designing RNA sequences- usinng reinforcement learning or blackbox optimization to generate particular RNA secondary structures (Runge et al. 2019; Churkin et al., (017). However, these prior works generally focus on optimizing sequences to conform to a specific secondary structure. contrast, our goal to define a generative model, which can facilitate the sampling and generation of diverse RNA molecules with meaningful secondary structures, while also providing a novel avenue for targeted RNA design via search over a tractable latent space.
Key contributions. We propose a series of benchmark tasks and deep generative models for the task of RNA generation, with the goal of facilitating future work this important and challenging problem. We propose three interrelated benchmark tasks for RNA representation and generation:
Unsupervised generation: Generating stable, valid, and diverse RNAs that exhibit complex secondary structures.
2. Semi-supervised learning: Learning latent representations of RNA structure that correlate with known RNA functional properties.
3. Targeted generation:
Generating RNAs that exhibit particular functional properties.
These three tasks build upon each other, with the first task only requiring the generation of stable and valid molecules, while the latter two tasks involve representing and generating RNAs that exhibit particular properties. In addition to proposing these novel benchmarks for the field, we introduce and evaluate three generative models for RNA. All three models build upon variational autoencoders (VAEs) (Kingma & Welling (2014) augmented with normalizing flows (Rezende & Mohamed] 20155] Kingma et al., 2016), and they differ in how they represent the RNA structure. To help readers better understand RNA structures and properties, a self-contained explanation is provided in appendix B
The simplest model (termed LSTMVAE) learns using a tring-based representation of RNA structure. The second model (termed Graph leverages graph-based representation and graph neural network (GNN) encoder approach Giimer et al. 2017). Finally, the most sophisticated model (termed HierVAE) introduces and leverages a novel hierarchical decomposition of the RNA structure. Exten- sive experiments on our newly proposed benchmarks highlight how the hicrarchical approach allows more effective representation and generation of complex RNA structures, while also highlighting important challenges for future work in the area.
2
TASK DESCRIPTION
Given a dataset of RNA molecules, i.c. sequences of nucleotides and corresponding secondary structures, our goals are to: (a) learn to generate structurally stable, diverse, and valid RNA molecules that reflect the distribution in this training dataset; (b) learn latent representations that reflect the functional properties of RNA. A key factor in both these representation and generation processes is that we seek to jointly represent and generate both the primary sequence structure as well as the secondary structure conformation. Together, these two goals lay the foundations for generating novel RNAs that satisfy certain functional properties. To meet these goals, we create two types of benchmark datasets, each one focusing on one aspect of the above mentioned goals:
Unlabeled and variable-length RNA. The first dataset contains unlabeled RNA with moderate and highly-variable length (32-512 nts). obtained from the human transcriptome (Aken et al., 2016) and through which we focus on the generation aspect of structured RNA and evaluate the validity, stability and diversity of generated RNA molecules. In particular, our goal with this dataset is to jointly generate RNA sequences and secondary structures that are biochemically feasible (i.e., valid), have
2

---

Published as a conference paper at ICLR 2021
low free energy (i.e., stable), and are distinct from the training data (i.e., diverse). We will give an extended assessment of the generation aspect under different circumstances, e.g., when constraining the generation procedures with explicit rules.
The second dataset pulled and processed from previous study on in vitro RNA- protein interaction, which features labeled RNAs with shorter and uniform length (40 nts) (Cook et al. 2017). With this dataset, our objective is slightly expanded (to include obj. a), so that the latent space is adequately organized and reflective of the interaction with proteins. Therefore, key assessment for the latent space includes AUROC for the classification of protein binding, which is crucial for the design of desired novel RNA molecules.
Labeled RNA.
Essentially, this creates slight variations in the task formulation, with the first dataset suited to unsupervised learning of generative model, while the second datasets involves additional supervision (e.g., for semi-supervised model or targeted generation). Our specific modeling choices, to be introduced section 3 are invariant to different task formulations, and flexible enough handle different representations of RNA secondary structures. We refer readers to appendix [ooo detailed explanation for the dataset and evaluation metrics on the generated molecules and latent embeddings.
3 METHODS
In this section, we introduce three different generative models for RNA. All three models are based upon the variational autoencoder (VAE) framework, involving three key components:
1. A probabilistic encoder network (=)ar)), which generates a distribution over latent states given an input representation of an RNA. We experiment with three different types of input encodings for RNA sequence and secondary structures (see Figure ST dot-bracket annotated string, graph with adjacency matrix representing base-pairings, and a graph augmented with a hicrarchical junction tree annotation for the secondary structure.
2. A probabilistic decoder network po (=](z) which defines joint distribution over RNA sequences and secondary structures, conditioned on latent input. As with the encoder network, we design architectures based on a linearized string decoding and graph-based hierarchical junction-tree decoding approach.
parameterized prior (2), which defines a prior distribution over latent states and is learned based on continuous normalizing flow (CNF) (Chen et al. 2018).
For all the approaches we propose, the model is optimized via stochastic gradient descent to minimize the evidence lower bound (ELBO): L == -E -gg (=|=) -
INA MED: friding
[Pe (z) + 8 (L (z)) where 3 is term to allow KL-anncaling over the strength of the prior regularization.
In the following sections, we explain our three dif- ferent instantiations of the encoder (section 3.1). decoder (section 3.2).. as well as our procedures to structurally constrain the decoding process using domain knowledge (section 3.3) and our proce- dures avoid posterior collapse (section 3.4.
deconspostion
3.1 ENCODING
Hierarchical encoding. Panel (A) shows the minimum free energy structure (for illustration purpose only) which represented as graph. In panel (B), the original molecule is de- composed by grouping structurally adjacent nu- cleotides into subgraphs, giving rise to a higher- order junction tree representation.
Figure 1:
RNA SECONDARY STRUCTURES
The input to the encoder is structured RNA molecule, with its sequence given by an or- dered array of nucleotides I EI, with x; E [A,C, G, U}. where is the length of the se- quence, and its secondary structure, either rep- resented as (1) a dot-bracket string S = i ... &1. with à; E {-, (. }} (2) or as a graph & with two types of edges covalent bonds along the RNA backbone, and hydrogen bonds between the base-
3

---

Published as a conference paper at ICLR 2021
pairs We use Zur to denote edge features between nucleotides u and o; (3) hypergraph T depth-first ordered arry of subgraphs 91 9D with L(Q:) E {S, H, M) indicating the subgraph label, and I(G;) Lili [1 (1...L} L}} indicating the assignment of nucleotides to each subgraph.
Encoding RNA secondary structure as sequence. First, we obtain a joint encoding over the nucleotide and the dot-bracket annotation, using the joint sequence-structure vocabulary N.C.G..) x (.)). Then, these one-hot encodings are processed by a stacked bidirectional LSTM (Hochreiter & Schmidhuber, 1997). followed by multi-head self-attention module (Vaswani ct al. 2017) to weigh different positions along the RNA backbone. A global max-pooling is used to aggregate the information into lıs, and then we obtain mean μs and log variance log øs from hs through linear transformations, and draw latent encoding 2s from N ((us. using the reparameteri- zation trick (Kingma & Welling 2014).
Learning graph representation of RNA secondary structure. To encode the graph view g of an RNA secondary structure, we pass rounds of neural messages along the RNA structure, which falls into the framework of Message Passing Neural Network (MPNN) as originally discussed in Gilmer et al. (2017) and similarly motivated by Jin et al. (2018).
For much longer RNAs, it is conceptually beneficial to pass more rounds of messages so that a nucleotide may receive information on its broader structural context. However, this may introduce undesired effects such as training instability and over-smoothing issues. Therefor we combine our MPNN network with gating mechanism, which is collectively referred as the G-MPNN:
c/o o locat [x || [mol
W
Σ )
(1)
of ue
GRU uv ,
(2)
mag
N(E)
where [...II...]] denotes concatenation, a denotes the activation function and GRU indicates the gated recurrent unit (Cho al.] 2014). Then, after T iterations of message passing, the final nucleotide level embedding is given by: ha = =(W ( nucleotide level embeddings into the graph level, we pass hi... hr through single bidirectional at each step, and hy max (hū,li i E I...I)). The latent encoding 2g is similarly obtained from hg using the reparameterization trick.
[T l| EveN(u) F.uul). Before pooling the
LSTM layer, obtaining hn h hr,
Hierarchical encoding of the RNA hypergraph. To encode the junction tree T of RNA, employ a type of GRU specifically suited to tree-like structures, which has previously been applied works such GGNN (Li et al. 6 and JTVAE (Jin et al., 2018). We refer to this tree encoding network as T-GRU, and the format of its input is shown in Figure
One major distinction between RNA junction tree and the one used for chemical compounds (Jin ct al. 2018) is that an RNA subgraph assumes more variable nuclcotide composition such that it is impossible to enumerate based on the observed data. Therefore, we need dynamically compute the features for each node in an RNA junction tree based on its contained nucleotides, in a hierarchical manner to leverage the nucleotide level embeddings learnt by G-MPNN.
Considering subgraph G. in the junetion tree T, we initialize its node feature with: Fg. [C(),) max,ezi(.) hn.]. Notably, max hy is max-pooling over all nucleotides assigned to ف. and nucleotide embedding h. comes from G-MPNN. To compute and pass neural messages between adjacent subgraphs in the RNA junction tree T, we use the T-GRU network in Eq
0, = T-GRR(E,, f f
I E N(G)))
(3)
hg. = (Wemblle ( ( 11 Σ
hel)
(4)
GEN(C.)
with details of T-GRU provided in the appendix [D_ and compute the embeddings for subgraphs with Eq. Further, we obtain a depth-first traversal of the subgraph embeddings hg, ... hgg which is also the order for hierarchical decoding to be discussed later. This ordered array of embeddings is processed by another bi-directional LSTM, and the final tree level representation hr is again given by the max-pooling over the bi-L-TMM outputs. Likewise, latent encoding 27 is obtained from hT.
We do not differentiate the number of hydrogen honds, which can he different depending on the hase-pairs. For example, G-C has three hydrogen bonds whereas A-U only contains two.
4

---

Published as a conference paper at ICLR 2021
3.2 RNA MOLECULAR GENERATION
Decoding linearized sequence and structure. In this setting, the decoder simply autoregressively decodes token at each step, from the joint sequence-structure vocabulary mentioned before in section 3.1] plus one additional symbol to signal the end of decoding. To simplify the design choice, we use a single-layered corwarddiirectionll LSTM, and its hidden state is initialized with the latent encoding 2, which can be either zs, 2g or 2T.
Hierarchically decoding hypergraph and nu- cleotide segments. The input to this more so- phisticated hierarchical decoder are latent en- codings 2g which contains order and basic con- nectivity information of the nucleotides, and 2T which contains higher order information about the arrangements of nucleotide branches and their interactions. We give concise description of the decoding procedures here, along with a de- tailed algorithm in appendix E On high level, we hierarchically decode the tree structure in depth-first manner, and autoregressively gener- ate a nucleotide segment for each visited tree branch. For these purposes, we interleave three types of prediction (Figure
tupological prodiction
node prediction
81
copeet
$1
M2
R
80
OAEE
nuclootide seginent docoding
MLP |er IAg
LA A.C.UU
U
cSTM
Denote the current tree node at decode step and at the i-th visit as gL., whose features include (1) its node label L(G+.)) and, (2) summary nucleotide seg-
G.S.A
0,0
over the already existing ments max(h") Iu E g and/ t and j < é}. with I denoting the nucleotide is decoded at step I, and j indicating the nucleotide belongs to the j-th branch (this feature is simply zeros when i = 1). Then, its local feature Egg. is defined as the concatenation of (I) and (2).
MO
Figure 2: Hierarchical decoding of a structured RNA, involving three types of predictions, that are on the topological level, node level, and nu- cleotide level. These three types of prediction are interleaved into the procedures of decoding the junction tree structure of RNA and the nucleotide segments.
We make use of notion called node state: hig... which is obtained by: high.
T-GRU(Z (ve.ex gus 19 E N(GT+))). Note its similarity to Eq. 3] and hg.. is used to make:
topological prediction in Figure 2 (A), to determine if the decoder should expand new tree node or backtrack to its parent node, based on MLPtopo (h
tree node prediction in Figure (B), on condition that new tree node is needed due to possible
topological expansion. This procedure determines the label of the new tree node from the set of (S, H, M), based on MTP MT.Pnate node@ehheh)...
nucleotide segment decoding in [igure[[C) using single-layered LSTM, whose initial hidden state is MLP a ((hg... æт =g]). The start token is the last nucleotide from the last segment.
Our hierarchical decoder starts off by predicting the label of the root node using 27, followed by topological prediction on the root node and decoding the first nucleotide segment. The algorithm terminates upon revisiting the root node, topologically predicted to backtrack and finishing the last segment of the root node. The decoded junction tree naturally represents an RNA secondary structure that can be casily transformed the dot-bracket annotation, and the RNA sequence is simply recovered by connecting nucleotide segments along the depth-first traversal of the tree nodes.
3.3 STRUCTURALLY CONSTRAINED DECODING
To better regulate the decoding process so that generated RNAs have valid secondary structures, set of constraints can be added to the decoding procedures at the inference stage. Essentially, a valid RNA secondary structure needs to observe the following rules: (1) base- mase-pairing complementarity.
5

---

Published as a conference paper at ICLR 2021
R JAN BROEER
PC PEIDNAAAAM
35X D40100
PC. 1540
AEMI MM
4813 XM
M.G.N.
Figure 3: RNAs generated with structural constraints from HierVAE on a random axis in the latent space (step size: le-4), for short (top), medium-length (middle), and long (bottom) RNAs. The Free energy (FE) and deviation (DEV) from the MFE are given for each structure.
which means only the canonical base-pairs and Wobble hase-pairs are allowed, i.e. [A-U], [G-C] and [G-U]: (2) hairpin loop should have a minimum of three unpaired nucleotides, i.e. for any two paired bases position and j. j jl > 3; (3) each nucleotide can only be paired once, and overlapping pairs are disallowed.
We will translate the above rules into specific and applicable constraints, depending specific decoders. For the sake of space, we only give a broad remark and leave more details in the appendix.
Linearized decoding constraints. Since the linearized decoder simply proceeds in autoregressive fashion, constraints can be easily enforced in a way that at each step, nucleotide with an appropriate structural annotation is sampled by making of masks and re-normalizing the probabilities. Likewise, a stop token can only sampled when all opening nucleotides have been closed. More details follow in appendix F
Hierarchical decoding constraints. The specific set of constraints for hierarchical decoding is discussed in appendix G Overall, considering the different natures of the three associated types of prediction, each one should require set of different strategies, which are once again applicable by adding proper masks before sampling. As shown in the algorithm in appendix E the set of constraints
are applied to line 13 24 and 14 with marked asterisk.
3.4 AVOIDING POSTERIOR COLLAPSE
As discussed in line of previous works, VAEs with strong autoregressive decoders susceptible to posterior collapse, an issue where the decoder simply ignores the latent encoding of the encoder (He) et al. 2019). Therefore, to avoid posterior collapsing, we make use of carefully chosen KL annealing schedule during training to help the encoder adapt its information content the latent encoding and in coordination with the decoder. This schedule is detailed in section 4 We also learn a parameterized prior as suggested in Chen et al. (2017), but using a CNF instead, following similar implementation Yang al. 2019). with details given in appendix H
Our KL annealing schedule is chosen based on empirical observations, to our knowledge, there has yet to exist any principled methods of selecting such schedule. We have used diagnostic metrics such as mutual information (He et al. (2019) and active units (Burda et al. (0166) along with a validation set to select a proper Ki. annealing schedule which is to described later in section
4 RESULTS
We consider three modes of evaluation: (1) unsupervised RNA generation; (2) generation using semi-supervised VAE models and (3) targeted RNA design from an organized latent space. Results presented below, and relevant hyperparameters
be found in Table SI
Unsupervised RNA generation. Here, we evaluate generated RNAs from models trained on the unlabeled RNA dataset for 20 epochs using KI. annealing schedule including 5 epochs of warm-up,

---

Published as a conference paper at ICLR 2021
Table 1: We evaluate RNAs sampled from the posterior distribution: q(z|ær, with held-out test set of 20,000 RNAs. Each molecule is encoded and decoded 5 times. We also evaluate samples from the prior distribution: (O, I) subject to the transformation of latent CNF, where we sample 10,000 encodings and each encoding is decoded 10 times. Normed refers to length normalized FE DEV.
Posterior Decoding
Prior Decoding
Model
Validity? FE DEV Normed, Diversityt Validityt FE DEV,, Normed,, Diversityt
Constrained & Stochastic
LSTMVAE
99.47%
18.197
0.061
6.786
99.50%
18.536 18.534
0.062
6.789
Graph VAE
99.47%
17.275
0.061
6.790
99.36%
0.065
6.791
HierVAB
99.97%
8.678
0.035
6.787
99. 99.86%
8.676
0.036
6.791
Unconstrained & Stochastic
LSTMVAE 62.98%
8,700
0.048
6.791
62.58%
9,060
0.049
6.793
Graph VAE
65.79%
9.508
0.051
6.792
63.45%
10.166
0,055
6.794
HierVAE 94.51%
8.257
0.035
6.787
92.75%
7.897
0.037
6.791
followed by gradually increasing the KL annealing term to 3e-3 (for LSTMVAE and GraphVAE), 2c-3 (for VAE). The KL anncaling schedule was chosen using a validation set of 1,280 RNAs.
Table D compares the generation capability of different models, from the posterior as well the prior distribution, and in scenarios such as applying structural constraints to the decoding process or not. It clearly shows that most advanced model, HierVAE which employs hierarchical view of the structure in its encoding/decoding aspects, achieves the best performance across different evaluation regimes, generating valid and stable RNAs even when the decoding processed is unconstrained. It is also observed that despite having structural constraints, the validity of our generated RNAs are always slightly below 100%. This can be explained by the threshold hyperparameter which sets the maximum number of steps for topological prediction as well as the maximal length of each nucleotide appendix EE]
segment, as shown Algorithm
To further demonstrate the benefits of model training from structural constraints, we sample RNAs from the prior of an untrained HierVAE model. With structural constraints, the validity amounts to 66.34% with an extremely high free energy deviation of 22.613. Without structural constraints, the validity translates to a mere 9.37% and the model can only decode short single stranded RNAs lacks the knowledge of constructing more complex structures. This comparison illustrates that model training is essential for obtaining stable RNA folding.
The junction tree hierarchy of RNAs developed in our work shares certain modelling similarities with the probabilistic context free grammar (Dowell & Eddy 2004) used by covariance models (CM) (Eddy& & Durbin 1994). Infernal (Nawrocki & Eddy (2033) is of the representative works based on CM, which is capable of sampling RNA secondary structures from a CM built around a consensus secondary structure for conserved RNA family. However, due the lack of homologous sequences in our dataset, Infernal is seriously limited and can only sample single stranded RNAs.
Figure 3 illustrate RNA structures generated using HierVAE from a randomly chosen short path through the latent space. Notably, latent encoding provided by HierVAE translates smoothly in the RNA structure domain: nearby points in the latent space result in highly similar, yet different, structures. The generated structures are particularly stable for short and medium size RNAs, and slightly less so for longer RNAs with highly complex structures. A side-by-side comparison between generated RNA secondary structures and MFE structures in Figure SSrresSSswwww shows that generated structures can evolve smoothly in the latent space along with their corresponding MFE structures. We also visualize neighborhoods of a Cysteine-carrying transfer RNA and 5S ribosomal RNA in figure S4 and S55
Supervised RNA generation. We then evaluate generative approaches in semi-supervised setting using seven RBP binding data sets from RNAcompete-S. First, we compare the efficacy of different representational choices while excluding the generative components, i.e. we jointly train VAE encoders followed by simple MLP classifiers on top of the latent encodings for binary classification on RBP binding.
7

---

Published as a conference paper at ICLR 2021
Table 2: Training semi-supervised HierVAE labeled RNAcompete-S dataset. A test split is used to
evaluate the accuracy of embedding classifiers and RNAs decoded from the posterior distribution under two settings: constrained and stochastic (C& S), unconstrained and deterministic (NC&D).
RECON ACC refers to reconstruction accuracy which measures the percentage RNA molecules
decoded exactly the input.
Test
Post C&S
Post NC&D
Dataset
AUROC
Valid
FE DEV
RECON ACC
Valid
FE DEV
RECON ACC
HuR
0.884
100%
0,426
55.85%
99.34%
0.269
68.73%
PTB
0.907
100%
0.409
55.24%
92.07%
0.570
51.96%
QKI
0.824
100%
0.439
55.80%
99.22%
0.296
66.83%
Vts1
0.900
100%
0,539
47.96%
98.90%
0.367
60.60%
RBMY
0.878
100%
0.634
48.09%
98.64%
0.419
61.84%
SF2
0.900
100%
0,616
44.57%
98.88%
0.409
57.15%
SLBP 0.792
100%
0.459
53.67%
98.60%
0.306
65.49%
Table S3 shows that incorporating RNA secondary structures is overall beneficial for the classification accuracy, except for RBMY where model with access to RNA sequence alone (LSTM-SeqQll) SegOnly) has the best performance. Notably, different choices for representing RNA secondary structures do not lead large variation in performance, with the exception of HuR and SLBP, where graph based representations have an advantage over the linearized structural representation. On the other hand, sequence based models often have comparable performance, possibly due the capability of inferring RNA secondary structures from short RNA sequences. It is also worth exploring other in-vitro selection protocols such HTR-SELEX which can select RNAs with higher binding affinities than RNAcompete-S that only involves single selection step.
Next, we train full generative models (encoder, decoder, latent CNF and MLP embedding classifier), and show the results in Table 2 Since our strategy for targeted RNA design makes use of seed molecules in the latent space, we mainly sample RNAs from the posterior distribution of these semi-supervised VAE models. Therefore, we select KI. annealing schedule that tends to retain more information the latent encodings, i.e. setting maximum 3 5e-4 and training 10 epochs.
Results are promising in that classification AUROC measured by the held-out test set is comparable to the fully supervised classification models in Table(S3) and much better compared models only using fixed and pretrained VAE embeddings shown in Table S2 Also, RNA structures generated from the posterior distribution, even under the setting of unconstrained and deterministic decoding, have high success rates, very stable conformation and good reconstruction accuracy.
Targeted RNA design. We next studied the task of de- signing RNAs with high RBP binding affinity. Starting from the latent encodings of 10,000 randomly chosen RNA molecules that have negative labels in each RNAcompete- S test set, and use activation maximization to gradually al- ter the latent encodings that the predicted binding proba- bility from the embedding classifiers increases. These em- bedding classifiers have been trained jointly with the VAE models with accuracy reported earlier (Table Then, we use separately trained full classifiers (also earlier shown in Table (3) as proxy of oracles for evaluating the "ground truth" probability of RBP binding. Table 3 report the success rate (fraction of RNAs whose "ground truth" RBP binding probability improved), along with the average improvement in binding probabilities. An example of trajectory of optimized RNAs is shown in Fig. S6
Table 3: Designing novel RNA with
higher chances RBP binding.
Dataset Success
Improvement
HuR PTB
96.88%
0.561
92.63%
0.331
QKI
90.82%
0.326++0.252
Vis1
54.63%
0.197+0.388
RBMY
84.33%
0.457
SF2 98.91%
0,655+1.2239
SLBP 83.98%
0.309
5 RELATED WORK
Over the years, the field of computational drug discovery has witnessed the emergence of graph- centric approaches. One the carliest method, proposed in defined on the linearized format of molecular structures and represents family of methods that
et al. (2018), is
8

---

Published as a conference paper at ICLR 2021
rely on sequential models represent and generate SMILES strings of chemical compounds. Later methods have sought to construct more chemical priors into the model, via (1) leveraging graph based representation and generation techniques, (2) enforcing direct chemical constraints to the decoding process, (3) considering a multi-scale view of the molecular structures, or (4) using reinforcement learning to integrate more training signal of the molecular structure and function. As result, greater success has been achieved by models such as Kusner et al. (2017); Liu et al.) (201 Jiin et al. (2018); You al. (2018) at generating and searching valid and more useful chemical compounds.
Graph representation learning is at the heart of these more recent approaches, to help understand the rules governing the formation of these molecular structures, as well as the correspondence between structures and functions. Duvenaud et al. (2015) were among the first to apply GNN to learn molecular fingerprints, and the general neural message passing framework for molecules is proposed in Gilmer et all. 2017), which demonstrate the power of MPNN across various molecular benchmarking tasks. These prior works on molecular MPNN, together with other GNN architectures developed in other areas, such considering relational edges (Schlichtkrull et al. 2018) and attention (Velickovic et al. 2018 have laid the foundation for the success of these deep generative models.
Despite the fact that RNA molecules can adopt complex structures, dedicated graph representation learning techniques have been scarce, with some recent works beginning to leverage graph related learning techniques to prediet RNA folding (Chen et al. 2020; Singh et al. 2019) and to represent RNA molecular structures (Yan et al. 2020 Oliver et al., (020). Prior to our work, the design of RNA has mostly focused on the inverse design problem, which is conditionally generate RNA sequence whose MFE secondary structure corresponds to an input secondary structure. Therefore, the line of prior works have predominantly relied on sequential techniques, with some representative methods based on reinforcement learning (Runge et al.) 2019), or more classically framed as a combinatorial optimization problem and solved with sampling based techniques (Churkin al. 2017). These prior works are mainly concerned with querying from an energy model with fixed thermodynamic parameters and fixed dynamics of RNA folding, which itself limited compared to learning based approaches (Chen et al. 2020 (Singh et al. 2019), and are unable to model a joint distribution over RNA sequences and possible folds.
CONCLUSION AND FUTURE WORKS
In this work we propose the first graph-based deep generative approach for jointly embedding and generating RNA sequence and structure, along with a series of benchmarking tasks. Our presented work has demonstrated impressive performance at generating diverse, valid and stable RNA secondary structures with useful properties.
For future works, there are several important directions to consider. First, would be beneficial to obtain non-coding RNA families from the RFAM database (Kalvari et al. 2017) which would representation indicative of RNA homology
help our models learn more biologically-meanigg and functions, in addition to the evolutionarily conserved RNA structural motifs that would enable the generation of more stable RNA secondary structures. In that context, a detailed comparison to Infernal and other probabilistic context-free grammar models would be meaningful.
On the methodological aspect, in light of the recent advances in protein sequences pretraining across a large evolutionary-seett (Rives et al. [019 Elnaggar et al.) 2020). our models for RNAs may similarly benefit by such a procedure with the data collected from RFAM. After the pretraining step, reinforcement learning can be used to finetune the generative component of our model with customizable rewards defined jointly on RNA structural validity, folding stability and functions such as binding to certain proteins.
On the evaluation side, it would be of great interest to analyze our models for any potential RNA tertiary structural motifs and to compare them with those deposited the CaRNAval (Reinharz et al.) 2018) or RNA 3D motifs database (Parlea et al., 2016). Our models would also need modifications to allow non-canonical interactions and pscudoknots, which are common in RNA tertiary structures.
All in all, the representation, generation and design of structured RNA molecules represent rich, promising, and challenging area for future research in computational biology and drug discovery, and an opportunity to develop fundamentally new machine learning approaches.

---

Published as a conference paper at ICLR 2021
REFERENCES
Bronwen L Aken, Premanand Achuthan, Wasiu Akanni, M Ridwan Amode, Friederike Bernsdorff, Jyothish Bhai, Konstantinos Billis, Denise Carvalho-Siike Carla Cummins, Peter Clapham, et al. Ensembl 2017. Nucleic Acids Research, 45(D1): 2016.
Yuri Burda, Roger B. Grosse, and Ruslan Salakhutdinov. Importance Weighted Autoencoders. In International Conference on Learning Representations (ICLR), 2016.
Tian Qi Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. Neural Ordinary Differential Equations. In Advances in Neural Information Processing Systems (NeurIPS), 2018.
Xi Chen, Diederik P. Kingma, Tim Salimans, Yan Duan, Prafulla Dhariwal, John Schulman, lly Sutskever, and Pieter Abbeel. Variational Lossy Autoencoder. In International Conference on Learning Representations (ICLR), 2017.
Xinshi Chen, Yu Li, Ramzan Umarov, Xin Gao, and Le Song. RNA secondary structure prediction by learning unrolled algorithms. In International Conference on Learning Representations (ICLR), 2020.
Kyunghyun Cho, Bart van Merrienboer, Çaglar Gülgehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014.
Alexander Churkin, Matan Drory Retwitzer, Vladimir Reinharz, Yann Ponty, Jérôme Waldispühh, and Danny Barash. Design of RNAs: comparing programs for inverse RNA folding. Briefings in 2017.
Bioinformatics, 19(2):350-358
K. B. Cook, S. Vembu, K. C. H. Ha, H. Zheng, K. U. Laverty, R. Hughes, D. Ray, and Q. D. Morris. RNAcompete-S: Combined RNA sequence/struucture preferences for RNA binding proteins derived from single-step in vitro selection. Methods, 126:18 1--2, 2017.
Francis Crick. Central Dogma of Molecular Biology. Nature, 227(5259) 1970.
Robin D. Dowell and Sean R. Eddy. Evaluation of several lightweight stochastic context-free grammars for RNA secondary structure prediction. BMC Bioinformatics, (()):) 2004.
David Duvenaud, Dougal Maclaurin, Jorge Aguilera-Iparraguirre, Rafael Gómez-Bombarelli, Tim- othy Hirzel, Alán Aspuru-Guzik, and Ryan P. Adams. Convolutional Networks on Graphs for Learning Molecular Fingerprints. In Advances Neural Information Processing Systems (NeurIPS), 2015.
Sean R. Eddy and Richard Durbin. RNA sequence analysis using covariance models. Nucleic Acids Research, 22(1 2020992 06 1994.
Ahmed Elnaggar, Michael Heinzinger, Christian Dallago, Ghalia Rehawi, Yu Wang, Llion Jones, Tom Gibbs, Tamas Feher, Christoph Angerer, Martin Steinegger, DEBSINDHU BHOWMIK. and Burkhard Rost. ProîTrans: Towards Cracking the Language of Life's Code Through Self- Supervised Deep Learning and High Performance Computing. bioRxiv, 2020.
Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. Neural Message Passing for Quantum Chemistry. In International Conference on Machine Learning (ICML), 2017.
Rafael Gómez-Bombarelli, Jennifer N Wei, David Duvenaud, José Miguel Hernández-Looato Benjamín Seanchez-Lengelingg Dennis Sheberla, Jorge Ryan P Adams, and Alán Aspuru-Guzik, Automatic chemical design using data-driven continuous representation of molecules. ACS central science, 4(2):268-21) 2766688-2666666 2018.
Timothy D Hirzel,
Will Grathwohl, Ricky T. Q. Chen, Jesse Bettencourt, Ilya Sutskever, and David Duvenaud. FFJORD: free-form continuous dynamics for scalable reversible generative models. International Confer- on Learning Representations (ICLR), 2019.
10

---

Published as a conference paper at ICLR 2021
Junxian He, Daniel Spokoyny, Graham Neubig, and Taylor Borg-Kirkpatrick. Lagging Inference Networks and Posterior Collapse in Variational Autoencoders. In International Conference on Learning Representations (ICLR), 2019.
S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural Computing, 98)) 80, 1997.
Wengong Jin, Regina Barzilay, and Tommi S. Jaakkola. Junction Tree Variational Autoencoder for Molecular Graph Generation. In International Conference on Machine Learning (ICML), 2018.
Ioanna Kalvari, Joanna Argasinska, Natalia Quinones-Olvera, Eric P Nawrocki, Elena Rivas, Sean R Eddy, Alex Bateman, Robert D Finn, and Anton Petrov. Rfam 13.0: shifting to a genome-centric resource for non-coding RNA families. Nucleic Acids Research, 46(D1):D335-D342, 11 2017.
Diederik P. Kingma and Max Welling. Auto-Encoding Variational Bayes. International Conference on Learning Representations (ICLR), 2014.
Durk P Kingma, Tim Salimans, Rafal Jozefowicz, Xi Chen, Ilya Sutskever, and Max Welling. Improved variational inference with inverse autoregressive flow. In Advances in Neural Information Processing Systems (NeurIPS), 2016.
Matt J. Kusner, Brooks Paige, and José Miguel Hernández-Looare Grammar Variational Autoencoder. In International Conference on Machine Learning (ICML), 2017.
Yujia Li, Daniel Tarlow, Mare Brockschmidt, and Richard S. Zemel. Gated Graph Sequence Neural Networks. International Conference on Learning Representations (ICLR), 2016.
Qi Liu, Miltiadis Allamanis, Marc Brockschmidt, and Alexander L. Gaunt. Constrained Graph Variational Autoencoders for Molecule Design. Advances Neural Information Processing Systems (NeurIPS), 2018.
R. Lorenz, S. H. Bernhart, C. Honer Zu Siederdissen, H. Tafer, Flamm, P. Stadler, and I. Hofacker. ViennaRNA Package 2.0. Algorithms for Molecular Biology, 6:26, 2011.
David H. Mathews, Matthew D. Disney, Jessica L. Childs, Susan J. Schroeder, Michael Zuker, and Douglas H. Turner. Incorporating chemical modification constraints into a dynamic programming algorithm for prediction of RNA secondary structure. Proceedings of the National Academy of Sciences of the United States of America, 101(19)72277, 2004.
Eric P. Nawrocki and Sean R. Eddy. Infernal 1.1: 100-fold faster RNA homology searches. Bioinfor- matics, 29(22):2933-2335, 09 2013.
Carlos Oliver, Vincent Mallet, Roman Sarrazin Gendron, Vladimir Reinharz, William L Hamilton, Nicolas Moitessier, and Jérôme Waldispühl. Augmented base pairing networks encode RNA-small molecule binding preferences. Nucleic Acids Research, 48( 48(14):7900-669 07 2020.
Norbert Pardi, Michael J. Hogan, Frederick W. Porter, and Drew Weissman. mRNA vaccines new era in vaccinology. Nature Reviews Drug Discovery, 17(4):261-279, 2018.
Lorena G. Parlea, Blake A. Sweeney, Maryam Hosseini-Asanjan, Craig L. Zirbel, and Neocles B. Leontis. The RNA 3D Motif Atlas: Computational methods for extraction, organization and evaluation of RNA motifs. Methods, 103:99 119, 2016.
D. Ray, H. Kazan, E. T. Chan, L. Pena Castillo, S. Chaudhry, S. Talukder, B. J. Blencowe, Q. Morris, and T. R. Hughes. Rapid and systematic analysis of the RNA recognition specificities of RNA- binding proteins. Nature Biotechnology, 2767):667-70, 2009.
D. Ray, H. Kazan, K. B. Cook, M. T. Weirauch, H. S. Najafabadi, X. Li, S. Gueroussov, M. Albu, H. Zheng, A. Yang, H. Na, M. Irimia, L. H. Matzat, R. K. Dale, S. A. Smith, C. A. Yarosh, S. M. Kelly, B. Nabet, D. Mecenas, W. Li, R. Laishram, M. Qiao, H. D. Lipshitz, E. Piano, A. H. Corbett, R. P. Carstens, B. J. Frey, R. A. Anderson, K. W. Lynch, L. O. Penalva. E. P. Lei, A. G. Fraser, B. J. Blencowe, Q. D. Morris, and T. R. Hughes. A compendium RNA-binding motifs for decoding gene regulation. Nature, 499(7457): 2013.
11

---

Published as a conference paper at ICLR 2021
Sashank J. Reddi, Satyen Kale, and Sanjiv Kumar. On the convergence of adam and beyond. In International Conference on Learning Representations (ICLR), 2018.
Vladimir Reinharz, Antoine Soulé, Eric Westhof, Jérôme Waldispühl, and Alain Denise. Mining for recurrent long-range interactions in RNA structures reveals embedded hierarchies in network families. Nucleic Acids Research, 46(8):3841-338 03 2018.
Danilo Jimenez Rezende and Shakir Mohamed. Variational Inference with Normalizing Flows. In International Conference on Machine Learning (ICML), 2015.
Alexander Rives, Joshua Meier, Tom Sercu, Siddharth Goyal, Zeming Lin, Demi Guo, Myle Ott, C. Lawrence Zitnick, Jerry Ma, and Rob Fergus. Biological Structure and Function Emerge from Scaling Unsupervised Learning to 250 Million Protein Sequences. bioRxiv, 2019.
Frederic Runge, Danny Stoll, Stefan Falkner, and Frank Hutter. Learning to Design RNA. In International Conference on Learning Representations (ICLR), 2019.
Roman Serrazin-Gendron, Hua-Ting Yao, Vladimir Reinharz, Carlos G. Oliver, Yann Ponty, and Jérôme Waldispühl. Stochastic Sampling of Structural Contexts Improves the Scalability and Accuracy of RNA 3D Module Identification. In Russell Schwartz. (ed.), Research Computational Molecular Biology, pp. 186-201. Springer International Publishing, 2020.
Thomas Schlake, Andreas Thess, Mariola Fotin-Mleczek, and Karl-Josef Kallen. Developing mRNA- vaccine technologies. RNA Biology, 9(1 (11):1319-13366 2012.
Michael Sejr Schlichtkrull, Thomas N. Kipf, Peter Bloem, Rianne van den Berg, Ivan Titov, and Max Welling. Modeling Relational Data with Graph Convolutional Networks. In The Semantic Web 15th International Conference, ESWC 2018, pp. 593-607, 2018.
Jaswinder Singh, Jack Hanson, Kuldip Paliwal, and Yaoqi Zhou. RNA secondary structure prediction using an ensemble two-dimensional deep neural networks and transfer learning. Nature Communications, 10(1):5407, 2019.
Richard Stefl, Lenka Skrisovska, and Frédéric H. T. Allain. RNA sequence- and shape-dependen recognition by proteins in the ribonucleoprotein particle. EMBO reports, 6(1):33-38, 2005.
Teague Sterling and John J. Irwin. ZINC 15 - Ligand Discovery for Everyone. Journal of Chemical Information and Modeling, 55(11):2324 2337, 2015.
Stokes, K. Yang, K. Swanson, W. Jin, A. Cubillos-Ruiz, N. M. Donghia, C. R. MaeNair, S. French, L. A. Carfrae, Z. Bloom-Ackermann, V. M. Tran, A. Chiappino-Pope, A. H. Badran, W. Andrews, E. Chory, G. M. Church, E. D. Brown, T. Jaakkola, R. Barzilay, and J. J. Collins. A Deep Learning Approach to Antibiotic Discovery. Cell, 181(2):475-45) 2020.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems (NeurIPS), 2017.
Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph Attention Networks. In International Conference on Learning Representations
(ICLR), 2018
Zichao Yan, William L. Hamilton, and Mathieu Blanchette. Graph neural representational learn- ing of RNA secondary structures for predicting RNA-protein interactions. Bioinformatics, 36 2020.
Guandao Yang, Xun Huang, Zekun Hao, Ming- Yu Liu, Serge J. Belongie, and Bharath Hariharan. PointFlow: 3D Point Cloud Generation With Continuous Normalizing Flows. In International Conference Computer Vision (ICCV), 2019.
Jiaxuan You, Bowen Liu, Zhitao Ying, Vijay S. Pande, and Jure Leskovec. Graph Convolutional Policy Network for sorretDeeeteed Molecular Graph Generation. In Advances in Neural Information Processing Systems (NeurIPS), 2018
12

---

Published as a conference paper at ICLR 2021
A ACKNOWLEDGEMENTS
We would like to thank all members of the Hamilton lab, Blanchette lab, and the four anonymous reviewers for their insightful suggestions. This work was funded by a Genome Quebec/Canada grant to MB and by the Institut de Valorisation des Données (IAVDO) PhD excellence scholarship to ZY. WLH supported by Canada CIFAR AI Chair. We also thank Compute Canada for providing the computational resources.
B BACKGROUND: RNA STRUCTURE AND KEY PROPERTIES
The representation of an RNA molecule starts from its primary sequence structure-iie. single chain of nu- cleotides (adenine (A), cytosine (C), guanine (G) and uracil (U)). RNA sequences are flexible and can fold onto themselves, enabling the formation of bonds between com- plementary nucleotides (Watson-Crick base-pairs [A-U, G-CJ, and Wobble base-pairt [G-UJ), hence stabilizing the molecule The set of pairs of interacting nucleotides in an RNA forms its so-called RNA secondary structure. In computational analyses of RNA, is standard to assume that secondary structure is nested: if [i, j] and [k, I] form base pairs with i < l., then either | < (nesting) or k > (non-overlapping). This enables simple string or planar graph representations (Figure ST b).
dot-bracket representatioon
B
graph
representation
representation
The nested structure assumption means that secondary structures can be modelled by probabilistic context free grammar (Dowell & Eddy (204), or by the closely related junction tree structure (Figure (TTc) c) (Sarrazin-Genni
A nested RNA secondary structure can be represented by: (A) dot- bracket annotation, where base-pairs cor- responding matching parentheses, (B) a molecular planar graph with two types of edges, corresponding to consec- utive nucleotides (backbone) and base- pairing interactions, or (C) junction tree where node are labeled as stems (S), hairpins (H), internal loops (I), or multi- loops (M), and edges correspond to the connections between these elements. All three forms equivalent.
Figure S1:
cct al. 2020), where each hypernode corresponds to a par- ticular secondary substructure element: (1) stem: consec- utive stacked base-pairs locally forming a double-stranded structure; (2) hairpin loop unpaired regions closed by a base-pair, (3) internal loop: unpaired regions located between two stems; (4) multiloop: unpaired regions at the junction of at least three stems. Edges link elements that are adjacent in the structure.
Validity and stability of RNA folding, The notion of free energy of RNA secondary structures can be used to characterize the stability of a particular conformation. Given an RNA sequence, there are combinatorially many valid RNA secondary structures which all need to obey a set of constraints (summarized in section 3.3). However, some structures are more stable than the others by having lower free energy. Therefore, these structures are more likely to exist (hence more useful) in reality due to the stochastic nature of RNA folding. The free energy of an RNA secondary structure can be estimated by an energy-based model with thermodynamic parameters obtained from experiments (Mathews et al. 2004), wherein the minimum free energy (MFE) structure can be predicted, up to a reasonable approximation (Lorenz et al. 2011
'There exists other non-canonical hase-pairs which are excluded from our current work.
'Throughout this work, use RNAfold (Lorenz et al. (2011) 1) to compute free energy as well as the MFE
structure, due its interpretahility and acceptable accuracy for moderately sized RNAs.
13

---

Published as a conference paper at ICLR 2021
C DATASET AND METRICS
The unlabeled dataset is obtained from the complete human transcriptome which is downloaded from the Ensembl database (Aken al. (2016); version GRCh38). We slice the transeripts into snippets with length randomly drawn between 32 and 512 nts, and use RNAfold to obtain the MFE structures. We randomly split the dataset into training set that contains 1,149,855 8599 RNAs, and 20,000 held-out RNAs for evaluating decoding from the posterior distribution. More information on the structural diversity and complexity of this dataset is shown in Figure S2 challenges for our algorithms.
which should present significant
The labeled dataset pulled from previous study on sequence and structural binding preference of RNA binding proteins (RBP), using an in vitro selection protocol called RNAcompete-S (Cook ct al. 2017) which generates synthesized RNA sequences bound or unbound to a given RBP. RNAs in this experiment are of uniform length, i.e. 40 nts, and offer a rich abundance of RNA secondary structures compared to its predecessor protocols such as RNAcompete (Ray et al. 2009: 2013). Since randomly sample 500,000 positive
no benchmark has been ever established since its publication, sequences bound to an RBP, and the same amount of negative sequences from the pool of unbound sequences, curate dataset for each of the seven RBPs investigated in the paper. Then, 80% of all RNAs are randomly selected to the train split, and the rest goes to the test split.
Our evaluation scheme for the generated RNA secondary structures includes the following metrics:
validity: percentage of generated RNA secondary structures that conform the structural con- straints specified in section 3.3
freee energy deviation (FE DEV): difference of free energy between the generated RNA secondary structure and the MFE structure of the corresponding sequence, which quantifies the gap of both structures from energy perspective. A lower FE DEV should indicate higher stability of generated RNAs.
free energy deviation normalized by length (Normed FE DEV): FE DEV divided by the length of generated RNA, which distributes the contribution of total FE DEV each base.
5-mer sequence diversity: entropy of the normalized counts of 5-mer substrings, which directly measures the diversity of RNA sequences, and indirectly for RNA secondary structures when this metric is combined with FE DEV, since monolithic structures of diverse sequences would lead to high FE DEV.
D TREE ENCODING GRU
Following Eq T-GRU computes message vi G.Gd from G. and فj, based on the features in
9, denoted by Tg.. as well as neural messages from neighboring subgraphs to ÿi. i... fo/= c. G E
N( The internal structure of T-GRU is equivalent to the tree encoder employed in Jin er al. (2018), which essentially neural analogue of the belief propagation algorithm on junction trees. Nevertheless, write down the message passing formulas of T-GRU here:
$0.0. = Σ
--1
(S1)
(.EN(H.)
29.9; = oW (xg
" +b)
(S2)
r r...0. = oW 1
Bn.0. (b*)
(S3)
G.. Tanh (=r
Σ
To 1-11
(S4)
GREN(GA)
G = (1 - =),09,) 2 $G.9, + +g 2 06..0
(S5)
14

---

Published as a conference paper at ICLR 2021
E ALGORITHM FOR HIERARCHICALLY DECODING STRUCTURED RNA
Algorithm 1: DFS decode RNA secondary structure
Given: 27, 2g. M.TI, M.SI
Initialize: stack t []
function decode(2T, zg)
rool <- sample(MMPP no)e
rrot.adddincoming message(2T))
stack.push((roc) 0))
t 2+0;; 0;
while t M.TI and stack 2 1 do c_node, last fuc t stack.get last last last.mmsttmmm:: all_msg < {msg Vnsg
proode.gel.ncooming (nconinggmessage(() local_field + [c_node.label()
11
12
new-msg - [- all all_msg)
// topological prediction
13
is backtrack sMLPle topo))))))
/ // nucleotide segment prediction
14
new new_msp msg, last_nac,, decoded segmeent,,,, seegmeen,,sgment..... segment_featuree +
decode sennoddssegneenneeee .msg, last_nue, 2T, æg. M.SI)*
15
www.ddd www.oeeeddddeeeeeeeeeeeeededeedd segmend decoodd_seggment c_node.add wenddssement-ffttteeee (wt.afffeeaauurssseentttt wwwgeettteetttrreeeeeeeeeeeeeeee backtrack = True then
17
// backtrack to the parent node
18
c_node.add wwt.jjjooonnmmssssseeeeee p_node,- - t- stack.get. stack..et.pmmfiimat..eeee p_node.ado neighbor((c.node):
_mag);
29
21
stack penultimate ittem(p_nooe last.nuc)); stack.pop()
23
else
// predict and expand new_node + sample(MLP modee (new-msg))"; new new.node.ado incoming message(new msg):
tree node
24
25
new new.node.ade www.nnner(cc.nnooie last nus))) Brock.push(new..ng
27
end 28
29
t +--to 1
30
end
return root
"M TI refers the threshold which set the maximum allowed numher of topological prediction steps; ST another threshold to limit the length each decoded nuclentide segment.
15

---

Published as a conference paper at ICLR 2021
F DETAILS FOR APPLYING RNA STRUCTURAL CONSTRAINTS TO LINEARIZED DECODING PROCEDURES
When decoding from the joint vocabulary of sequence and dot-bracket structure C,,) x {-r ()),, whenever a nucleotide nuc; with a left bracket sampled at step i, we append them to a stack, {[nuci io) ... (nue,, i) Then, decode step
if j - S 3. proper mask will be added the categorical logits of the vocabulary, to avoid sampling any nucleotides with right brackets, which means only an unpaired nucleotide or one that comes with left bracket be sampled;
if j - - 3. mask will be applied to make sure that only nucleotide complementary to nuc; can be sampled with the right bracket. Sampling nucleotides with other forms of structures are allowed.
As nucleotide with closing right bracket is sampled, we pop out (nuc;; from the stack. The special symbol for stop decoding can only be sampled when the stack has become empty.
G DETAILS FOR APPLYING RNA STRUCTURAL CONSTRAINTS TO HIERARCHICAL DECODING PROCEDURES
Additional constraints to be enforced during the hierarchical decoding process ensure the validity of the decoded RNA secondary structure. Recall in section 3.2 that three types of predictions are involved with the hierarchical decoding, therefore, each type is associated with its own of rules. All set of rules can be observed by adding proper masks to the categorical logits before sampling, which are detailed below.
Constraints for making topological prediction, when the current node is
stem node, then the algorithm always expands to a new node upon its first visit, or backtracks to its parent node upon risvisit;
hairpin node, then the algorithm always backtracks;
internal loop, then the algorithm acts similarly as for stem node;
multi-loop, then the algorithm always expands upon first visit and the next re- visit... Further re- re-visits the same multi-loop node are not regulated.
Constraints for predicting new tree node, when the current node is
stem node, then its child node when exists can be either a hairpin loop, an internal loop, or multi-loop;
hairpin node, internal loop or multi-loop, then its child node must be a stem node.
Constraints for decoding nucleotide segment. Due the property of non-empty intersection between adjacent subgraphs, the start token for decoding a segment at the current node, is always the last nucleotide decoded at the last node. Therefore, without explicitly mentioning, the algorithm needs decode at least one new nucleotide at each segment. When the current node is
stem node, and it is upon its first visit (i.e. decoding the first segment of stem), then there is no for constraints. Otherwise, upon its re-visit, the algorithm needs to decode exactly the complementary bases and the reverse order, according to the first decoded segment; hairpin node, then the decoder needs to decode at least four nucleotides before seeing the stop symbol, unless the hairpin is also the root node.
internal loop node, and if it is upon its first, then constraint not necessary. Otherwise, upon its revisit, the algorithm needs to decode least unpaired nucleotide on condition that the first decoded internal loop segment does not contain any unpaired nucleotides;
multi-loop node, then there is no need for constraints.
16

---

Published as a conference paper at ICLR 2021
H DETAILS FOR PARAMETERIZING PRIOR DISTRIBUTION USING NORMALIZING FLOW
normalizing flow involves series of bijective transformation with tractable Jacobian log- determinant, to map an observed datapoint x ~ Po(x) from a complex distribution to a simpler one, such as the standard normal distribution.
Considering the simplified case where we have a single bijective function fo Z - X' to map some simple latent variables to observed datapoint æ, then, using the change of variable theorem, the likelihood of the observed datapoint can be evaluated as:
(x) dx
(= p=
(S6)
where p=(.) denotes some simple base distribution, c.g. (N(O;)) I). Then, it becomes clear the efficiency of this scheme heavily relies on the efficiency of inverting the forward mapping fo as well as computing its Jacobian log-determinant.
In this project, we use a type of continuous normalizing flow (CNF) which simplifies the above mentioned computation (Chen et al., 2018). Consider time continuous dynamics (=(2((), 1) of some intermediate data representation =(l). and again =(4o) Pz(-), the transformation of variable, along with its inverse mapping, can be expressed as:
L(d fo(=(1),
# - x(4) = =(10) +
(S7)
to
=(tn) =(ti)+ + ["x f=(x(1),
(S8)
and the change of probability density can be expressed as:
O / tr( dz(()) dt
log pv (z) = log Pa (=(ta))
(S9)
Note that the invertibility issue is no longer a concern under some mild constraints (Chen et al. 2018).
Also, Eq. S9 only involves more light-weight trace operation the Jacobian rather than eva evaluating its log-determnnatt.
Therefore, we learn parameterized prior using CNF, and observe the decomposition of the KL term in the VAE objective:
(1)r)) - (ele [[go(
(S10)
Therefore, during training our CNF parameterized with v works the transformation of complex latent encodings ~ (2)
(2) 2 to some simple 2(ta) ~ (0(0; I). with an exact likelihood described by Eq. (9 and integrated into Eq. S10 for the complete training objective. During inference, we simply sample 2t, ~ (O; f), and use our CNF to reversely transform it to 2 (.)() which should be closer to the approximate posterior.
Our specific parameterization of the CNF follows from Yang et al. (019) and Grathwohl et al. (009 interleaving two hidden concatsquash layers of dimensionality 256 with Tanh non-linearity.
17

---

Published as a conference paper at ICLR 2021
I INFORMATION OF THE UNLABELED RNA DATASET
A
B
10-
300
300
300
400
390
100
252
300
400
303
D
20000
30000
25000
20000
15400
20000
20000
Figure S2: This figure contains information of the unlabeled RNA dataset. (A) The number of hypernodes appears grow linearly with the length of RNA, and (B) the junction tree height also grows as the length increases but on a more moderate scale. (C) and (D) have shown bar-plots of the number of hypernodes and tree height, indicating that the junction depth hence contributing to the diversity and complexity of RNA secondary structures represented in this dataset.
of RNA can take on significant
18

---

Published as a conference paper at ICLR 2021
J HYPERPARAMETERS
Table S1: Hyperparameters for training VAE and full classifier models. Note that hidden units refer to the dimensionality of encoders and decoders from LSTMVAE, Graph VAE as well as HierVAE models. Dropout is applied to the embedding MLP classifier in case training semi-supervised VAEs, which contains one hidden layer.
for VAE models
latent dimensionality
128
hidden units
512
G-MPNN iterations
5
T-GRU iterations
10
learning rate
1e-3
batch size
32
optimizer
AMSGrad (Reddi al. 2018
dropout ratio
0.2
M.TI
300
S_TI (hierarchical decoder)
100
TI (linearized decoder)
1000
for full classifier models
(overriding some above hyperparameters)
learning rate epochs
2e-4
200
early stopping epochs
5
K RNACOMPETE-S CLASSIFIERS ON PRETRAINED AND FIXED VAE EMBEDDINGS
Performance of simple MLP classifiers on top of fixed latent embeddings from VAE models, which have been pretrained on the unlabeled RNA dataset as originally shown in Table I
Table S2:
RBP
LSTMVAE
GraphVAE HierVAE
HuR
0.867
0.858
0.860
PTB
0,886
0.878
0.883
QKI
0.748
0.756
0.746
Vtsl
0.775
0.758
0.774
RBMY
0.734
0.725
0.731
SF2
0.867
0,862
0,866
SLBP
0.749
0.737
0.747
19

---

Published as a conference paper at ICLR 2021
L END-TO-END RNACOMPETE-S CLASSIFERS
Table S3:
We use the same encoding architectures as in the generative models, and report their AUROC averaged across 6 runs, for each RNAcompete-S RBP dataset.
RBP
LSTM-SeqOnly
LSTM
Graph
Hierarchical
HuR
0.880 + 0.000
0.880 = 0.000
0.880 + 0.000
0.888 + 0.002
0,900 0.000
0.910 = 0.000
0.910 0.000
0.910 + 0.000
QKI
0.820 + 0.000
0.830 = 0.000
0.825 + 0.002
0.830 + 0.000
Vts1
0.900 + 0.000
0.908 = 0.002
0.637 = 0.079
0.910 + 0.000
RBMY
0.905 + 0.002
0.880 = 0.003
0,802 = 0.055
0.870 0.002
SF2
0,890 0.000
0.900 = 0.000
0.900 0.000
0.900 + 0.000
SLBP
0.777 + 0.002
0.790 = 0.000
0.797 0.002
0.797 + 0.002
M ALTERNATIVE HIERVAE TRAINING ON RNACOMPETE-S
Table S4: Training HierVAE on supervised RNAcompete-S dataset. All models are trained with 20 epochs, including epochs for warm-up. epochs linearly raise beta from 3e-3, and remaining epochs with beta fixed at 3e-3. The test set measures AUROC and posterior decoding on the final model.
Test
Post R&S
Post NR&D
Dataset
AUROC
Valid
FE DEV
RECON ACC
Valid
FE DEV
RECON ACC
HuR
0,871
100%
0,951
18.97%
99.34%
0.702
31.52%
PTB
0.899
100%
0.826
21.17%
98.64%
0.674
31.28%
QKI
0.822
100%
0.867
17.82%
99.40%
0.627
30.62%
Vts1
0.874
100%
1.056
13.71%
99.39%
0.770
24.97%
RBMY
0,872
100%
0,963
11.86%
98.68%
0.690
22.91%
SF2
0.874
100%
0.921
14.44%
99.32%
0.668
25.99%
SLBP
0.764
100%
1.033
14.84%
99.44%
0.743
26.93%
20

---

Published as a conference paper at ICLR 2021
N COMPARISON OF GENERATED RNA SECONDARY STRUCTURES TO MFE STRUCTURES
1E:-19.10 (D00)
MFE
-12 80 (DEM: 0.00)
FEI (DEV 1.601
4E:-15.40 (DEV: 0.00)
MFE
-6.3601 (DEV: 0.00)
KFE
OCand
OCed
FC:-15.00 (DEV:
MFE
FE: 33.800 (DEV. 0.00)
NFE
7.20)
-19.70 (DEV: 2.001
FE:: 100. 00 (DEV: 1.70)
MITE
(DEV.
-18.70
PE:-115 40 (DEV: 2.601
MPE
FE:-10.80 (DEV: 0.00)
MFE
FE: -8-08020 (DEV: 6.82)
MFE
Coe
Figure S3: A comparison of generated RNAs (left) to their corresponding MFE structures (right). RÑAs are generated with structural constraints from HierVAE on three random axes. The ground truth MFE structures in the latent space along with their corresponding MFE structures which have also shown relatively smooth transitions.
predicted by RNAfold, and the generated RNAs shown to evolve smoothly
21

---

Published as a conference paper at ICLR 2021
O NEIGHBORHOOD VISUALIZATION OF A CYSTEINE-CARRYIN TRANSFER-RNA
38 (DEV. 1801
PE:43 05 500M 5.800
FE: 22.60 19EV 6,30
PE: -2.00 IDEV 4.10
PE: IDEV 7.901
22 (ee
90 sav 8.900
1850
8.000
IDEV
50 IDeV: 0.001
45 10 jOPv 5.000
45.44 JDEV 6.20
rE 33.50 1.00
201
Figure S4: Neighborhood visualization of tRNA-Cys which is marked by the red bounding box the center and the walk in the latent space takes place on two random orthogonal axes. Note that actual secondary structure tRNA-Cys plotted in the figure is different compared to the one deposited online due to the prediction of RNAfold.
22

---

Published as a conference paper at ICLR 2021
P NEIGHBORHOOD VISUALIZATION OF A 5S RIBOSOMAL RNA
FC:24.1
rE: -2.20 (DEV 4,39)
-4--2480010 (0EV:0.0)
-7.40 (D2V: 0400
TE:-42.50 ICEV: 1.00)
-1. IDEV 0.00)
I0EV:
-37.10 (DDV. 0.00)
1E:-45.00 0.0
Compu
FC:440.0
TD: -22.0 (DEV 1.000
F2: 422.0 50 IDEV:
-38.0 (Ozv. 0.200
TE: T2:-42.10 ICEV 0.83)
6.001
6.300
-01-0 IBEV: 00)
44.10 gaov 4.000
4
Coe
FC: 12:-42.59
FE: -7.077000 IDEN 0.100
FR: FE:467.0 IDEV. 0.001
-7.80 (ozV. 1.500
43.40 2.0V
Que OeeD
Figure S5: Neighborhood visualization of 5S ribosomal RNA which is marked by the red bounding box in the center and the walk the latent space takes place on two random orthogonal axes. Note that actual secondary structure of this 5S ribosomal RNA plotted in the figure is different compared to the one deposited online due to the prediction of RNAfold.
23

---

Published as a conference paper at ICLR 2021
Q TARGETED RNA GENERATION AN EXAMPLE
0.40,
DEV:
DEV: PRED.
FE
PRED:
2.90, PRED:
DEV: PRED: 1.00
FE 2.10, PRED:
DEV: 1.50, PRED: 1.00
FE 0,00, PRED: 0.99
DEV: 2.50, PRED: 1.00
DEV: 0.00, PRED: 1.00
FE DEV: 7.70, PRED: 1.00
DEV: PRED: 1.00
FE DEV: 8.40, PRED: 1.00
DEV: 7.30, PRED: 1.00
Figure S6: An example of searching novel structured RNAs with higher chance of binding to HuR. The optimization takes place in the latent space of HierVAE, starting from the initial encoding of random RNA molecule the test set, and at each step altering the latent encoding by using activation maximization on the embedding classifier. The trajectory of generated RNAs is shown in the order of left to right and top to bottom, and the field PRED indicates that the probability of binding, as predicted by another external full classifier the decoded molecular structure, is overall increasing as the decoded RNA structures smoothly evolving.
24


Reviews:
[{"review_id": "u1mN0RVn0oV", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary: This paper proposes 3 deep generative models based on VAEs (with different encoding schemes for RNA secondary structure) for the generation of RNA secondary structures. They test each model on 3 benchmark tasks: unsupervised generation, semi-supervised learning and targeted generation.  This paper has many interesting contributions — a comparison of VAE models that use different RNA secondary structure encoding schemes, including traditional dot-bracket notation and a more complex hierarchical encoding, and they also introduce various decoding schemes to encourage valid secondary structures. \n\nThis is an important problem for drug discovery and basic biology, but the evaluation is very tricky because there are not that many solved RNA structures (unlike proteins). The tasks proposed in this paper are not comprehensive enough to entice a (comp) biologist to be convinced one way or another, but it does provide an introduction to the problem for the ML field. \n\nComments:\n* While the trends in their results are clear, it’s difficult to know which values of Diversity, FE DEV, and Validity are good enough to say that they have made significant progress relative to the field. The main issue is that all model comparisons are only with their own VAEs. It would be nice to show how their VAEs stack up against other established methods. For unsupervised generation, perhaps Infernal (Nawrocki and Eddy. Bioinformatics, 2017), which uses a stochastic context-free grammar built up from sequences belonging to a single RNA family, but could still form the basis for a comparison. Other models include CONTRAfold (Do et al. Bioinformatics, 2006) and many other thermodynamic models (Lorenz et al. Algorithms for Mol Biol, 2011). These other approaches may lack the scalability of a VAE, but an effort should be made for at least a limited comparison.\n* It’s difficult to assess the quality of the generated secondary structures shown in various plots throughout the paper and appendices. This paper should plot a side by side comparison with the ground state structure that they are comparing the FE DEV scores against to see how well the structure corresponds. For instance, the generated (but invalid) structure may visually look similar as they walk through latent space, but the key mismatches can fundamentally change the “ground truth\" structure of the RNA (which is not shown in their generated structure). The ground truth structure is given by RNAfold, which itself is not perfect.  \n* Evaluation is very tricky and it’s unclear how to navigate this.  Careful wording can help differentiate true objectives (functional RNA) from difficult evaluations (FE DEV scores). If minimizing FE DEV is the objective, then I worry that models will simply become function approximators for RNAfold. \n* A good comparison (though limited) would be to compare the predicted structures with solved structures deposited in the PDB. Another comparison could be to sample about the posterior near well known non-coding RNAs, such as tRNA or 5S ribosomal RNA. Many more consensus RNA structures can be found in RFAM (Kalvari et al. NAR, 2018).  \n* On a side note, the comparison on RNAcompete-S in Table S3 shows that a sequence model is comparable to models that incorporate RNA secondary structures — there is only a small gain by including secondary structures. This is not that surprising as sequence-based models have demonstrated learning secondary structures when trained only on sequences (for the RNAcompete dataset — not RNAcompete-S yet).\n* One lingering question I had is, is the latent space of the VAE meaningful? For instance, does it capture well-established non-coding RNA families? \n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "ntTCg-4BgyX", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Brief summary: The authors describe a generative model constrained to model the primary and secondary structure of RNA.\n\nPros:\n\n- I applaud the authors for working with RNA data. This is less characterization of modeling RNA data.\n\n- I think the constraints for the RNA generation structure are reasonable, and both the manual and ML constrained generative process are well thought out.\n\n- I think the comparison between constrained and unconstrained generation of molecules was well done.\n\nCons:\n\n- This paper falls into the common pitfall of not controlling for homologous sequences between the training and test set. For each sequence in the training set, what is the sequence with the largest sequence ID in the test set by alignment?\n\n- It would be great to have a null distribution to characterize your model to. What is FE DEV, Normed, and Diversity for randomized, but kmer controlled, RNA sequences?\n\n- The definition of \"valid structures\" is poorly defined. Could generated sequences still fold if optimized in a traditional biophysical simulation?\n\n- I think a comparison with an algorithm like INFERNAL (http://eddylab.org/infernal/Userguide.pdf) is necessary. This is the most simple context-free grammar generative model.\n\nNeutral:\n\n- It is difficult to assess the range of \"good\" values in Table 1. To aid in interpretability to the reader, I would advise more explicitly denoting whether diversity, FE DEV, and Normed should be metrics that go up or down with improved performance.\n\n- I think it would be wise to train this sort of model on more data, such as RFAM. This is an already curated database of RNAs by sequence family from a large number organisms, much more than just human sequences.\n\n- Is there any way to evaluate valid tertiary contacts? These would be sequence motifs that are preserved across organisms with constrained sequence and function that may not necessarily follow Watson-Crick basepairing rules.\n\n- It is slightly concerning to me that in Table 1, \"Validity\" is the same for LSTMVAE and GraphVAE in *Constrained & Posterior Decoding* and \"Diversity\" is the same for GraphVAE and HierVAE *Constrained & Prior Decoding*. Why is this the case?\n\n- Page 2, typo: “benchmhark datasets”\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "fenvjcpgcmc", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "##########################################################################\n\nSummary:\n\n \nThis paper sheds light into an impactful problem of neural representation and generation for RNA secondary structures. Authors presented benchmark tasks in unsupervised, semi-supervised and targeted generation setting and presented deep generative models to solve this tasks. They presented three different generative models using variational auto encoders based on sequence, graph and hierarchical representation. \n\n\n##########################################################################\n\nPros:\n \n1. The paper deals with a very important problem of neural representation and generation for RNA secondary structures. I think this area needs a lot more work to help solving real-world biological problems.\n\n \n2. As far as I can tell their method is novel in terms of idea. Generation of RNAs are not trivial. It requires extra attention to detail in terms of checking for validity and stability in folding. There are also diverse families of RNA which are very different from each other. In this work, the validity and stability constraints are implemented inside a depth first search traversal of building the RNA. Without these constraints its much harder to learn the structures of the RNA from embedding only as it is evident from the result on the unsupervised setting as well. In supervised setting, the authors showed their models efficacy in RBP datasets. Related works also contains recent works in RNA.\n \n3. Overall the paper is well written. I liked the illustrations for explaining the methods. Result section is also well structured. It clearly shows the effectiveness of the generative model used.\n \n##########################################################################\n\nCons: \n\n \n1. Is it possible to use this generation technique to use in solving problems like RNA folding or targeted RNA design? Specially the authors mentioned RL based method from (Runge et al., 2019). Is it possible to use this generative model and combine it with the existing RL techniques for RNA design or ML techniques for folding to show improvement in their result? That would definitely increase the impact of this work.\n\n2. Is it also feasible to learn the structural constraints from the data? Can we get rid of the dependency on constraints with more automated learning?\n\n3. I am also interested to know if we can compare it against any discriminative learning based baseline for any task. For example a recent work from (Yan et. al 2020) for RNA protein interaction.\n\n##########################################################################\n\nQuestions during rebuttal period: \n\n \nPlease address and clarify the cons above \n\n \n#########################################################################\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "_BEPdG5sPEZ", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "**Summary**\nThe work studies of the problem of jointly modelling RNA sequence and secondary structure using the framework of variational autoencoders. Several encoder and decoder architectures with increasing inductive bias for RNA structures are proposed. These are compared on two novel RNA sequence datasets and supervised and unsupervised RNA generation tasks.\n\n**Score justification**\nThe work presents a comparison of several interesting approaches to the problem of joint modelling of RNA structure and sequence using VAEs. Encoders and decoders that make use of (i) RNA sequence and structure in a string representation; (ii) in a graph representation; and (iii) in a junction tree representation are considered. This works particularly well on the encoder side, where the appropriate deep learning architectures are used; but becomes considerably less elegant on the decoder side, where the authors have to make use of sample masking with complex rules to prevent generation of invalid sequence-structure combinations. Furthermore it is not immediately clear from the empirical evaluation that the added complexity of the HierVAE, or that using VAEs are an optimal choice for the supervised generation task.\n\n\n**Major comments**\n* As noted by the authors, their decoders (linearized sequence and structure, as well as the hierarchical decoder) may generate invalide sequence-structure combinations. To eliminate this possibility the authors restrict the autoregressive decoder from sampling invalid sequence-structures by masking out some samples using a number of heuristic rules. First of all, it is unclear from the text whether this is also done during training and how the probabilities at masked stages are treated (are they re-normalized after the masking is applied?)\n* I appreciate that the authors include a comparison between decoding with and without the heuristic masking rules. Could the authors explain why the FE DEV in Table 1 is lower for samples produced without the masking rules (unconstrained generation)? This result seems counter-intuitive.\n* From the results in Table 1 it is still difficult to tease apart the relative contributions of (i) model training; (ii) inductive bias of decoder architecture; and (iii) decoding constraints. I would appreciate it if the authors also included decoding results for untrained decoders with and without constraints.\n* Comparing the AUC ROC in Table 2 to the results in Tables S2 and S3 it is not obvious what the benefit of using a VAE setup for the supervised problem is. The purely supervised approach is conceptually and technically easier.\n* I am a bit skeptical about the results presentable in Table 3 and section \"Targeted RNA design\" - judging improvement of a sequence using a different classifier trained on the same data is not very convincing. Ideally, empirical wet lab validation should be carried out for the improved designs; but absent that it would be great to see a more independent/less correlated evaluation. Perhaps the authors could demonstrate a reduction of edit distance of an improved negative test set sequence to a positive (test set / train set) sequence?\n* When generating the dataset for the unsupervised RNA modelling task the authors take human transcripts and draw random short (32 to 512 nucleotide) sequences from them for their dataset. Since the authors introduce a new dataset it would great to see more information about its composition - which kind of RNAs compose the dataset (e.g. mRNA, tRNA, lincRNA, shRNA, rRNA, etc); does this include alternatively spliced RNA (i.e. could there be leakage between the training and holdout sets)? What is the reasoning between random slicing of the RNAs into shorter sequences? Are the smaller RNA slices biologically plausible/relevant?\n* Could the authors provide additional justification / ablations of the choice of $\\beta \\ll 1$? How do the models behave if KL annealing is not used?\n\n\n**Minor comments**\n* The abstract states \"the design of large scale and complex biological structures *requires* dedicated graph-based deep generative modelling techniques\", which I believe is a too strong statement. These techniques work better right now, but does not mean that they are required (and the only appropriate way forward).\n* In \"Task Description\" section: \"functional properties *or* RNA\" -> \"functional properties of RNA\"\n* Is Figure 1 using some domain-specific terminology? It refers to cliques, but the nucleotides put into boxes are not all fully connected.\n* Why doesn't constrained generation (Table 2) achieve 100% validity? It seems to me that it should be possible at least for the linearized sequence-structure decoder.\n* Apologies if I missed it in the text, but what does \"RECON ACC \" in Table 2 refer to?\n* Incorrect opening quotes used for \"ground truth\" in section \"Targeted RNA design\"\n* In Appendix A there is \"$j > j$\"\n* In Appendix B: \"in-vitro\" -> \"*in vitro*\" and probably in italic\n* Why is FE DEV defined as the absolute value? Should not the MFE structure always have smaller Free Energy than any structure output by the model?", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Four knowledgeable referees support acceptance for the contributions, and I also recommend acceptance. There is agreement among all reviewers that this paper is about  a highly relevant topic, that the model presented is technically sound and has significantly novel aspects, and that the experimental results are convincing. There were several points of criticism raised by the reviewers, concerning, for instance, further comparison experiments,  the heuristic nature of masking rules, or the treatment of homologous sequences. In my opinion, however, most of these points have been addressed in a rather convincing way during the rebuttal phase.  
Decision: Accept (Poster)