Neural representation and generation for RNA secondary structures

Abstract: Our work is concerned with the generation and targeted design of RNA, a type of genetic macromolecule that can adopt complex structures which influence their cellular activities and functions. The design of large scale and complex biological structures spurs dedicated graph-based deep generative modeling techniques, which represents a key but underappreciated aspect of computational drug discovery. In this work, we investigate the principles behind representing and generating different RNA structural modalities, and propose a flexible framework to jointly embed and generate these molecular structures along with their sequence in a meaningful latent space. Equipped with a deep understanding of RNA molecular structures, our most sophisticated encoding and decoding methods operate on the molecular graph as well as the junction tree hierarchy, integrating strong inductive bias about RNA structural regularity and folding mechanism such that high structural validity, stability and diversity of generated RNAs are achieved. Also, we seek to adequately organize the latent space of RNA molecular embeddings with regard to the interaction with proteins, and targeted optimization is used to navigate in this latent space to search for desired novel RNA molecules.

Reviews and author responses:

As two of our reviewers (R2, R4) suggested , it is worth considering a comparison to Infernal (Nawrocki and Eddy. 2013), to which our hierarchical approach shares certain similarity due to the modelling of RNA secondary structure using stochastic context free grammar (SCFG). 

In Infernal, the use of SCFG leads to the so-called covariance model (CM), which can be understood as an adaption of profile HMM to the tree-like structure of nested RNA secondary structures. And much similar to the idea of searching new DNA/protein sequences with a profile HMM, which extends simple position specific scoring with insertions and deletions around a consensus sequence that captures conserved positions and residues in a multiple sequence alignment (MSA), **a CM is built around a consensus RNA secondary structure** that models conservation of base-pairs (covariation of paired nucleotides) with an essentially tree-structured HMM. 

Therefore, a CM is mostly used to search new membership in a sequence database, which belongs to a highly conserved RNA family, or to align sequences to make secondary structure predictions. On the other hand, as a generative model, we can also sample from a CM model to obtain an RNA sequences along a parse tree which should represent its secondary structure, which as we understand is where the suggestions of comparison have come from.

Conceptually, there are several issues with this evaluation approach:
1. A CM reflects the consensus secondary structure of a conserved RNA family which itself is built on, therefore all sampled structures from this CM will have by and large similar secondary structures, hence of low structural diversity.
2. A CM captures covariation of paired bases, and it alone is not good at predicting RNA folding, unless the RNA family is highly conserved. Therefore, sampled structures have no guarantee of folding stability.

We have also attempled to build a CM model using our datasets, using procedures:
MAFFT (randomly selected 3000 sequences) --> RNAalifold --> cmbuild (infernal) --> cmemit (infernal samples)
where MAFFT is used to obtain MSA of RNAs (sliced short snippets in the unlabeled dataset, or oligonucleotides in the labeled RNAcompete-S dataset), and RNAalifold is used to obtain consensus secondary structure.

Our attempts always failed at steps 1 or 2. The consensus secondary structure predicted by RNAalifold does not contain any conserved columns, simply due to the fact that both datasets lack homologous sequences. The outcome is that all sampled RNAs from these CMs are single stranded.

However, a comparison set-up with Infernal will become more meaningful, once we move our approach to RFAM and its many conserved RNA families. We consider it a very important direction, which we have indicated in the revised manuscript. We also thank the reviewers for pointing out Infernal.

**Summary**
The work studies of the problem of jointly modelling RNA sequence and secondary structure using the framework of variational autoencoders. Several encoder and decoder architectures with increasing inductive bias for RNA structures are proposed. These are compared on two novel RNA sequence datasets and supervised and unsupervised RNA generation tasks.

**Score justification**
The work presents a comparison of several interesting approaches to the problem of joint modelling of RNA structure and sequence using VAEs. Encoders and decoders that make use of (i) RNA sequence and structure in a string representation; (ii) in a graph representation; and (iii) in a junction tree representation are considered. This works particularly well on the encoder side, where the appropriate deep learning architectures are used; but becomes considerably less elegant on the decoder side, where the authors have to make use of sample masking with complex rules to prevent generation of invalid sequence-structure combinations. Furthermore it is not immediately clear from the empirical evaluation that the added complexity of the HierVAE, or that using VAEs are an optimal choice for the supervised generation task.


**Major comments**
* As noted by the authors, their decoders (linearized sequence and structure, as well as the hierarchical decoder) may generate invalide sequence-structure combinations. To eliminate this possibility the authors restrict the autoregressive decoder from sampling invalid sequence-structures by masking out some samples using a number of heuristic rules. First of all, it is unclear from the text whether this is also done during training and how the probabilities at masked stages are treated (are they re-normalized after the masking is applied?)
* I appreciate that the authors include a comparison between decoding with and without the heuristic masking rules. Could the authors explain why the FE DEV in Table 1 is lower for samples produced without the masking rules (unconstrained generation)? This result seems counter-intuitive.
* From the results in Table 1 it is still difficult to tease apart the relative contributions of (i) model training; (ii) inductive bias of decoder architecture; and (iii) decoding constraints. I would appreciate it if the authors also included decoding results for untrained decoders with and without constraints.
* Comparing the AUC ROC in Table 2 to the results in Tables S2 and S3 it is not obvious what the benefit of using a VAE setup for the supervised problem is. The purely supervised approach is conceptually and technically easier.
* I am a bit skeptical about the results presentable in Table 3 and section "Targeted RNA design" - judging improvement of a sequence using a different classifier trained on the same data is not very convincing. Ideally, empirical wet lab validation should be carried out for the improved designs; but absent that it would be great to see a more independent/less correlated evaluation. Perhaps the authors could demonstrate a reduction of edit distance of an improved negative test set sequence to a positive (test set / train set) sequence?
* When generating the dataset for the unsupervised RNA modelling task the authors take human transcripts and draw random short (32 to 512 nucleotide) sequences from them for their dataset. Since the authors introduce a new dataset it would great to see more information about its composition - which kind of RNAs compose the dataset (e.g. mRNA, tRNA, lincRNA, shRNA, rRNA, etc); does this include alternatively spliced RNA (i.e. could there be leakage between the training and holdout sets)? What is the reasoning between random slicing of the RNAs into shorter sequences? Are the smaller RNA slices biologically plausible/relevant?
* Could the authors provide additional justification / ablations of the choice of $\beta \ll 1$? How do the models behave if KL annealing is not used?


**Minor comments**
* The abstract states "the design of large scale and complex biological structures *requires* dedicated graph-based deep generative modelling techniques", which I believe is a too strong statement. These techniques work better right now, but does not mean that they are required (and the only appropriate way forward).
* In "Task Description" section: "functional properties *or* RNA" -> "functional properties of RNA"
* Is Figure 1 using some domain-specific terminology? It refers to cliques, but the nucleotides put into boxes are not all fully connected.
* Why doesn't constrained generation (Table 2) achieve 100% validity? It seems to me that it should be possible at least for the linearized sequence-structure decoder.
* Apologies if I missed it in the text, but what does "RECON ACC " in Table 2 refer to?
* Incorrect opening quotes used for "ground truth" in section "Targeted RNA design"
* In Appendix A there is "$j > j$"
* In Appendix B: "in-vitro" -> "*in vitro*" and probably in italic
* Why is FE DEV defined as the absolute value? Should not the MFE structure always have smaller Free Energy than any structure output by the model?

* Indeed we agree with the reviewer that it is hard to evaluate our models, and comparison to existing tools are lacking for a more comprehensive evaluation. The reasons why we have not considered Infernal or other RNA inverse design methods are as follow:
   * For the comparison with Infernal, a covariance model needs to be built on top of the multiple sequence alignment in an RNA family. However, due to the lacking of homology information in either unsupervised or semi-supervised setting, the multiple sequence alignment is of poor quality and the generated sequence are not useful, nor do they form a reasonable comparison to our design RNA structures since Infernal does not actively fold RNA structures.
   * Concerns with the other thermodynamic models are that they lack internal representation, nor are they learning based, which can result in unfair comparison with our approach since they can query the underlying thermodynamics system which we have used as an "oracle" to evaluate the quality of our generated RNA secondary structures. Besides, these are conditional models that search sequences that can fold into given secondary structures, as compared to our work which considers a joint distribution over sequences and structures.
    
    That being said, we are still searching for an adequate comparison set-up which will very likely make up our future works.

* For the pairwise comparison between decoded RNA structures and their corresponding minimum free energy structures, we included figure S3 in the appendix. It turns out that as we walk in the latent space, both decoded structures and their corresponding MFE structures evolve smoothly. 
* We thank the reviewer for this important concern and we hope to clarify any misunderstanding here.

   The main objective of our VAE model is to learn the grammar of RNA secondary structures and the dynamics of RNA folding so that generated RNA structures are valid and stable. We also seek to organize the latent space of RNA representation as in the semi-supervised generation setting, to reflect certain functional properties such as protein interaction.

    The other concern that our model may become a functional approximator of RNAfold is a good one. We mainly use RNAfold as an “oracle” which is to be frank very far from perfect since predicting RNA folding is still an open question. That being said, our unsupervised dataset has also relied on RNAfold to annotate the secondary structures, which is a limitation and the reason why we did not use experimentally observed data (e.g. on RFAM) is simply because our current evaluation measure cannot explain the observed data well.

    However, one major advantage lies in that our deep generative approach is not restricted to any thermodynamics models and it can learn directly from observed data, which is an immediate step we will explore in the next step. And of course, we would also need to identify better evaluation measures.
* We added Figure S4 which samples RNAs around a Cys-tRNA.
* Compared to the RNAcompete dataset which mainly consists of weakly structured RNAs, RNAcompete-S features higher structural diversity and its primary focus is to investigate the roles played by RNA secondary structures in RNA protein interaction.In addition to the reviewer’s comment that sequence model should have the capability of inferring secondary structures after seeing enough training examples, which may well explain why the inclusion of secondary structures does not tend to improve the accuracy, another plausible explanation may be the noise contained in the RNAcompete-S dataset due to the protocol only involves a single selection steps which cannot differentiate RNAs with high and low binding affinities.

    We thank the reviewer for providing this explanation which we have added into the manuscript. As a future direction, we will investigate datasets curated from other experimental approaches such RNA Bind-n-seq and HTR-SELEX.
* The latent space is meaningful in the sense that (1) similar latent encodings translate to similar RNA sequence and secondary structures, and (2) under the semi-supervised setting the latent space is organized to reflect properties of RBP bindings. Due to limitations posed by our datasets which do not contain non-coding RNA families, we believe our current VAE model cannot capture such information since it is never learnt. However, it may be salvaged by considering a dataset curated from RFAM coupled with new learning techniques, which is included in our next step.


Address to the comments in Cons
* The unlabeled dataset used for unsupervised generation is obtained from the human transcripts by randomly slicing messenger RNAs into short and unique RNA snippets (32-512ns). Therefore, there do not tend to exist homologous sequences in this dataset. The labeled dataset acquired from RNAcompete-S consist of oligonucleotides (uniformly 40 nts) synthesized for in vitro experiments which are not derived from common ancestors, therefore there is also no such notion of homology in this dataset.
* For the sampled RNAs from the dataset which comprise the null distribution, their FE DEV, Normed would be uniformly 0 since they already have the minimum free energy structures. We can add a footnote pointing this out, since having a null comparison can indeed be quite useful.
* The valid structures considered in our study are ones that have canonical and nested basepairs. However, even if an RNA sequence has folded into a “valid” secondary structure as defined above, it is not necessarily as stable as the minimum free energy structure, which will be the primary concern for potentially failing the biophysical simulation.
* Building a covariance model with Infernal requires starting from a multiple alignment of several members of an RNA family. This is problematic in our setting, because for example in the supervised dataset, the positive sequences are not homologous to each other (i.e. they are not derived from a common ancestor, but are instead selected from a random pool of oligonucleotides). We attempted to build a multiple alignment from positive sequences using MAFFT, but the alignment was of extremely poor accuracy, making the covariance model learned from it useless.

Address to the comments in Neural
* We have added up/down arrows next to the metrics in Table 1.
* Indeed, we agree with the reviewer that it is an important future direction to consider the evolutionary information inherent in the RFAM database, which will by no doubt enable us to design RNA secondary structures that have meaningful functions pertaining to certain RNA families such as regulatory roles, since there exists crucial RNA structural motifs from certain RNA families that are evolutionarily conserved. We have updated our manuscript to include a discussion on this topic for future works.
* We thank the reviewer for this suggestion. Indeed, it would be interesting to analyze our models in depth and see if it can learn RNA tertiary structural motifs from the CaRNAval RNAMotifs database, which will become feasible when we switch to the non-coding RNA families dataset from RFAM. Our current method would also require major upgrades to account for non-canonical interactions and pseudoknots which are common in tertiary contacts. This is an important future direction which we have indicated in our updated manuscript.
* After double checking we can assure our reviewer that the scores reported in Table 1 are accurate. Some values are the same due to the rounding to the 2nd decimal in the percentages. For example, if rounding to the 5th decimal, the validity scores for LSTMVAE and GraphVAE under the Constrained & Posterior Decoding setting would be 99.47496% and 99.46935% respectively. The diversity scores for GraphVAE and HierVAE under the Constrained & Prior Decoding setting would be 6.79065 and 6.79079
* Thanks for pointing out the typo

We thank the reviewer for the interest in our work! We hope to address the current limitations with new data combined with new generative modeling techniques.

1. For RNA folding or RNA inverse design, the mode of generation is conditional in the sense that a secondary structure is predicted based on the input sequence (folding), or an RNA sequence is predicted based on the input secondary structure (inverse design).

    For folding, one would certainly consider combining graph representation learning with reinforcement learning, where a graph neural network is used to acquire representation of an intermediately folded RNA secondary structure, and based on this intermediate representation, an RL agent samples actions such as pairing two bases in the graph. There already exists similar works such as You et al. 2018 where the authors proposed a graph convolutional policy network for targeted molecular design.

    For the inverse design problem, one could also imagine that in the beginning an input secondary structure is given which misses node identities, and we need to specify a nucleotide for each node in the graph. Once again, GNN can be used to compute representation of this intermediately filled graph, and an RL agent samples a nucleotide for each unpaired region, or a pair of nucleotides in the paired region, in a similar fashion as in Runge et al. 2019.

    In our work, the hierarchical modelling technique of RNA molecules can be certainly used as functional approximators in the RL algorithm under the conditional generation setting. Alternatively, under the joint sequence and structure modelling setting as employed by our work, RL can be potentially used to further fine tune our decoder to decode more stable and meaningful RNAs. 
2. We believe the answer is yes and a possible solution lies in generative pre-training using evolutionary data collected from a broad spectrum of organisms and RNA families, followed by finetuning using with reinforcement learning. The pre-training step is essential for the purpose of learning biologically-relevant and evolutionarily conserved knowledge into the model, such as meaningful and stable folding of RNAs that belong to certain RNA families, which will play a big role in the subsequent RNA generation step. Afterwards, reinforcement learning with rewards defined on structural validity, folding stability as well as customizable goals for example RBP binding can be used to further finetune the pre-trained model. And notably, the structural constraints mentioned in our current work can be applied to the reinforcement learning as a form of rejection sampling during training.
3. Indeed our modelling choices for the graph perspective of RNA molecules also use graph neural networks as in Yan et al.2020. One major distinction, however, lies in that Yan et al. 2020 developed computational models for in-vivo RBP binding predictions, which were trained and evaluated with RNA snippets that come from much longer mRNAs in their actual cellular environment. On account of that, they used base-pairing probability annotated adjacency matrix to represent an ensemble of possible foldings, contrary to our work which uses a single minimum free energy structure for each RNA, because our work has focused on in-vitro RBP bindings, which means all the RNAs are experimentally synthesized short sequences with tractable secondary structures. Therefore, our work is not directly comparable with Yan et al. 2020 although both have studied RNA protein interaction.


We thank the reviewer for the constructive feedbacks. We have updated our manuscript with clarification to points that were previously confusing, and new results that were suggested.

> This works ... becomes considerably less elegant on the decoder side, where the authors have to make use of sample masking with complex rules to prevent generation of invalid sequence-structure combinations

To our knowledge, manually adding structural constraints to the decoding process is a common practice in computational drug discovery (similar construction can be seen in JTVAE, CGVAE, GCPN, GraphAF andMolecularRNN). Rather than merely adding to the complexity of our approach, these rules can help our models decode close to 100% valid RNAs, and they are a crucial source of domain knowledge can describes understanding of the RNA secondary structures, based on which we can influence the design process.

#### Address to major comments:

* The heuristic masking rules are only applied at test time (i.e., inference) and not needed during training; since the nucleotide sequences and secondary structures are known during training, we can use a teaching forcing strategy, meaning that the structures generated during training are always valid. Regarding the treatment of probabilities, when we apply the masks at test time, the constraints masks are added to the logits before the probabilities can be calculated using softmax (i.e., renormalization is performed). We have clarified both of these points in the manuscript. 
* The set of masking rules are designed in a way that decoded RNAs will adopt valid secondary structures by constraining the sample taken at each decoding step to valid basepairs and structural topology. However, these constraints do not have any knowledge of the stability of RNA secondary structures, nor do they understand how these actions will influence the free energy of RNA folding since they are hard coded. Therefore, by manually asserting the influence of domain knowledge, RNA stability is adversely influenced. However, we also showed that the inductive bias constructed into the decoder architecture can simultaneously improve the validity and stability of decoded RNAs, and coupled with hard coded structural constraints, the validity can approach 100% without greatly compromising the stability.
*  Adding a comparison to untrained decoders is a great suggestion, and we have updated our manuscript accordingly. Using an untrained HierVAE model, we find out that the posterior and prior decoding under the constrained and stochastic setting will produces validity of 66.30% and 66.34% respectively, with very high free energy deviation — 22.313 and 22.613, indicating that the generated secondary structures of RNA are of very low stability. Under the unconstrained setting, the validity drops to 9.51% and 9.37% for the posterior and prior, and the model can only decode single stranded RNAs that lack more complex secondary structures. The structures decoded under this setting are not useful at all, since the model does not have the knowledge of how to construct more complex RNA topology, as it is neither learned nor enforced via decoding constraints. In the end, this comparison shows that although decoding constraints help improve the validity, it alone is not capable of generating RNAs with more stable secondary structures. Modeling training is imperative for obtaining more stable RNA folding.
* For targeted RNA generation, we need to organize the latent space to help the search for RNAs with desired properties, and adding auxiliary supervision on the latent encodings helps provide this structure. In the end, we demonstrated that a semi-supervised VAE has the discriminative power comparable to a fully supervised classification model. Indeed, a fully supervised classification model would be more straightforward and simpler to fit, however, our main objective is to show that our VAEs can adequately organize the latent space, rather than to compare with these predictive systems.






* >what does "RECON ACC " in Table 2 refer to?

    It refers to reconstruction accuracy, which measures the fraction of RNA molecules decoded exactly as the input. We have clarified this abbreviation in the Table 2 caption.
* Thanks for pointing out the typo.
* > Why is FE DEV defined as the absolute value?

    That is correct. The RNA secondary structure generated by our model would always have higher free energy than that of the MFE structure when it is evaluated by RNAfold, even if the predicted structure is indeed biologically more stable. We have removed “absolute” from the manuscript.

*  About the results in Table 3, we thank the reviewer for pointing out this possible correlation in the evaluation, which we will acknowledge in the revision, and for the interesting suggestion for an alternative evaluation approach. Unfortunately, however, there are some issues with a edit-distance-based evaluation approach, namely:

    * Sequence/structure based similarity measures cannot accurately assess the capability of RNA interacting with proteins. First of all, RBP binding motifs are known to be short (5-10nt) which can be embedded into a relatively larger RNA. Therefore, two sequences with low edit distance may interact differently with the same protein if the sequence differences alters key binding motif; on the other hand, two sequences with high edit distance may bind to the same protein if they possess the same motif disregarding the rest in the RNA.
    * Secondly, RNAcompete-S experiment protocol uses a single selection step which does not differentiate high and low binding affinities of RNAs, suggesting the existence of false positives in the curated dataset. Without explicitly extracting the knowledge of RBP binding, simply calculating the edit distance to the positive sequences as in a k-nearest-neighbor approach would thus provide a fairly inaccurate assessment of binding potential.
    * In the RNAcompete-S dataset, all sequences are of uniform length (40nts), compared to our designed RNAs which are of variable length and not necessarily constrained to preserve the composition of its nucleic acids or the topology of its secondary structure. Therefore, we observe the edit distance computed by super-n-motifs (Glouzon et al. 2017) or ERA  (Zhong et al. 2013), which both take into account RNA sequence and structure into their similarity measures, contains a lot of noise and is not an trustworthy metrics for evaluating the protein binding capability of our designed sequences.

    Based on these issues, we opted to use the evaluation approach proposed in the paper. Our hope is that the train/val/test split used in training our models is sufficient to provide a meaningful signal in this type of evaluation (e.g., the test set is never used for training either model). Of course, we acknowledge it is imperfect (e.g., due to an overlapping training set) and more sophisticated (e.g., wet lab) validation strategies is an active area for our future work.  

*  Indeed, it would be interesting to consider a dataset that contains functional annotation and evolutionary information from RNA families. However, our project is solely concerned with the representation and generation of RNA secondary structures, therefore we do not have those information in the dataset, i.e. an RNA sequence is randomly sliced from the human messenger RNAs, having its secondary structure predicted by RNAfold and it does not belong to any of the non-coding RNA categories. The reason why we slice them to shorter snippets is for computational tractability, in addition to the fact that we cannot reliably obtain MFE for much longer RNA sequences. We admit it will be an important direction in the future to incorporate these information, and we have talked about this in the discussion section in the updated manuscript.

* Using a KL annealing schedule is necessary for our models to avoid posterior collapse, where the decoder becomes independent of the latent encodings provided by the encoder, which is a common failure mode for VAEs with strong autoregressive components, as discussed in He et al. 2019. Our choice of the annealing schedule is entirely based on empirical observations, as to our knowledge, there has yet to exist any principled methods of selecting KL annealing schedule. We have used diagnostic metrics such as mutual information and active units along with a validation set to select a proper annealing schedule which is the one reported in our paper. We added these information to our manuscript.


Address to minor comments:
*  We agree with the reviewer that this is a strong statement as the supporting evidence so far has been sparse and preliminary—deep learning and graph representation methods have yet to dominate the field of design for biological structures and druggable molecules. We replaced the word “requires” to “spurs” in an attempt to tone down the sentence.
* Thanks for pointing out the typo.
* We thank the reviewer for pointing out this abuse of terminology, and we have replaced “clique” with “subgraph” in the manuscript.
* We use a threshold to limit the number of decoding steps, which means the decoding of an RNA will terminate (hence fail) when the threshold has been reached when decoding the RNA topological structure or nucleotide segments. These thresholds are indicated in Algorithm 1 in the appendix, and we have made clarifications in the result section of the manuscript.


Summary: This paper proposes 3 deep generative models based on VAEs (with different encoding schemes for RNA secondary structure) for the generation of RNA secondary structures. They test each model on 3 benchmark tasks: unsupervised generation, semi-supervised learning and targeted generation.  This paper has many interesting contributions — a comparison of VAE models that use different RNA secondary structure encoding schemes, including traditional dot-bracket notation and a more complex hierarchical encoding, and they also introduce various decoding schemes to encourage valid secondary structures. 

This is an important problem for drug discovery and basic biology, but the evaluation is very tricky because there are not that many solved RNA structures (unlike proteins). The tasks proposed in this paper are not comprehensive enough to entice a (comp) biologist to be convinced one way or another, but it does provide an introduction to the problem for the ML field. 

Comments:
* While the trends in their results are clear, it’s difficult to know which values of Diversity, FE DEV, and Validity are good enough to say that they have made significant progress relative to the field. The main issue is that all model comparisons are only with their own VAEs. It would be nice to show how their VAEs stack up against other established methods. For unsupervised generation, perhaps Infernal (Nawrocki and Eddy. Bioinformatics, 2017), which uses a stochastic context-free grammar built up from sequences belonging to a single RNA family, but could still form the basis for a comparison. Other models include CONTRAfold (Do et al. Bioinformatics, 2006) and many other thermodynamic models (Lorenz et al. Algorithms for Mol Biol, 2011). These other approaches may lack the scalability of a VAE, but an effort should be made for at least a limited comparison.
* It’s difficult to assess the quality of the generated secondary structures shown in various plots throughout the paper and appendices. This paper should plot a side by side comparison with the ground state structure that they are comparing the FE DEV scores against to see how well the structure corresponds. For instance, the generated (but invalid) structure may visually look similar as they walk through latent space, but the key mismatches can fundamentally change the “ground truth" structure of the RNA (which is not shown in their generated structure). The ground truth structure is given by RNAfold, which itself is not perfect.  
* Evaluation is very tricky and it’s unclear how to navigate this.  Careful wording can help differentiate true objectives (functional RNA) from difficult evaluations (FE DEV scores). If minimizing FE DEV is the objective, then I worry that models will simply become function approximators for RNAfold. 
* A good comparison (though limited) would be to compare the predicted structures with solved structures deposited in the PDB. Another comparison could be to sample about the posterior near well known non-coding RNAs, such as tRNA or 5S ribosomal RNA. Many more consensus RNA structures can be found in RFAM (Kalvari et al. NAR, 2018).  
* On a side note, the comparison on RNAcompete-S in Table S3 shows that a sequence model is comparable to models that incorporate RNA secondary structures — there is only a small gain by including secondary structures. This is not that surprising as sequence-based models have demonstrated learning secondary structures when trained only on sequences (for the RNAcompete dataset — not RNAcompete-S yet).
* One lingering question I had is, is the latent space of the VAE meaningful? For instance, does it capture well-established non-coding RNA families? 



Brief summary: The authors describe a generative model constrained to model the primary and secondary structure of RNA.

Pros:

- I applaud the authors for working with RNA data. This is less characterization of modeling RNA data.

- I think the constraints for the RNA generation structure are reasonable, and both the manual and ML constrained generative process are well thought out.

- I think the comparison between constrained and unconstrained generation of molecules was well done.

Cons:

- This paper falls into the common pitfall of not controlling for homologous sequences between the training and test set. For each sequence in the training set, what is the sequence with the largest sequence ID in the test set by alignment?

- It would be great to have a null distribution to characterize your model to. What is FE DEV, Normed, and Diversity for randomized, but kmer controlled, RNA sequences?

- The definition of "valid structures" is poorly defined. Could generated sequences still fold if optimized in a traditional biophysical simulation?

- I think a comparison with an algorithm like INFERNAL (http://eddylab.org/infernal/Userguide.pdf) is necessary. This is the most simple context-free grammar generative model.

Neutral:

- It is difficult to assess the range of "good" values in Table 1. To aid in interpretability to the reader, I would advise more explicitly denoting whether diversity, FE DEV, and Normed should be metrics that go up or down with improved performance.

- I think it would be wise to train this sort of model on more data, such as RFAM. This is an already curated database of RNAs by sequence family from a large number organisms, much more than just human sequences.

- Is there any way to evaluate valid tertiary contacts? These would be sequence motifs that are preserved across organisms with constrained sequence and function that may not necessarily follow Watson-Crick basepairing rules.

- It is slightly concerning to me that in Table 1, "Validity" is the same for LSTMVAE and GraphVAE in *Constrained & Posterior Decoding* and "Diversity" is the same for GraphVAE and HierVAE *Constrained & Prior Decoding*. Why is this the case?

- Page 2, typo: “benchmhark datasets”


##########################################################################

Summary:

 
This paper sheds light into an impactful problem of neural representation and generation for RNA secondary structures. Authors presented benchmark tasks in unsupervised, semi-supervised and targeted generation setting and presented deep generative models to solve this tasks. They presented three different generative models using variational auto encoders based on sequence, graph and hierarchical representation. 


##########################################################################

Pros:
 
1. The paper deals with a very important problem of neural representation and generation for RNA secondary structures. I think this area needs a lot more work to help solving real-world biological problems.

 
2. As far as I can tell their method is novel in terms of idea. Generation of RNAs are not trivial. It requires extra attention to detail in terms of checking for validity and stability in folding. There are also diverse families of RNA which are very different from each other. In this work, the validity and stability constraints are implemented inside a depth first search traversal of building the RNA. Without these constraints its much harder to learn the structures of the RNA from embedding only as it is evident from the result on the unsupervised setting as well. In supervised setting, the authors showed their models efficacy in RBP datasets. Related works also contains recent works in RNA.
 
3. Overall the paper is well written. I liked the illustrations for explaining the methods. Result section is also well structured. It clearly shows the effectiveness of the generative model used.
 
##########################################################################

Cons: 

 
1. Is it possible to use this generation technique to use in solving problems like RNA folding or targeted RNA design? Specially the authors mentioned RL based method from (Runge et al., 2019). Is it possible to use this generative model and combine it with the existing RL techniques for RNA design or ML techniques for folding to show improvement in their result? That would definitely increase the impact of this work.

2. Is it also feasible to learn the structural constraints from the data? Can we get rid of the dependency on constraints with more automated learning?

3. I am also interested to know if we can compare it against any discriminative learning based baseline for any task. For example a recent work from (Yan et. al 2020) for RNA protein interaction.

##########################################################################

Questions during rebuttal period: 

 
Please address and clarify the cons above 

 
#########################################################################



Meta-review: Four knowledgeable referees support acceptance for the contributions, and I also recommend acceptance. There is agreement among all reviewers that this paper is about  a highly relevant topic, that the model presented is technically sound and has significantly novel aspects, and that the experimental results are convincing. There were several points of criticism raised by the reviewers, concerning, for instance, further comparison experiments,  the heuristic nature of masking rules, or the treatment of homologous sequences. In my opinion, however, most of these points have been addressed in a rather convincing way during the rebuttal phase.  

Decision: poster-presentations