SentRNA: Improving computational RNA design by incorporating a prior of human design strategies

Designing RNA sequences that fold into specific structures and perform desired biological functions is an emerging field in bioengineering with broad applications from intracellular chemical catalysis to cancer therapy via selective gene silencing. Effective RNA design requires first solving the inverse folding problem: given a target structure, propose a sequence that folds into that structure. Although significant progress has been made in developing computational algorithms for this purpose, current approaches are ineffective at designing sequences for complex targets, limiting their utility in real-world applications. However, an alternative that has shown significantly higher performance are human players of the online RNA design game EteRNA. Through many rounds of gameplay, these players have developed a collective library of "human" rules and strategies for RNA design that have proven to be more effective than current computational approaches, especially for complex targets. Here, we present an RNA design agent, SentRNA, which consists of a fully-connected neural network trained using the eternasolves dataset, a set of 1.8 x 104 player-submitted sequences across 724 unique targets. The agent first predicts an initial sequence for a target using the trained network, and then refines that solution if necessary using a short adaptive walk utilizing a canon of standard design moves. Through this approach, we observe SentRNA can learn and apply human-like design strategies to solve several complex targets previously unsolvable by any computational approach. We thus demonstrate that incorporating a prior of human design strategies into a computational agent can significantly boost its performance, and suggests a new paradigm for machine-based RNA design.  Solving the inverse folding problem for RNA is a critical prerequisite to effective RNA design, an emerging field of modern bioengineering research.1,2,3,4,5 A RNA molecule's function is highly dependent on the structure into which it folds, which in turn is determined by the sequence of nucleotides that comprise it. Therefore, designing RNA molecules to perform specific functions requires designing sequences that fold into specific structures. As such, significant efforts have been made over the past several decades in developing computational algorithms to reliably predict RNA sequences that fold into a given target.6,7,8,9,10,11 Existing computational methods for inverse RNA folding can be roughly separated into two types. The first type generates an initial guess of a sequence and then refines the sequence using some form of stochastic search. Published algorithms that fall under this category include RNAInverse,6 RNA-SSD,7 INFO-RNA,8 NUPACK,10 and MODENA.11 RNAInverse, one of the first inverse folding algorithms, initializes the sequence randomly and then uses a simple adaptive walk in which random single or pair mutations are successively performed, and a mutation is accepted if it improves the structural similarity between the current and the target structure. RNA-SSD first performs hierarchical decomposition of the target and then performs adaptive walk separately on each substructure to reduce the size of the search space. INFO-RNA first generates an initial guess of the sequence using dynamic programming to estimate the minimum energy sequence for a target structure, and then performs simulated annealing on the sequence. NUPACK performs hierarchical decomposition of the target and assigns an initial sequence to each substructure. For each sequence, it then generates a thermodynamic ensemble of possible structures and stochastically perturbs the sequence to optimize the "ensemble defect" term, which represents the average number of improperly paired bases relative to the target over the entire ensemble. Finally, one of the most recent algorithms, MODENA, generates an ensemble of initial sequences using a genetic algorithm, and then performs stochastic search using crossovers and single-point mutations. The second type of design algorithm, exemplified by programs such as DSS-Opt, foregoes stochastic search and instead attempts to generate a valid sequence directly from gradient-based optimization. Given a target, DSS-Opt generates an initial sequence and then performs a gradient-based optimization of an objective function that includes the predicted free energy of the target and a "negative design" term that punishes improperly paired bases. Both types of algorithms have proven effective given simple to moderately complex structures. However, there is still much room for improvement. A recent benchmark of these algorithms showed that they consistently fail given large or structurally complex targets, 12 limiting their applicability to designing RNA molecules for real-world biological applications. A promising alternative approach to RNA design that has consistently outperformed current computational methods is EteRNA, a web-based graphical interface in which the RNA design problem is presented to humans as a game. 13 Players of the game are shown 2D representations of target RNA structures ("puzzles") and asked to propose sequences that fold into them. These sequences are first judged using the ViennaRNA 1.8.5 software package6 and then validated experimentally. Through this cycle of design and evaluation, players build a collective library of design strategies that can then be applied to new, more complex puzzles. These strategies are distinct from those employed by design algorithms such as DSS-Opt and NUPACK in that they are honed through visual pattern recognition and experience. Remarkably, these human-developed strategies have proven more effective for RNA design than current computational methods. For example, EteRNA players significantly outperform even the best computational algorithms on the Eterna100, a set of 100 challenging puzzles designed by EteRNA players to showcase a variety of RNA structural elements that make design difficult. While top-ranking human players can solve all 100 puzzles, even the best-scoring computational algorithm, MODENA, could only solve 54 / 100 puzzles.12 Given the success of these strategies, we decided to investigate whether incorporating these strategies into a computational agent can increase its performance beyond that of current state-of-the-art methods. In this study, we present SentRNA, a computational agent for RNA design that significantly outperforms existing computational algorithms by learning human-like design strategies in a data driven manner. The agent consists of a fully-connected neural network that takes as input a featurized representation of the local environment around a given position in a puzzle. The output is length-4, corresponding to the four RNA nucleotides (bases): A, U, C, or G. The model is trained using the eternasolves dataset, a custom-compiled collection of 1.8 x 104 playersubmitted solutions across 724 unique puzzles. These puzzles comprise both the “Progression” puzzles, designed for beginning EteRNA players, as well as several “Lab” puzzles for which solutions were experimentally synthesized and tested. During validation and testing the agent takes an initially blank puzzle and assigns bases to every position greedily based on the output values. If this initial prediction is not valid, as judged by ViennaRNA 1.8.5, it is further refined via an adaptive walk using a canon of standard design moves compiled by players and taught to new players through the game's puzzle progression. Overall, we trained and tested an ensemble of 165 models, each using a distinct training set and model input . Collectively, the ensemble of models can solve 42 / 100 puzzles from the Eterna100 by neural network prediction alone, and 80 / 100 puzzles using neural network prediction + refinement. Among these 80 puzzles are all 15 puzzles highlighted during a previous benchmark by Anderson Lee et al.12 Notably, among these 15 puzzles are 7 puzzles yet unsolvable by any computational algorithm. This study demonstrates that teaching human design strategies to a computational RNA design agent in a data-driven manner can lead to significant increases in performance over previous methods, and represents a new paradigm in machine-based RNA design in which both human and computational design strategies are united into a single agent.  Code availability: The source code for SentRNA, all our trained models, and the full eternasolves dataset can be found on GitHub: https://github.com/jadeshi/SentRNA.  We performed all computations (training, validation, testing, and refinement) using a desktop computer with an Intel Core i7-6700K @ 4.00 GHz CPU and 16 GB of RAM. Creating 2D structural representations of puzzles: During training and testing of almost all models, we used the RNAplot function from ViennaRNA 1.8.5 to render puzzles as 2D structures given their dot-bracket representations. However, when training and testing two specific models M6 and M8 on two highly symmetric puzzles, “Mat Lot 2-2 B" and “Mutated chicken feet” (see Results and Discussion), we decided to use an in-house rendering algorithm (hereafter called EteRNA rendering) in place of RNAplot, as we found the RNAplot was unable to properly render the symmetric structure of the puzzles. Neural network architecture: Our goal is to create an RNA design agent that can propose a sequence of RNA bases that folds into a given target structure, i.e. fill in an initially blank EteRNA puzzle. To do this, we employ a fully connected neural network that assigns an identity of A, U, C, or G to each position in the puzzle given a featurized representation of its local environment. During test time, we expose the agent to every position in the puzzle sequentially and have it predict its identity. The neural network was implemented using TensorFlow14 and contains three hidden layers of 100 nodes with ReLU nonlinearitie