RIDER: 3D RNA Inverse Design with Reinforcement Learning-Guided Diffusion
Authors: Tianmeng Hu
Yongzheng Cui
Biao Luo
Ke Li
Venue: ICLR 2026 Poster (2026)
Keywords: RNA Inverse Design
Reinforcement Learning
RNA Structure
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: We propose an RL-guided diffusion model for RNA inverse design that directly optimizes 3D structural similarity, generalizes across predictors, and achieves over 100% gains over baselines while generating novel sequences.

Abstract: The inverse design of RNA three-dimensional (3D) structures is crucial for engineering functional RNAs in synthetic biology and therapeutics. While recent deep learning approaches have advanced this field, they are typically optimized and evaluated using native sequence recovery, which is a limited surrogate for structural fidelity, since different sequences can fold into similar 3D structures and high recovery does not necessarily indicate correct folding. To address this limitation, we propose RIDER, an RNA Inverse DEsign framework with Reinforcement learning that directly optimizes for 3D structural similarity. First, we develop and pre-train a GNN-based generative diffusion model conditioned on the target 3D structure, achieving a $9\\%$ improvement in native sequence recovery over state-of-the-art methods. Then, we fine-tune the model with an improved policy gradient algorithm using four task-specific reward functions based on 3D self-consistency metrics. Experimental results show that RIDER improves structural similarity by over $100\\%$ across all metrics and discovers designs that are distinct from native sequences.

Published as a conference paper at ICLR 2026
RIDER: 3D RNA INVERSE DESIGN WITH REIN- FORCEMENT LEARNING-GUIDED DIFFUSION
Tianmeng Hu University of Exeter t aiameng00228mmm comm
Yongzheng Cui
Central South University yongzhengcu1 @csu.edu.cm
Biao Luo
Ke Li
Central South University biac. www@huutmailllic com
University of Exeter k. k.li@exeter.. ac..e.raa.uuu.
ABSTRACT
The inverse design of RNA dire-dimensional (3D) structures is crucial for engi- neering functional RNAs in synthetic biology and therapeutics. While recent deep learning approaches have advanced this field, they are typically optimized and evaluated using native sequence recovery, which is a limited surrogate for struc- tural fidelity, since different sequences can fold into similar 3D structures and high recovery does not necessarily indicate correct folding. To address this limita- tion, propose RIDER, RNA Inverse DEsign framework with Reinforcement learning that directly optimizes for 3D structural similarity. First, we develop and pre-train a GNN-based generative diffusion model conditioned the target 3D structure, achieving a 9% improvement in native sequence recovery over state-of- the-art methods. Then, we fine-tune the model with an improved policy gradient algorithm using four task-specific reward functions based on 3D self- celf-consistency metrics. Experimental results show that RIDER improves structural similarity by over 100% across all metrics and discovers designs that are distinct from native sequences.
INTRODUCTION
Ribonucleic acid (RNA) is fundamental biomolecule with diverse biological functions, such as catalysis, gene regulation, and metabolite sensing (Serganov & Patel 2007). RNA function is closely linked its complex three-dimensioonal (3D) structure (Zhang al. 2022). The RNA inverse design problem (Hofacker et al. 1994b) finding a nuclcotide sequence that will fold into desired target
structure is therefore critical for designing RNAs with tailored functions for therapeutics and
synthetic biology Taft et al. 2010 Serganov & Nudler 2013
Dykstra et al.. 2022 Guo. 2010).
RNA structure is hierarchical, progressing from the primary nucleotide sequence to secondary struc- tures formed by local base pairing and culminating in the tertiary 3D structure, which involves long- range interactions, both canonical Watson-Crick and non-canonical pairs. While most computational
RNA inverse design target RNA secondary structure (Andronescu et al. 2004 Busch & Backofen,
2006 Garcia-Martin et al. 2013 2015 Kleinkauf et al. 2015 Eastman et al. 2018: Runge et al,
2019
Zhou et al. 2023). achieving precise biological function often requires control over the fi-
nal 3D structure, as secondary structure only partially dictates the tertiary fold (Huang et al. 2024) Consequently, recent research has focused tertiary seuettre-based inverse design. State-of-the-we art
methods employ deep learning, using graph neural networks (GNNs) (Joshi et al., 2025) Wong et al.) 2024 Tan et al. (2024) or generative diffusion models (Huang et al. 2024). These approaches are
spired by successes in protein design (Dauparas et al. 2022). demonstrate significant improvements
speed and performance over traditional physically- based method (Leman et al. 2020). However,
"Corresponding author

---

Published as a conference paper at ICLR 2026
20
(a)
(h)
(g)
(7)
Figure 1: Visualization of sequences (a), (b), and (c), and their corresponding 3D structures (a), (3), and (7) predicted by RhoFold (Shen et al. 2024). Although sequences (a) and (b) differ by only nucleotides, and (b) and (c) by nucleotides, their folded structures exhibit clear differences.
they are typically optimized and evaluated based on their ability to recover the native sequence for given structure, measured by native sequence recovery (NSR) (Joshi et al. 2025).
Optimizing solely for NSR problematic for RNA design. Unlike proteins, the relationship between RNA sequence and structure is highly degenerate: multiple distinct sequences can fold into similar structures (Assmann et al., 2023). Furthermore, similar sequences do not necessarily result similar structures, as illustrated in Figure I The true objective of RNA inverse design is structural fidelity finding any sequence that folds correctly - not necessarily recovering the single native sequence observed in the training data. Relying NSR proxy objective has limitations: 1) It does not directly optimize for structural similarity; improving sequence similarity to the native sequence secondary structure design methods avoid NSR, instead optimizing structural distance (Kleinkauf et al. 2015 Eastman al., [0188] Runge al. 2019 Zhou et al. 2023). 2) key problem in RNA biology is to identity RNA sequences with structural similarity to natural sequences (Morandi et al. 2022). Over- optimizing for NSR restricts exploration to sequences near the native one, potentially preventing the discovery of novel, non-native sequences.
does not ensure the designed structure matches the target (Liu al. 2025). Notably,
To overcome these limitations, we propose RIDER, a novel approach that directly optimizes struc- tural similarity using reinforcement learning (RL), integrating the strengths of generative models with the targeted optimization capabilities of RL. We first pre-train a generative diffusion model on RNA structure datasets, optimizing for NSR to enable the model to learn 3D structural represen- tations and capture grquence-structure relationships. Subsequently, we fine-tune this model using RI., employing reward function based the structural similarity between the predicted structure folded from the generated sequence and the target structure. This RI. phase allows exploration the vast sequence space, potentially uncovering novel sequences far from the native one in the dataset. Conceptually, this aligns with recent successes in using RL to fine-tune large generative models for specific objectives (Black al. 2024).
Our main contributions are:
We propose the first reinforcement learning framework for RNA 3D inverse design, which directly optimizes structural similarity rather than relying on surrogate sequence recovery objectives.
We develop and pre-train RIDE, a generative diffusion model for RNA inverse design con- ditioned on the target structural information, achieving an NSR of 61% and surpassing the strongest baseline at 56%.
We improve existing RL algorithms for fine-tuning diffusion models and apply them to the pre-trained RIDE, resulting in the complete RIDER framework. Specifically, we adopt a batch-mean baseline for advantage estimation and further incorporate moving average strategy to stabilize training. We design four novel reward functions based on 3D structural self-consistency metrics to guide RL fine-tuning.
Through experiments, we demonstrate that RIDER achieves over a 100% improvement over existing state-of-the-ar methods across all three 3D self-consistency metrics.
2

---

Published as a conference paper at ICLR 2026
2 RELATED WORK
RNA inverse design. Computational RNA inverse design seeks sequences that fold into a desired
target structure. Most existing methods primarily focus on secondary structure. These approaches employ RNA secondary folding prediction tools (Lorenz et al.) 2011 Markham & Zuker] [2008] Ali al., 2023) to compute the structural distance between the designed and target structures, and then search for sequences that meet the design objectives. Methods in this category include local
search (Hofacker al. 1994a/ Andronescu al. 2004] [uucch & Backofen 2006), metaheuristic al- gorithms (aaneaa 201 2 Iismalii--aaa Taheri et al., 2014] Esmaili-Taheri & Ganjtabesh, 2015: Kleinkauf
ict al.
2015; Zhou et al. 2023), and reinforcement learning (Eastman et al., 2018 Runge et al,
2019
Recent efforts have shifted toward 3D structure design. Physics-basee approaches such as
Rosetta (Leman et al. 2020) support fixed-backbone design, but are computationally expensive.
More recent methods incorporate deep learning: RiboDiffusion (Huang et al. 2024) employs gener-
ative diffusion models with Transformer components. RDesign (Tan et al. 2024), RhoDesign (Wong
2025 leverage GNNs extract 3D structural representations and train generative models to recover sequences from structural in- puts, achieving high sequence recovery. A common thread among these state-of-the-art methods is their reliance on supervised learning maximizing sequence recovery, which we argue is an indirect and potentially suboptimal proxy for structural fidelity. Our work addresses this by introducing an RL framework for direct structural optimization.
et al. 2024), R3Design (Tan al., 2025). and gRNAde (Joshi
RNA structure prediction. Since experimentally determining RNA structures is -abor-intensive and expensive, computational prediction methods have become indispensable (Piao et al. 2017).
Early approaches such as RNAComposer (Popenda et al.) 2012) assembled structures from librries of known fragments. More recently, deep learning has become dominant, leveraging statistical pat- terns learned from existing RNA structures. Representative examples include trRosettaRNA (Wang) et al. 2023). DRFold (Li et al. 2023), and RhoFold (Shen et al. 2024). Although RNA structure prediction is essential, progress in RNA functionality and drug discovery ultimately depends on methods for designing novel sequences. Nonetheless, continued improvements in prediction meth- ods are expected to further enhance the effectiveness of our design framework.
Reinforcement learning for generative models. Reinforcement Learning provides framework for optimizing policies toward specific goals defined by reward functions and has been widely ap- plied to solve dynamic decision-making problems. RL has achieved success in range of highly challenging tasks, surpassing human expert performance in competitive games (Vinyals et al. 2019), the game of Go (Silver et al. 2017). and autonomous drone racing (Kaufmann al.) 2023). Re- cently, RL has been employed to fine-tune large language models to align their outputs with human intent (Ouyang et al., 2022, Bai al. 2022), or to enhance reasoning abilities for solving complex mathematical and programming tasks (Guo et al.] 22025; Wang et al.] (024). In addition, recent works apply RL to fine-tune pretrained generative models, aligning them with human preferences or optimizing them for specific objectives (Fan et al. 2023 Zhang et al. 2024 Black et al. 2022).. For
example, DDPO (Black et al.] 2024) designs reward functions for rext-to-image generation, enabling diffusion models to adapt toward targets such as image compressibility or aesthetic quality.
METHOD
Our approach for 3D RNA inverse design is formulated as a conditional generative process based on a diffusion model. Given target RNA 3D backbone structure, the model learns to generate com- patible RNA sequence. The method comprises two main stages: first, a geometric graph representa- tion of the RNA backbone is extracted and processed by structure encoder to obtain conditional embedding; second, a diffusion model, conditioned on this structural embedding, iteratively refines noisy sequence representation produce the final RNA sequence sequence. Building upon this, we further introduce a reinforcement learning fine- tuning stage to directly optimize the structural similarity of the generated sequences. The pseudocode of the algorithms is provided in Appendix A
3

---

Published as a conference paper at ICLR 2026
Notsy oeqque ennoe
model
Reward luneson
lairning
Strucuurl GVP-ENN
Strucural impresenttioo
p(c
Scaler Seatures
struduue
Q(SSPa))
Sequenoe
structure
Figure 2: Overview of the RIDER framework. RNA tertiary structures processed by GVP- GNN encoder to produce structural embeddings. These emheddings condition the diffusion model for sequence generation, which further optimized by RL to maximize structural similarity.
3.1 TERTIARY STRUCTURE REPRESENTATION
To capture the geometric properties of RNA tertiary structures, we represent each molecule as geometric graph, where nodes correspond nucleotides and edges encode spatial proximity. A structural encoder based on GVP-GNN (Jing et al. 2021) processes this graph to produce node- level embeddings that are equivariant to 3D rotations and translations. These embeddings compactly summarize the local and global geometry of the RNA backbone and serve conditioning context for the diffusion model. Details the structural encoder are provided Appendix B.I Figure illustrates the overall pipeline.
3.2 CONDITIONAL DIFFUSION MODEL FOR RNA SEQUENCE GENERATION
We formulate RNA sequence design as learning a conditional distribution p(xg he). where xo denotes the one-hot encoded sequence (N nucleotides, C = 4 bases, Xo E {) 11-...x and h, is the structural embedding from the encoder. To model this distribution, we adopt variance- pariance-preserving diffusion probabilistic model (Ho et al.] [2020] Song et al.) (022b)), which learns to predict the noise added to clean sequence. The forward diffusion process gradually adds Gaussian noise to the clean data xo over continuous time variable & tme T'. where ét is small positive constant and the final time, The noised sample x, at time is given by:
X 0x80 orE,
where 8 -V(0)), and Otz, are schedule functions derived from predefined noise schedule Sr Specifically, af = exp fo 3,ds) and of = 11-00 aza- The noise level for a given t is often represented as A, = log /
(x (x,,,, he) with parameters This network is trained to predict the noise e from the noisy input x,, conditioned on the time and the structural context h The network ég is composed of L. 5 GVP-GNN layers. The input to this noise prediction network for each node combines its noisy sequence representation (x:);, an embedding of the time step teab. and its structural embedding (h.):. This combined representation is then processed by the network to predict the noise component (é); for each nucleotide. The model trained by minimizing the mean squared error between the true noise e and the predicted noise Eg;
The core of our generative model is noise prediction network e
Spretrain (0) Et,x,...h [||e c toe,, h.))
During inference, generate sequences by reversing the diffusion process. Starting from ran- dom Gaussian noise sample Xy A (0. I), we iteratively denoise it using the Denoising Diffusion Implicit Models (DDIM) (Sngg et al., aa sampler. After N Nateps iterations, obtain esti- mate of the clean sequence Xg. The final discrete RNA sequence is obtained by applying an argmax operation over the C channels for each nucleotide in Xo-

---

Published as a conference paper at ICLR 2026
3.3 REINFORCEMENT LEARNING FOR STRUCTURAL SIMILARITY OPTIMIZATION
Although the pre-trained diffusion model can learn sequence-struutture relationships, its training ob- jective does not directly maximize the structural similarity between the folded structure of the gen- erated sequence and the target structure. To overcome this limitation, we employ a reinforcement learning-base method to fine-tune the pre-trained diffusion model, inspired by the denoising diffu- sion policy optimization (DDPO) framework (Black et al. 2024), to directly optimize for rewards related to downstream tasks.
We frame the denoising sampling process of the diffusion model as Markov decision process (MDP). this MDP:
State sy: Defined as (x,, t, he), representing the current noisy sequence, time step, and structural condition.
Action og: Defined as the transition from x, to X in one denoising step. In our context, this corresponds to the noise (or equivalently, the predicted ko) predicted by the model co given 8t, which determines the mean of the next state Xr-Ar-
Policy To (a st): Parameterized by the diffusion model Eg.
Reward R ( h ): Obtained only at the end of the trajectory (i.e., after generating the complete &n).
Ad Advantage estimation We employ a policy gradient method fine-tune the diffusion model. The core idea is to update the model parameters 0 using the complete denoising trajectories {xT. xr- ..., } and their final rewards Reraj = R( barget. The original DDPO applies an importance sampling (IS) based estimator:
Nasera
To To(ux(5a)
VOJURA = -Er-prrieedd
Vo log To(ak 18k )
(3)
T0 (ax (sx)
where ag represents the action transitioning from state Sy (i.e., (Xt, 4. he)) to Xt, at time step k, To(ag) (&)) is the probability of this action under the current policy, and Te... is the old policy used for sampling. While this approach is effective in image generation, we found that directly using the reward Reray in our RNA design context leads to high variance and instability during optimization. Therefore, we replace the reward function with an advantage function A(xo. h ) to reduce the variance of gradient estimates. Specifically, for each target structure, we collect batch of experience trajectories by rumning the denoising sampling process multiple times. Each trajectory includes a sequence of latent states x,, the log-probability log No..a (arlst) for each step, and the final structural similarity reward Reraj. We then compute the average reward over this batch as baseline b:
b= -= (Rera))
(4)
However, we observe in our experiments that using baseline based solely on the batch average can lead to instability. Due to the stochasticity of the sampling process, baseline values computed from adjacent batches may vary significantly, resulting in large fluctuations in advantage estimates and hindering stable learning. Increasing the batch size may mitigate this issue but at the cost of reduced training efficiency. To address this, we adopt moving average strategy for baseline estimation. Let ((()
3(0 denote the moving average baseline after the i-th batch of experience collection, and let Ratch be the average reward of the i-th batch. The baseline is updated according to:
(+)) feaseline (6--1) (1 (psaccins) R(())
where ((0) initialized as the average reward of the first batch, and fruseline a smoothing factor that controls the update rate. The smoothed baseline b 660) is then used in advantage computation f The advantage function is then defined as A(x,, htarget) Reeaj b. and the policy gradient estimator based on advantage is correspondingly modified to:
Natopa
no ma(ax (sa) Vo log To (ax/ss)) A A(xo: heaeget)
VoJKL. Eywpprtbseaa)
(6)
To (ax (k)
Appendix
D.I investigates the impact of advantage function estimation.
5

---

Published as a conference paper at ICLR 2026
During the sampling process, to ensure diversity in each batch of samples and effectively explore the policy space, we perform one deterministic sampling (temperature parameter set to 0) and multiple stochastic samplings. In the stochastic sampling phase. uniformly sample a temperature from a discrete set (e.g.. (0.1, 0.3, 0.5, 0.7, 0.9} with a probability 1/5 for each) to control the ran- domness of the generation process. Furthermore,
adopt the clipping technique (Schulman al., 2017) to constrain the policy updates. We apply the importance sampling ratio rx.(0) Seloy Rg. TM (ex from Equation 66) to the advantage function and clip the result, forming the following optimization objective:
Nature
CRL Ersp(r|res))
cinirx())),.iii((((())) Celip, 1 (c1s))
(7)
where A is the advantage estimation of the entire trajectory, and <ciip is the clipping parameter.
Reward function The reward function is designed to quantify the structural similarity between the predicted 3D cotueture-obtainng by folding the generated RNA sequence-and the target 3D structure. We use the computational structure prediction model RhoFold (Shen et al., (024) to predict the folded structure of the generated sequence. We employ three widely used structural similarity metrics:
GDT TS (Zemla et al. 1999) (Global Distance Test Total Score): measures the percentage of backbone atoms within preset distance thresholds (1,, 2Á, 4A, 8Å) of their counterparts after optimal superposition. It emphasizes well-aligned regions and ranges from 0 to 1, with higher scores indicating better structural alignment.
RMSD (Kabsch 1976) (Root Mean Square Deviation): computes the average displacement between corresponding backbone atoms after superposition. It is sensitive to outliers; lower values indicate higher similarity.
TM-score (Zhang & Skolnick 2004) (Template Modeling Score): evaluates global fold similarity using a length-none length-normalize function. It is tolerant to local deviations and suitable for comparing structures of different lengths. Values range from to 1, with higher scores reflecting better agreement.
A detailed explanation of these metrics is provided in Appendix C Based these metries, explore various reward functions. First, we define base reward functions using individual metrics to analyze the model's ability to optimize for specific structural features:
Rgdt (GDT_TS. x Wgdt_ocalo)
(8)
R (TM-score Wom.scale)
(9)
Remsd (RMSD Wsmsd_ccale)",
(10)
where WydtLacales Win.scaler and Wimsd.scale are scaling factors. Considering that RMSD and GDT_TS reflect structural similarity from different perspectives, we also design combined reward function to balance the pursuit of precise atomic matching and overall fold similarity:
Redt.amed
(RMSD x Womod (GDT.TS x Wgdt
(11)
It is considered that when GDT-TS > 0.5 RMSD < 2.0Å, the predicted structure has significant similarity to the target structure, meeting an acceptable design standard (Tan et al. 2024 Joshi et al. 2025). To encourage the model to generate sequences with high structural fidelity. design an additional reward component Reonus for sequences that meet these criteria, computed as:
(GDT TS Tgot)) ) Whonus_gat, if GDT TS > Tgdt
Reanus
(Trmad RMSD) Wponus.rme else if RMSD < Trmad 0,
(12)
otherwise
where Tyat = 0.5 and Temsd = 2.0 are threshold values, and Wponus.gat and Wponss.cme are reward scaling coefficients. The final total reward function Rrinel used in our experiments is defined the sum of a base reward and a bonus reward:
Reinal = Roase + Roonus:
(13)
where Rease corresponds one of the four base reward functions defined above.

---

Published as a conference paper at ICLR 2026
1.0
0.8
20
0.8-
GDT_13
0.8
GDT 7.
0.8
081
-20
18
0.55
0.4
0.4-
10
0.5
0.5
1.0
gRNAdo
Hiboiftusion
RIDE
Native
TMM-szore
Figure 3: Results supervised learning pre-training. A. Comparison native sequence recovery on the test set. RIDE compared against RiboDiffusion and gRNAde. The best NSR among 16 designs per target (sampled at temperature 0.1) is reported. B. Relationship between NSR and structural similarity (GDT_T T and RMSD) for RIDE designs. Color denotes RMSD. C. Correlation among GDT.TS, TM-score, and RMSD for the designed structures.
EXPERIMENTS
4.1 SETTINGS
Dataset We use the RNA tertiary structure dataset published by Joshi et al. (2025) for pre-training and evaluating our model. The dataset is derived from the RNASolo repository (Adamczyk et al,) 2022) and contains 4, 223 unique RNA sequences and total of 12, 011 RNA structures. It is pre- partitioned based on structural similarity into training set, a validation set, and a test set, with the latter two each containing 100 samples.
Hyperparameters Both the structure encoder and the noise prediction network the diffusion model use layers of GVP-GNNs. The initial learning rate is to 3 x 10-4 and is decayed by factor of 0.9 if the validation performance does not improve for consecutive epochs. The model trained for total of 150 epochs. The overall number of parameters in our model is 10.2 million. All GVP-GNN layers apply a dropout rate of 0.5 to mitigate overfitting. To improve robustness, Gaussian noise with standard deviation of 0.1 added to the node coordinates during training. For sequence generation, we use the DDIM sampler (Song et al., 2021a) with Nateps 50 denoising steps. After pre-training, we fine-tune the diffusion model using policy gradient algorithm. The learning rate is set to 5 x 10-5 without learning rate scheduling. Training is conducted for 80 reinforcement learning epochs, with policy updates performed each epoch. The batch size is 60, meaning that 60 experience trajectories (each corresponding to full denoising process) are sampled per epoch. Appendix BeernndxxxBproviddd Plprovides detailed information on the model architecture and training hyperparameters.
4.2 SUPERVISED LEARNING PRE-TRAINING
We compare our proposed conditional diffusion model RIDE with two recent state-of-the-arr (SOTA) methods: RiboDiffusion (Huang et al. 2024) and gRNAde (Joshi et al. 2025). RiboDiffu- sion is Transformer-basee diffusion model, whereas gRNAde is a GNN-based generative model that employs an autoregressive decoder to generate RNA sequences.
RIDE achieves the best sequence recovery We evaluate the three methods the test set by comparing their NSR. For each target structure, each method performs 16 sampling runs at a tem- perature of 0.1 generate 16 candidate sequences, and the best NSR among them recorded. The results are shown in Figure A. RIDE achieves an average sequence recovery of 61%, representing improvements of 9% and 11% over RiboDiffusion and gRNAde, respectively. These results demon- strate that, even without reinforcement learning fine-tuning, our method outperforms existing SOTA baselines in terms of sequence recovery.
Native sequence recovery does not reflect structural similarity Using the same methodology as in Figure BJ, we employ RIDE to perform inverse design on the 100 structures in the test set. For each structure, record the best native sequence recovery and its corresponding 3DD structural

---

Published as a conference paper at ICLR 2026
A 1.0
gRiNAde RIDE RIDER
0.8
0.8
NSR (RIDER)
0.6
109
0.4
0.4
0.2
0.2
0.6 NSR (RIDE)
0.4
0.8
2
RNA 10
11
12
Figure 4: Results of reinforcement learning fine-tuning, A. GDT-TS comparison on 14 RNA struc- tures of interest (Das et al. 2010 for gRNAde, RIDE (pre-trained), and RIDER (fine-tuned with Redd.reed). B. Comparison of native sequence recovery before (RIDE) and after (RIDER) RL fine- tuning. Color indicates GDT.TS after RL fine-tuning. The results for the other two metrics are provided in Appendix D.4
GDT-TS 0.822
GDT-TS 0.776
CDT 0.3
TM-score = 0.825
TM-score 0.797
TM-score = 0.411
RMSD = 1.444
RMSD 1.902
RMSD 2.819
PDB: 3GAO
PDB: 2R8S
PDB: 1ET4
Figure 5: Visualization of designed examples. Structures folded from sequences generated by RIDER are shown in color, while the target structures are shown in semi-transparent yellow.
similarity metrics, as shown in Figure 3B. The results reveal no clear correlation between sequence recovery and 3D structural similarity. For example, when sequence recovery around 50%, the as- sociated GDT -TT values vary widely from 0 to 0.9. As sequence recovery increases, the distribution of GDT.TS does not consistently shift toward higher values. In fact, even at high levels sequence recovery, the resulting structures may still exhibit low similarity to the target structure. These ob- servations suggest that native sequence recovery alone is insufficient to assess the structural fidelity of designed sequences. This further highlights that optimizing sequence recovery not ideal surrogate objective for 3D RNA inverse design.
Relationship among three self-consistency metrics Figure illustrates the relationships among GDT TS, TM-score, and RMSD. GDT TS and TM-score show strong correlation, with Pearson correlation coefficient of 0.885. A few outliers highlight differences between these metrics. For instance, some samples exhibit low RMSD values (indicated by darker colors), high GDT.TS, but unexpectedly low TM-scores. This discrepancy arises because TM-score is more sensitive to de- viations in shorter structures. Since these three metrics capture structural similarity from different perspectives, design distinct reward functions based each of them during the RL fine-tuning phase to investigate their respective impact on performance.
4.3 REINFORCEMENT LEARNING FINE-TUNING
After pre-training, further fine-tune RIDE using reinforcement learning, denoted as RIDER.
8

---

Published as a conference paper at ICLR 2026
Table 1: Performance comparison of different methods and reward functions on the test set. Results are reported as the mean and standard deviation over 5 independent runs. Percentages in parentheses indicate the proportion of designs that meet the structural designability thresholds (GDT_TS 2 0.5, TM-score 2 0.45, RMSD < 2A).
3D self-consistency metrics
Method
Reward function
NSR
GDT TS
RMSD 1
TM-score 1
R
0.62 0.04(72%)
4.31 0.70(31%)
0.61 0.03(72%)
0.45 + 0.04
Rode
0.58 (63%)
4.60 0.55(30%)
0.56 0.05(64%)
0.47 + 0.06
BIDER
0.56 = 0. (.03(60%)
4.09 1: 0.40(31%)
0.55 1 0.06(64%)
0,49 1 0.05
Rotturned
0.62 + 0.02 (72%)
3.35 = 0.44 (33%)
0.56 + 0.03(68%)
0.47 = 0.05
RIDE
0.33 = 0.03(31%)
10.36 0.19(8%)
0.33 + 0.03(36%)
0.61 + 0.02
RIDE (Best-of-N)
0.42 = 0.06(46%)
8.63 0.90(15%)
0.39 1 (.0((45%)
0.50 1 0.06
RiboDiffusion
0.13 (302(
15.68 0.26(19)
0.14 0.03(6%)
0.56 + 0.04
RNAde
0.28 0.06(27%)
10.89 0.61
0.30 + 0.03(28%)
0.55 + 0.03
Table 2: Cross-predictor validation using the independent AlphaFold3 oracle.
Model
Reward function
GDT.TS
RMSD
TM-score f
RIDER (AlphaFold3)
R
0.56 i (607)
4.69 0.79 (30%)
0.56 0.06(65%)
RIDER (AlphaFold3)
Ridh
0.57 + 0.06 (65%)
3.80 + 0.61 (32%)
0.56 (67%)
gRNAde (AlphaFold3)
0.26 + 0.04(26%)
9.96 + 0.77(3%)
0.28 + 0.04(26%)
RIDER achieves substantial improvements in 3D structural similarity Figure shows results on 14 RNA structures of interest identified by Das et al. (2010), all which are included in the test set. RIDER delivers significant improvements on the vast majority of these structures, with more than 100% improvement on half of them. Table I summarizes the complete results across the test set. Across all three 3D self-consistency metrics, RIDER achieves more than 100% improvement relative to the pest-performing baseline. Using GDT TS as the evaluation criterion, 72% of RIDER designs exceed the threshold of 0.5, compared to only 27% for gRNAde. In terms of RMSD, 33% of RIDER sequences achieve < 2 compared to only 3% for gRNAde. Figure illustrates three representative examples of designed structures.
RIDER can discover high-quality designs different from native sequences Figure 4B shows the change in NSR before and after RL fine-tuning. The color of each point denotes the GDT.TS value after fine-tuning. We observe that, in most cases, the NSR of RIDER designs is lower than that of RIDE, yet these sequences achieve higher GS.
RIDER generalizes across different oracles To further evaluate the generalization capability of RIDER models, we design variant using AlphaFold3 (Abramson et al. (2024) to replace the orig- inal RhoFold. From the results in Table 2 we find that RIDER maintains its superiority under this cross-oracle evaluation. In particular, there minor performance differences when using different oracles: with the Agat_rmsd reward, our GDT TS score 0.57 with AlphaFold3, which is more than doubled (+119%) the of the baseline (0.26). These results demonstrate that RIDER captures generalizable principles of RNA design rather than overfitting a single predictor.
Impact of reward functions Table reports the impact of the four reward functions introduced in Section 3.3 The TM-score-based reward 8RT performs strongly on both TM-score and GDT.TS, though it is slightly less effective on RMSD. In contrast, Rrrsd and p yield weaker results when used individually. Notably, the composite reward Rast read which combines GDT.TS and RMSD, achieves the most balanced performance overall. We attribute this the complementary nature of the two metrics, which jointly capture both local atomic accuracy and global fold similarity. Appendix D.2 provides additional evaluation results for all four reward functions.

---

Published as a conference paper at ICLR 2026
5 CONCLUSION
We propose a novel two-stage framework for 3D RNA inverse design that directly optimizes struc- tural similarity. In the first stage, we develop RIDE, a conditional diffusion model trained via su- pervised learning that surpasses SOTA methods in native sequence recovery. In the second stage, we develop RIDER, which significantly improves the similarity between designed and target struc- tures through reinforcement learning. Future work may extend this framework with multi nulti-objective rewards and explore experimental validation of the generated sequences.
ACKNOWLEDGMENTS
This work was supported by the UKRI Future Leaders Fellowship under Grant MR/S0170622 and MR/X011135/1; in part by National Natural Science Foundation of China under Grant 62373375, U2341216, 62376056 and 62076056; part by the Science and Technology Innovation Program of Hunan Province under Grant 2024RC1011 and the Natural Science Foundation of Human Province under Grant 2025JJ 10007; in part by the Isambard-All Royal Society Faraday Discovery Fellowship 100//) BBSRC Transformative Research Technologies (UKRI1875) 875), Royal Society In- ternational Exchanges Award (IES/R3/243 136), Kan Tong Po Fellowship (KTP/R1/231017); and the Amazon Research Award and Alan Turing Fellowship.
REFERENCES
Josh Abramson, Jonas Adler, Jack Dunger, Richard Evans, Tim Green, Alexander Pritzel, Olaf Ronneberger, Lindsay Willmore, Andrew J Ballard, Joshua Bambrick, et al. Accurate structure prediction of biomolecular interactions with alphafold 3. Nature, -3060016) 2024.
Bartosz Adamczyk, Maciej Antczak, and Marta Szachniuk. Rnasolo: repository cleaned pdb-
derived ma 3d structures. Bioinformatics, (114)3666--3670) 2022.
Sara E. Ali, Abhinav Mittal, and David H. Mathews. RNA secondary structure analysis using RNAstructure. Curr. Protoc., 3(7):8846, 2023.
Mirela Andronescu, Anthony P. Fejes, Frank Hutter, Holger H. Hoos, and Anne Condon. A new algorithm for RNA secondary structure design. J. Mol. Biol., 336(3):607-24, 2004.
Sarah M Assmann, Hong-Li Chou, and Philip Bevilacqua. Rock, scissors, paper: How ma struc- ture informs function. The Plant Cell, 3576)1177 2023.
Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Constitutional ai: Harm- lessness from ai feedback. arXiv preprint arXiv:22 2022.
Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training diffusion models with reinforcement learning. In The Twelfth International Conference on Learning Rep- resentations, 2024.
Anke Busch and Rolf Backofen. INFO-RNA-a fast approach to inverse RNA folding. Bioinformat- ics, 22(15): 1823-31, 2006.
Rhiju Das, John Karanicolas, and David Baker. Atomic accuracy in predicting and designing non- canonical ma structure. Nature methods, 7(4):291 -294, 2010.
Justas Dauparas, Ivan Anishchenko, Nathaniel Bennett, Hua Bai, Robert J Ragotte, Lukas F Milles, Basile IM Wicky, Alexis Courbet, Rob de Haas, Neville Bethel, et al. Robust deep learning- based protein sequence design using proteinmpnn. Science, 378(6615):49-56 2022.
Peter B. Dykstra, Matias Kaplan, and Christina D. Smolke. Engineering synthetic RNA devices for cell control. Nat. Rev. Genet, 23:215-228, 2022.
Peter Eastman, Jade Shi, Bharath Ramsundar, and Vijay S Pande. Solving the RNA design problem with reinforcement learning. PLo8 computational biology, 14(6)ee1001176, 2018
10

---

Published as a conference paper at ICLR 2026
Ali Esmaili-Taheri and Mohammad Ganjtabesh. ERD: fast and reliable tool for RNA design including constraints. BMC Bioinformatics, 16(20), 2015.
Ali Esmaili-Taheri, Mohammad Ganjtabesh, and Morteza Mohammad-Noori. Multi-objective ge- netic algorithm for pscudoknotted RNA sequence design. Bioinformatics, 3009;1250-1125, 2014.
Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, Kangwook Lee, and Kimin Lee. Dpok: Reinforcement learning for fine-tuning tmxae diffusion models. Advances in Neural Information Processing Systems, 36:79858-79885, 2023.
Juan Antonio Garcia-Martin, Peter Clote, and Ivan Dotu. RNAiFOLD: constraint programming algorithm for RNA inverse folding and molecular design. J. Bioinform. Comput. Biol., 11(2), 2013.
Juan Antonio Garcia-Martin, Peter Clote, and Ivan Dotu. RNAiFold 2.0: web server and software to design custom and Rfam-based RNA molecules. Nucleic Acids Res., 43(W1): 2015.
Daya Guo. Dejian Yang. Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501. 129488 2025.
Peixuan Guo. The emerging field of RNA nanotcchnology. Nat. Nanotechnol, 5:833-842. 2010.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
I. L. Hofacker, W. Fontana, P. F. Stadler, L. S. Bonhoeffer, M. Tacker, and P. Schuster. Fast folding and comparison of RNA secondary structures. Monatsh. Chem., 125:167-188, 1994a.
Ivo L Hofacker, Walter Fontana, Peter F Stadler, L. Sebastian Bonhoeffer, Manfred Tacker, Peter Schuster, et al. Fast folding and comparison of RNA secondary structures. Monatshefte fur chemie, 125:167-167, 1994b.
Han Huang, Ziqian Lin, Dongchen He, Liang Hong, and Yu Li. Ribodiffusion: tertiary structure- based maa inverse folding with generative diffusion models. Bioinformatics, 40(Supplement I): 1347-i356, 2024.
Bowen Jing, Stephan Eismann, Patricia Suriana, Raphael John Lamarre Townshend, and Ron Dror. Learning from protein structure with geometric vector perceptrons. International Conference on Learning Representations, 2021.
Chaitanya K Joshi, Arian R Jamasb, Ramon Viñas, Charles Harris, Simon V Mathis, Alex Morehead, Rishabh Anand, and Pietro Liò. grnade: Geometric deep learning for 3d ma inverse design. In Proc. International Conference on Learning Representations, 2025.
Wolfgang Kabsch. A solution for the best rotation to relate two sets of vectors. Foundations of Crystallography, 32(5):922-92 9233, 1976.
Elia Kaufmann, Leonard Bauersfeld, Antonio Loquercio, Matthias Müller, Vladlen Koltun, and Davide Scaramuzza. Champion-level drone racing using deep reinforcement learning. Nature, 620(7976):982 987, 2023.
Robert Kleinkauf, Martin Mann, and Rolf Backofen. antaRNA: ant colony-based RNA sequence design. Bioinformatics, 31(19):31443121 2015.
Julia Kochler Leman, Brian D Weitzner, Steven Lewis, Jared Adolf- Bryfo-Bryfoole, Nawsad Alam, Re- becca F Alford, Melanie Aprahamian, David Baker, Kyle A Barlow, Patrick Barth, al. Macro- molecular modeling and design in rosetta: recent methods and frameworks. Nature methods, 17 (7):665-680, 2020.
Yang Li. Chengxin Zhang, Chenjie Feng, Robin Pearce, P Lydia Freddolino, and Yang Zhang. Inte- grating end-to-end learning with deep geometrical potentials for ab initio ma structure prediction. Nature Communications, 14(1):5745, 2023.

---

Published as a conference paper at ICLR 2026
Ke Liu, Shuaike Shen, and Hao Chen. From sentences to sequences: Rethinking languages in biological system. arXiv preprint arXiv:2507.00953 2025.
Ronny Lorenz, Stephan H Bernhart, Christian Höner zu Siederdissen, Hakim Tafer, Christoph Flamm, Peter F Stadler, and Ivo L Hofacker. ViennaRNA package 2.0. Algorithms for molecular biology, 6:1-14, 2011.
Nicholas R. Markham and Michael Zuker. UNAFold: software for nucleic acid folding and hy- bridization. Methods Mol Biol., 453:3-31, 2008.
Edoardo Morandi, Martijn J van Hemert, and Danny Incarnato. Shape-guided ma structure homol- ogy search and motif discovery. Nature Communications, 13(1):1722, 2022.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to fol- low instructions with human feedback. Advances in neural information processing systems, 35: 27730-27744, 2022.
Meiling Piao, Lei Sun, and Qiangfeng Cliff Zhang. Rna regulations and functions decoded by transcriptome-wine wide ma structure probing. Genomics, proteomics & bioinformatics, 15(5):267- 278, 2017.
Mariusz Popenda, Marta Szachniuk, Maciej Antezak, Katarzyna J Purzycka, Piotr Lukasiak, Natalia Bartol, Jacek Blazewicz, and Ryszard W Adamiak. Automated 3d structure composition for large mas. Nucleic acids research, 40(14)xe112<111, 2012.
Frederic Runge, Danny Stoll, Stefan Falkner, and Frank Hutter. Learning design RNA. In Pro- ceedings of the International Conference on Learning Representations, 2019.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.000 2017.
Alexander Serganov and Evgeny Nudler. A decade of riboswitches. Cell, 152:17-24, 2013.
Alexander Serganov and Dinshaw Patel. Ribozymes, riboswitches and beyond: regulation of gene expression without proteins. Nature Reviews Genetics, 8(10):776-790, 2007.
Tao Shen, Zhihang Hu, Siqi Sun, Liu, Felix Wong, Jiumning Wang, Jiayang Chen, Yixuan Wang, Liang Hong, Jin Xiao, et al. Accurate ma 3d structure prediction using language model-based deep learning approach. Nature Methods, pp. 1-12, 2024.
David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al. Mastering the game of go without human knowledge. nature, 550(7676):354-23 339, 2017.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusson implicit models. In Interna- fional Conference on Learning Representations, 2021a.
Yang Song, Jascha Sohl-Dickstein. Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In Interna- tional Conference on Learning Representations, 2021b.
Ryan J. Taft, Ken C. Pang, Timothy R. Mercer, Marcel Dinger, and John S. Mattick. Non-coding RNAs: regulators disease. J. Pathol., 220(2):126-33 39, 2010.
Cheng Tan, Yijie Zhang, Zhangyang Gao, Bozhen Hu, Siyuan Li, Zicheng Liu, and Stan Li. Rde- sign: Hierarchical data-efficient representation learning for tertiary mantture-based mn design. In
Proc. International Conference on Learning Representations, 2024.
Cheng Tan, Yijie Zhang, Zhangyang Gao, Hanqun Cao, Siyuan Li, Siqi Ma, Mathieu Blanchette, and Stan 7, Li. R3design: deep tertiary structure-based ma sequence design and beyond. Briefings in Bioinformatiis, 26() bbbbb6668 2025.
Akito Taneda. Multi-objecnie genetic algorithm for pseudoknotted RNA sequence design. Front. Genet., 26:3-36, 2012.
12

---

Published as a conference paper at ICLR 2026
Oriol Vinyals, Igor Babuschkin, Wojciech M Czamecki, Michaël Mathieu, Andrew Dudzik, Juny- oung Chung, David H Choi, Richard Powell, Timo Ewalds, Petko Georgiev, et al. Grandmaster level in starcraft ii using multi-agent reinforcement learning, nature, 575(7782):350-33555 2019.
Junqiao Wang, Zeng Zhang, Yangfan He, Yuyang Song, Tianyu Shi, Yuchen Li, Hengyuan Xu, Kunyu Wu, Guangwu Qian, Qiuwu Chen, et al. Enhancing code Ilms with reinforcement learning in code generation. arXiv preprint arXiv:2412.20366 2024.
Wenkai Wang, Chenjie Feng, Renmin Han, Ziyi Wang, Lisha Ye, Zongyang Du, Hong Wei, Fa Zhang, Zhenling Peng, and Jianyi Yang. trrosettarna: automated prediction of ra 3d structure with transformer network. Nature communications, 14(1):7266, 2023.
Felix Wong, Dongchen He, Aarti Krishnan, Liang Hong. Alexander Z. Wang. Jiuming Wang, Zhi- hang Hu, Satotaka Omori, Alicia Li, Jiahua Rao, et al. Deep generative design of ma aptamers using structural predictions, Nature Computational Science, pp. 2024.
Adam Zemla, Česlovas Venclovas, John Moult, and Krzysztof Fidelis. Processing and analysis of casp3 protein structure predictions. Proteins: Structure, Function, and Bioinformatics, 37(S3): 22 29, 1999.
Jinsong Zhang, Yuhan Fei, Lei Sun, and Qiangfeng Cliff Zhang. Advances and opportunities in RNA structure experimental determination and computational modeling. Nature methods, 19 (10):1193-127,,22 2022.
Yang Zhang and Jeffrey Skolnick. Scoring function for automated assessment of protein structure template quality. Proteins: Structure, Function, and Bioinformatics, 57(4):702-710, 2004.
Yinan Zhang, Eric Tzeng, Yilun Du, and Dmitry Kislyuk. Large-scale reinforcement learning for diffusion models. In European Conference on Computer Vision, pp. 1-17. Springer, 2024.
Tianshuo Zhou, Ning Dai, Sizhen Li, Max Ward, David H Mathews, and Liang Huang. RNA de- sign via structure-aware multifrontier ensemble optimization. Bioinformatics, 39(Supplement.)) 1563-i571, 2023.
13

---

Published as a conference paper at ICLR 2026
A ALGORITHM PSEUDOCODE
This section provides detailed pseudocode for the key algorithmic components of our proposed method. Specifically, we include three main procedures:
Algorithm describes the supervised pre-training stage of the conditional diffusion model (RIDE), where the model learns to predict noise added to clean RNA sequences given target 3D structure.
Algorithm 2 presents the DDIM- Maadd--aeddd-- sampling procedure for generating RNA sequences conditioned on a target structure using the pre-trained model. This forms the basis both inference and experience collection in the RI stage.
Algorithm (ooritthmoutlinee outlines the reinforcement learning fine-tuning stage (RIDER), where a policy gradient method is used optimize the diffusion model toward improved structural fidelity based on 3D evaluation metrics (GDT.TS, TM-score, RMSD).
Together, these algorithms define the full learning pipeline of our framework.
Algorithm RIDE; Pre-training of Conditional Diffusion Model
Require: Training dataset D {v0 xx number of diffusion steps Tarts model parameters
(t)
Require: Structure encoder Eggns GVP-GNN noise prediction network co
Ensure: Pre-trained model parameters
1: Initialize model parameters (for Eg)
2: for each training iteration do
3: Sample a minibatch (yo. Xurget) from
4: Compute structure condition C = gan (X
5: Sample timestep f ~ Uniform( (( (1,,..,, ".a....,Tann]
6: Sample standard Gaussian noise ~ NN 7)
7:
Generate noisy sequence Y/ = 0,30 + 0,5 using the forward process
8: Predict noise using the model: & eg EB C)
9:
Compute pre-training loss: Lepearain
E E
Compute gradients V V Lpperran
11: Update parameters: 0 + 0 V Leeerain
end for
13:
14: return Trained model parameters
Algorithm Sequence Sampling via DDIM
Require: Target structure Xaagets trained fine-tuned model parameters 0 (for Eg)
Require: Structure encoder Egan, GVP-GNN noise prediction network
Require: Total number of sampling steps Naeps
Ensure: Designed RNA sequence ýo Compute structure condition C Egnn (uugeet) (O,
2: Sample initial noise vector 1/Naa
3: for = Nsstps down do
4: Determine current and previous time steps ty and t
5: Predict noise: εN = 6g (3t,- te. C)
6: Compute yt,,,, using DDIM update rule with ye, and E
7: end for
8: Set ûo 3/
9: Obtain discrete sequence: discrree = arggax (fo)
11: return di,disceete
14

---

Published as a conference paper at ICLR 2026
Algorithm 3 RIDER: Reinforcement Learning Fine-Tuning
Require:
Pretrained policy network wa; target structure dataset D ingett structure encoder Egnn: RNA structure predictor P reward function I
Require: Clipping coefficient <lipi learning rate vi number of RL epochs N; number of trajectories per epoch Muaji number of policy update steps per epoch apddtt
Ensure: Fine-tuned policy parameters 0
1: for epoch = to do
2: Oold +
3: Initialize experience buffer B
4: for 1 to Mraj do
5: Sample target structure Xurget from D Diarget
6: Compute structure condition C Egan' (argett
7:
(ūo. trajectory data,,) + sampling X Xpred (Vo)
8:
R R R(peeed (argee)
9:
10: Store (trajectory datam Russ) in B
11: end for
12: Compute baseline
for 1 to Kupdate do
14:
for each (trajectory_dae _data, Reaj) in B do
Compute advantage: for each (sk. ak, log Ne.u (ax sk)) in trajectory do
Rusj
16:
Compute log my (*
18:
Compute importance ratio (0) exp(log exp(logTo(agles) log Tow (ax (sk)
Tunj Tuaaj min(rx clip(rx(()) <elip: + <clip) A)
20:
end for
21:
end for
22:
R +- Luy/ Muaj
23:
Compute gradient VIL
24:
Update parameters: +- 0 - 17 oLLRL.
25:
end for
26:
end for
28:
return Fine-tuned parameters 0
B MODEL ARCHITECTURE AND TRAINING HYPERPARAMETERS
This section provides detailed descriptions of the RIDE architecture, pre-training and fine-tuning hyperparameters, as well the computational setup used during training. Figure 6 illustrates the overall architecture and learning pipeline of our proposed method. The framework consists two stages: (1) RIDE, conditional diffusion model that is pre-trained to generate RNA sequences from target 3D structures, and (2) RIDER, a reinforcement learning fine-tuning stage that optimizes the diffusion model for structural fidelity using 3D self-consistency metrics (GDT_TS TM-score, RMSD). The model takes as input 3D structure embedding and iteratively denoises a sequence ini- tialized from Gaussian noise, guided by multi-layer GVP-GNN encoder-decoder architecture. The predicted sequences are evaluated via structure prediction model, and reward signals are computed fine-tune the policy.
B.1 STRUCTURE REPRESENTATION
To effectively capture the geometric intricacies of RNA tertiary structures, first convert the input backbone into a graph representation, which is then processed by Geometric Vector Perceptron Graph Neural Network (GVP-GNN) (Jing et al. 2021 encoder.
Featurization We represent each RNA tertiary structure as a geometric graph G (V,E), where V denotes nucleotides nodes and & encodes spatial adjacencies. Each nucleotide characterized
15

---

Published as a conference paper at ICLR 2026
First Stage: RIDE
Second Stage: RIDER
Reinforcesent Learning
Rwward Function
Diffueion
Predieted
model
Struc
3D sel felff-aasstenn net metracs
RMA Structure Prediction Model
Nossed Seq
Nossed Seq
Recevered Beq
GBT 18
Denoising
Noised Sequence
Diffusion model
Recovered Sequence
DDACKJJJAAUCOE
BUGBAJA
DeP
DAMUUNCUCCOG
BOAGOBUMMUUSE
DBACUNUSU0D
OVP-ONN Encodor Layer
Structure Embodding
Figure 6: The proposed framework consists of two stages: RIDE (pre-training via supervised learning) and RIDER (fine-tuning via reinforcement learning). A multi-layer GVP-GNN encoder- decoder backbone is used iteratively denoise RNA sequences conditioned on the target 3D struc- ture. Structural fidelity is evaluated using separate RNA structure predictor and fed back through reward function.
by the coordinates of three backbone atoms: P, C4', and NI (for pyrimidines) N9 (for purines). The node coordinate H; E R$ is defined as the centroid of these atoms. To capture local geometry, edges connect each node its k 32 nearest spatial neighbors.
Node features n consist scalar features 8; E R and vector features ō E Rd x3, Scalar features include intra-nuce nucleotie distances and angles, while vector features capture backbone ori- entation, such as the unit vector Σ,+1 - Z; and vectors from C4' to and N1/N9. Edge features (0) for edge between node and j similarly consist of scalar components Sij E d (e.g., 3D distance ||ž øil12 encoded via radial basis functions, and sequence separation j encoded via sinusoidal positional encodings) and vector components Egg E Rd x3 (e.g., the unit vector ž -
Structure encoder The featurized graph is processed by structure encoder composed of LE GVP-GNN layers. Each layer / updates the node representations (s!") of tion from neighboring nodes and edges an ((3))equivarian) manner. The update rule for node at layer can be expressed as:
(()) by aggregating informa-
m mm
(1)
AGG (ol), (1s( 2 ofll o(l-1)),, (f jEN(E)
(1-1)
(6 (s,,,
(14)
8 ( Pupdate
(off
(1-1) all 1) (m, m!!))
(15)
where (ii denotes the neighbors of node i, Paeg and @ functions respectively, and AGG an aggregation operator (e.g.. mean). The encoder network first embeds the initial node and edge features using GVP layers W,, and Wa:
(1)
Pupdate GVP-based message and update
0) W. W.((st". ")"),, ( 2 W. We((st).
(16)
The final output of the structure encoder set of node-level conditional embeddings he ((s("=) ; s(L=))} for all E V. These embeddings, h.s encapsulate the essential 3D structural information and serve the conditioning context for the diffusion model.
16

---

Published as a conference paper at ICLR 2026
B.2 RIDE: CONDITIONAL DIFFUSION MODEL ARCHITECTURE
The RIDE model is conditional diffusion network built upon Geometric Vector Perceptron Graph Neural Networks (GVP-GNNs), specifically designed for RNA inverse folding.
GVP-GNN configuration. Both the structure encoder and the noise prediction network within the diffusion model comprise LE LD = 5 layers of GVP-GNNs. The key architectural parameters and node/edge feature dimensions to each GVP-GNN layer to mitigate overfitting. To enhance robustness, Gaussian noise with a standard deviation of 0.1 is added to node coordinates during training. The model incorporates time-step embeddings as additional conditioning inputs to guide the denoising process. In total, RIDE contains 10, 214, 213 trainable parameters.
summarized in Table B] A dropout rate of 0.5 is applied
Table 3: RIDE architecture and associated hyperparameters.
Hyperparameter
Value
Number of GVP-GNN layers (LE, LD)
Node input dimensions (scalar, vector)
(15, 4)
Node hidden dimensions (scalar, vector)
(256, 24)
Edge input dimensions (scalar, vector)
(131, 3)
Edge hidden dimensions (scalar, vector)
(128, 4)
Dropout rate
0.5
Output dimension
Number of nearest neighbors per node
32
Number of radial basis functions (RBFs)
32
Positional encoding dimension for edges
Diffusion model pre-training. RIDE is pre-trained via noise prediction using a Mean Squared (VP) Stochastic Dif-
Error (MSE) loss. The forward noising process follows ferential Equation (SDE). Key hyperparameters for the SDE and training schedule are summarized in Table 4 The initial learning rate is set to 3 x 10-4 and is decayed by factor of 0.9 using ReduceLROnF lateau scheduler when the validation performance (measured by NSR) fails to improve for 5 consecutive epochs.
B.3 DETAILS OF RIDER
Following pre-training, the RIDE model is fine-tuned using an advantage- based policy gradient algorithm, referred to as RIDER.
RL Algorithm Settings. Reinforcement learning fine-tuning is performed using the Adam op- timizer without any learning rate scheduling. Key hyperparameters for the training loop, policy updates, and experience collection are summarized in Table
Reward Function Scaling Factors. The reward functions used during RL fine-tuning based three structural similarity metrics. The scaling coefficients for these components, as defined in summarized in 6
Section 3.3
B.4 TRAINING HARDWARE
All model pre-training and reinforcement learning fine-tuning experiments were conducted on servers equipped with NVIDIA RTX 4090 GPUs and Intel Core 19-13900K CPUs. With automatic mixed precision (AMP) enabled, training requires approximately 16 GB of GPU memory; without AMP, the memory usage increases to around 23 GB. RL fine-tuning requires approximately hours on single RTX 4090 GPU, while the critical inference cost remains unchanged and highly efficient.
17

---

Published as a conference paper at ICLR 2026
Table 4: Training hyperparameters for RIDE pre-training.
Hyperparameter
Value
SDE schedule
Linear
Initial noise scale Po
0.1
Final noise scale 31
20.0
Total diffusion time T Minimum time step imee
1.0
0.001
Number of epochs Initial learning rate Learning rate scheduler Scheduler decay factor Scheduler patience Optimizer
150
3 10-4
ReduceLROnPlateau
0.9
5 epochs
Adam
Activation function
SiLU
Max nodes per batch
3000
Max nodes per sample
5000
DDIM denoising steps Nateps
50
Sampling temperature
0.1
Table 5: Training hyperparameters for RIDER fine-tuning.
Hyperparameter
Value
Learning rate
5 10
Optimizer
Adam
Number RL training epochs
80
Policy updates per epoch
2
Gradient accumulation steps
60
Clip range felip
0.5
Max gradient norm
1.0
Batch size
60
DDIM denoising steps during RL
30
Table 6: Scaling factors for reward function components.
Component
Scaling Factor
GDT.TS weight (Wist.
TM-score weight (Wem.soala)
RMSD weight (Wemsd_scaie)
0.5
GDT TS weight for bonus reward (Wbonus.gat)
100
RMSD weight for bonus reward (Woonus smsa)
20
C 3D STRUCTURE SELF-CONSISTENCY METRICS
To evaluate how accurately a designed RNA sequence folds back into its intended 3D backbone structure, adopt three complementary global metrics: Root Mean Square Deviation (RMSD),
18

---

Published as a conference paper at ICLR 2026
GDT_TS TS
0.4
0.2
RIDER
RIDER-ADV
RIDER-RWD
Figure 7: Comparison of RIDER, RIDER-RWD, and RIDER-ADE on the GDT TS metric across the test set. Bar heights represent the mean, and red error bars indicate the Standard Error of the Mean (SEM).
50
40
RMSD
30
20
12.820
10
3.368
www.
RIDER
RIDER-ADV
RIDER-RWD
Figure 8: Comparison of RIDER, RIDER RWD, and RIDER ADV on the RMSD metric across the test set. Bar heights represent the mean, and red error bars indicate the SEM.
Global Distance Test Total Score (GDT.TS), and Template Modeling Score (TM-score). These metries capture distinct aspects of structural similarity and are defined as follows.
C.1 ROOT- MEAN-MAAANAAA DEVIATION (RMSD)
RMSD quantifies the average atomic displacement between two structures after optimal superposi- tion. Given two sets N equivalent atoms with coordinates {x,} (predicted) and {y:} (reference), RMSD is defined as:
N
RMSD
1
X; 18--vll
yi
(17)
N
Lower RMSD values indicate closer structural alignment. High-accuracy models typically achieve RMSD < 2A, while poorly modeled regions may exceed 10Å.
19

---

Published as a conference paper at ICLR 2026
TM-score
0.4
0.2
RIDER
RIDER-ADV
RIDER-RWD
Figure 9: Comparison of RIDER, RIDER-RWD and RIDER LIDER-ADV on the TM-score metric across the test set. Bar heights represent the mean, and red error bars indicate the SEM.
RIDER-AWD RIDER-AD
RIDER
0.8
GDT TS
0.6
0.4
0.2
5
6
7
8
9
10
11
12
13
14
Figure 10: GDT.TS scores of RIDER, RIDER-RWD, and RIDER-ADY 14 target RNA structures.
C.2 GLOBAL DISTANCE TEST TOTAL SCORE (GDT_TS
GDT-TS quantifies the fraction of residues that can be superimposed within multiple distance cut- offs. For a reference structure of length N, let N4. denote the number Co atom pairs within a distance threshold de E {1,2, 2, 4, 8)A, computed under independently optimized superpositions for each dz. GDT TS is then defined as:
1
GDT TS
No. x 100%.
(18)
4N
kE
By focusing only on well-aligned residues (d, < 8A), GDT.TS is robust to local distortions and emphasizes the proportion of accurately modeled structure.
C.3 TEMPLATE MODELING SCORE (TM-SCORE)
TM-score evaluates global fold similarity using a single optimized superposition with a length- dependent normalization term. Given LN residues in the reference structure and Lr aligned residue pairs with pairwise distances d,, the TM-score is defined as:
TM-score = max
(19)
LN
de
JolLN))
where the normalization scale dy is function of LN:
(TN) 1.24 V L,N 15 1.8.
(20)
20

---

Published as a conference paper at ICLR 2026
RIDER-RWI RIDER-ADY RIDER
25
RMSD
7
8
9
10
11
12
13
14
RNA
Figure 11: RMSD scores of RIDER, RIDER-RWD, and RIDER-ADV on 14 target RNA structures. Lower values indicate better structural alignment.
RIDER- RWD RIDER-ADY RIDER
0.8
0.6
TM MM-soore
0.4
5
7
8
9
10
11
12
13
14
RNA
Figure 12:
TM-score values of RIDER, RIDER-RWD, and RIDER- ADV on 14 target RNA struc- tures. Higher values indicate better structural similarity.
This length correction makes TM-score more stringent for short structures (smaller dg) and more permissive for longer ones, enabling fair comparison across proteins RNAs of varying lengths.
C.4 COMPARISON OF METRICS
RMSD provides direct measure of average atomic deviation and is highly sensitive outliers. GDT-TS quantifies the percentage of residues that fall within multiple distance thresholds, empha- sizing well-modeled regions. TM-score yields length- norgmlized global similarity score, which is less sensitive to local deviations and more suitable for comparing structures of different sizes. To- gether, these three metrics offer comprehensive evaluation framework for assessing 3D structural fidelity in RNA inverse design.
D ABLATION STUDY
We conduct ablation studies on the advantage estimation strategy, reward design, and key hyperpa- rameters used in both RIDER and RIDE.
D.1 ADVANTAGE ESTIMATION STRATEGY
In RIDER, we replace raw rewards with advantage estimates and use the batch mean baseline for advantage calculation. Additionally, we introduce a moving average strategy to stabilize the baseline and reduce variance during training. In this section, compare the full RIDER with two variants: (1) RIDER-RWD, which directly uses raw rewards without advantage estimation; and (2) RIDER-ADV, which uses batch-based advantage but without the moving average baseline.
21

---

Published as a conference paper at ICLR 2026
0.8
GDT_TS TS
0.4
Rgat-rmed
Rggtt
Armed
Ram
Figure 13: Comparison of four reward functions (/t=m, R Rast, Rgdt on the GDT.TS
metric across the test set. Bar heights represent the mean, and red error bars indicate the SEM.
20
15
RMSD
10
5-
Risht rmsd
Right
Rymsd
R
Figure 14: Comparison of four reward funetions on the RMSD metric across the test set. Bar heights represent the mean, and red error bars indicate the SEM.
Figures 7 and show the performance of the three methods on the test across three 3D self-consistency metrics. RIDER consistently achieves the best performance across all metrics. In contrast, RIDER-RAiN performs poorly on many samples-for for example, exhibiting numerous points with GDT TSS scores below 0.1 or RMSD values exceeding 10A. This instability is likely due to the lack of advantage estimation, which causes high variance and sometimes leads to training collapse in later stages.
Compared RIDER-ADV, our full RIDER achieves around 15% improvement GDT-TS, demonstrating the effécctiveness of the moving average strategy in stabilizing learning and enhancing structural quality.
Figures 10 and 12 further present the evaluation results of RIDER, RIDER- RWD, and -IDER-ADV on the 14 RNA structures of interest Das et al. (2010). On the GDT.TS metric, RIDER achieves the best performance across 12 out of the 14 samples, and it consistently outperforms the other variants across all three evaluation metrics. For the remaining two samples, RIDER performs slightly worse than RIDER-ADY ADV but still significantly outperforms RIDER-RWD.
22

---

Published as a conference paper at ICLR 2026
0.8
TM-score
0.6
0.2
Rgdt-rmed
Rgott
Armed
Ram
Figure 15:
Comparison of four reward functions on the TM-score metric across the test set. Bar heights represent the mean, and red error bars indicate the SEM.
D.2 REWARD FUNCTIONS
Figures B 14 and 15 present the evaluation results of the four reward functions we designed, measured across all three structural similarity metrics on the test set.
Among them, the reward function based solely TM-score, RT performs well both TM-score and GDT TS, but slightly worse on RMSD. The variants using only Reaad or Rggt: perform slightly worse than Rt= overall.
We hypothesize that the inferior performance of Rudt may be due to the fact that GDT.TS only considers well-modeled residues (i.e., those within 8Á), and ignores errors in poorly modeled re- gions. This may cause the RL optimization to become trapped in local optima. On the other hand, while RMSD accounts for deviations across all residues, it does not apply length normalization like TM-score, making it potentially unsuitable for comparing RNAs of varying lengths. Moreover, in the course of exploration, some structures may yield very large RMSD values, leading to abrupt changes in reward, which can destabilize the training process.
In contrast, the composite reward Rudt..mmdd which combines both GDT and RMSD, demon- strates strong performance across all metrics. We attribute this to the complementary nature of the two metrics-balanced local accuracy (GDT_TS) with global deviation sensitivity (RMSD).
D.3 HYPERPARAMETER SENSITIVITY RIDE
We also perform hyperparameter sensitivity analysis for the RIDE model. summarizee the native sequence recovery performance under different configurations of optimizer, activation func- tion, initial learning rate, and the number of GVP-GNN layers (Lg,LD), The results are averaged over five independent runs with different random seeds.
The results show that the setting used in our main experiments-AAda optimizer, SiLU activation, initial learning rate of 3 10-4, and 5 GVP-GNN layers-achiever achieves the best sequence recovery (0.61 0.02). Deeper networks (8 layers) tend to underperform. possibly due to overfitting or op- timization instability. We also observe that SiLU consistently outperforms ReLU, and that Adamk performs slightly worse than Adam under most configurations.
D.4 ADDITIONAL. RESULTS
Figure in Section 4 presents the GDT.TS comparison 14 RNA structures of interest Das et al. 2010 for gRNAde, R RIDE (pre-trained), and RIDER (fine-tuned with rmad). Here, we provide the corresponding supplementary results on the other two metries: RMSD and TM-score.
Figure 16 the RMSD comparison among the three methods, while Figure 17 presents the TM- score comparison. These results further demonstrate that RIDER substantially outperforms both
23

---

Published as a conference paper at ICLR 2026
Table Ablation study of RIDE hyperparameters on native sequence recovery. Results are reported as the mean and standard deviation over five independent runs.
GVP-GNN layers (L LD)
Optimizer
Activation function
Initial LR
Native sequence recovery
0.61 + 0.02
3 x 10-4
0.38 + 0.04
SiLU
0.60 + 0.02
x 10-4
Adam
0.32 1 0.08
5
0.58 +: 0.04
3 x 10-4
0.59 0.03
ReLU
0.57 + 0.03
x 10-4
0.51 0.04
0.59 + 0.03
x 10-4
0.59 0.04
SiLU
0.47 1 0.10
X 10
AdanW
0.50 0.08
0.56 0.03
10
0.53 0.09
ReLU
0.56 0.04
10m4
0.50 0.10
25
RIDER
RMSD
5
0
2
10
11
14
RNA
Figure 16: RMSD comparison of gRNAde, RIDE, and RIDER on 14 RNA structures of interest.
Lower RMSD indicates better structural alignment.
gRNAde and the pre-trained RIDE on the RMSD and TM-score metrics. In most cases, RIDER achieves improvements exceeding 100%, highlighting the effectiveness of our proposed method.
DISCUSSION
E. FUTURE WORK & LIMITATIONS
The proposed RIDER framework is flexible and can be extended in several directions. Future work may explore the design of multi-objective reward functions that incorporate additional factors such as sequence diversity or functional properties. Another promising avenue is to integrate biophysical metrics, for example, energy based terms, into the reward function. Such integration could guide the model towards sequences with improved thermodynamic stability, which is critical requirement for practical applications.
24

---

Published as a conference paper at ICLR 2026
gRINAde RIDE RIDER
0.8
0.6
TM mt--sorre
2
7
10
11 12
13
14
RNA
Figure 17: TM-score comparison gRNAde, RIDE, and RIDER on 14 RNA structures of interest.
Higher TM-score reflects better global fold similarity.
Our current method focuses on in silico RNA inverse design. However, RNA molecules are dynamic and can adopt multiple conformations in vivo, further increasing the complexity of the inverse design task. As future work, we plan to experimentally validate our designed sequences and extend the framework support multiple, potentially competing, optimization objectives.
E.2 BROADER IMPACTS
The proposed method may accelerate progress in synthetic biology. RNA-based drug development, and the study of gene regulatory mechanisms, thereby contributing to novel therapeutic strategies. By enabling the design of RNAs with precise 3D structures, it opens opportunities for engineer- ing functional RNA circuits, aptamers, and ribozymes. However, applying Al models to biolog- ical sequence design introduces challenges related to interpretability and reliability. Enhancing the interpretability of such models-e, by identifying which structural features influence pre- dictions may help mitigate these concerns and promote responsible use in biological design.
25


Reviews:
[{"review_id": "ei46YxiwYV", "reviewer": "Reviewer_RHxN", "summary": "The paper tackles RNA sequence design conditioned on a target 3D structure. The main contribution is a RL methodology that optimizes for structural similarity metrics (as opposed to previous supervised learning methods). Results show that the RL training leads to significant improvements over the baseline gRNAde model.", "questions": "- A major question I had based on the manuscript: Can you show a case study or explain what may happens in the following situation: I wish to design a particular backbone 3D structure. I have its native sequence. The 3D structure predictor I happen to use simply cannot fold the native sequence into the target 3D structure (its a poor predictor). So what will happen if I use the proposed RL finetuning method? Won't the model sort of 'hack' the 3D structure predictor I am using and end up designing something that folds well according to the structure predictor, but may not do so in nature?\n\n- Additionally, and relatedly to the weakness I identified (+ my Q above), can the RL methodology be extended to other oracles? Can you show experiments with, say, secondary structure ensemble properties such as Mean Free Energy, or some other reasonable properties of interest for RNA design? I think that could strengthen the paper and also show the generality of RL finetuning a strong base supervised model.\n\nOther suggestions:\n\n- On contribution claim: \"We identify the limitations of native sequence recovery...\" - I don't think this should be presented as a claim. This is well known in the community.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The problem chosen is significant, as RNA inverse design methods are used practically and validated in wet labs recently. The paper identifies an important research gap and tackles the problem of RL-driven structural similarity optimization in RNA design well. To the best of my knowledge, this is one of the first works to tackle this important problem.\n\n- The paper is generally well written overall. The appendix/supplementary could be better organized.\n\n- I found the experiments and results convincing and supportive of the main claims of the paper around RL optimization. The result of the supervised trained model also outperforming gRNAde on the challenging single-state Das split is also a significant gain.", "weaknesses": "My main issue with this paper is how its trying to optimizer via RL directly for structural metrics using RNA 3D structure prediction that is not itself very reliable. This way, the model may optimizer for structure as predicted by the folding model, but the folding model may be very poor (as recent CASP contests for RNA show...) -- this means that the model may not really be doing what its meant to do. \n\nI realise that this is not the current aim of the paper. I would still have liked to see discussions on this limitation within the paper, as it can limit practical applicability.\n\nOther than this, I don't see any major issues with the methodology/from a technical/ML point of view.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zf7F3T5P4Q", "reviewer": "Reviewer_io7z", "summary": "The paper proposes a two-stage framework for RNA inverse folding. First, a graph neural diffusion model (DM-GNN) is pretrained to reconstruct native sequences from 3D RNA structures. Then, the model is fine-tuned via reinforcement learning (DM-RL) to directly optimize structural fidelity metrics (GDT-TS, TM-score, RMSD) predicted by a structure oracle. This approach substantially improves 3D self-consistency compared to prior RNA design methods, while maintaining sequence diversity and foldability.", "questions": "See weaknesses.", "limitations": "", "rating": 2, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper is technically sound and well executed. The writing is clear, structured, and easy to follow. The empirical results are strong, demonstrating consistent and substantial improvements over prior state-of-the-art RNA inverse-folding approaches. The overall methodology—a diffusion model pretrained on sequence recovery and fine-tuned via reinforcement learning for 3D structural fidelity—is reasonable and well motivated. The success of this framework aligns with prior evidence in the protein inverse-folding literature, where diffusion combined with RL optimization has been shown to yield similar performance gains.", "weaknesses": "While the paper is well executed, it lacks genuine methodological novelty. The proposed framework, diffusion-based sequence generation followed by reinforcement learning fine-tuning for structural fidelity, has already been explored extensively in the context of protein inverse folding. The current work largely ports that established recipe to RNA, with the main differences being the input domain (RNA rather than protein), the smaller token vocabulary (4 bases vs. 20 amino acids), and the use of RNA-specific structure predictors.\n\nThe use of RL to steer diffusion models is also not novel; similar formulations have been reported in both general diffusion literature and multiple protein design papers. As such, the contribution lies primarily in application rather than new ML methodology. Moreover, the paper underplays or omits discussion of this related body of work, giving the impression that concepts such as “native sequence recovery not reflecting structural fidelity” are new insights, whereas they are well established in the protein design literature.\n\nIn its current form, the paper presents a well-engineered application of known ideas to a new molecular domain, which would be more appropriate for a domain-specific or biological modeling venue rather than ICLR, which prioritizes algorithmic or conceptual advances in machine learning.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DLz7I6WKzB", "reviewer": "Reviewer_ebQg", "summary": "The authors propose to extend recent RNA inverse folding methods such as RiboDiffusion and gRNAde to generate non-native RNA sequences with near-native structural similarities. Overall, the method's performance is compelling, pointing to the importance in future work of using reinforcement learning to steer pretrained generative RNA models to more relevant regions of biochemical design space.", "questions": "1. Have the authors investigated how well their fine-tuned (DM-RL) checkpoints perform for multi-state RNA design, as investigated in previous works such as gRNAde? This would interesting to see (i.e., whether reinforcement learning can capture such multi-state RNA conformational information implicitly).", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The authors cleverly combine the best architectural aspects of previous RNA inverse folding methods such as gRNAde and RiboDiffusion.\n2. Structure-focused and reinforcement learning-based finetuning of RNA inverse folding methods does seem to contribute a substantial improvement to their ability to design structurally-native RNA sequences.\n3. The authors follow best practices in RNA inverse folding evaluation by adopting gRNAde's datasets and benchmarking protocols.\n4. The authors' ablation experiment showing that using AlphaFold 3 instead of RhoFold in their method does not impact performance much is encouraging to see.", "weaknesses": "1. Although the authors made their code available for review, it could be cleaned up and documented more carefully. For instance, including a descriptive README.md file with it would go a long way to helping users navigate the codebase.\n2. The authors claim they are the first to point out the limitations of using native sequence recovery as a primary metric for RNA inverse folding. However, this has already been well discussed within the context of an adjacent topic (i.e., protein inverse folding).\n3. Lines 119-122 seem a bit indirect in saying that RNA structure prediction is important, yet not the most important, but still important for RNA design. Although most structural metrics related to RNA design are ultimately dependent on the accuracy of RNA structure prediction algorithms, the authors' descriptions here are vague and unclear concerning which directions are important to pursue to improve RNA design in future work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)