BAnG: Bidirectional Anchored Generation for Conditional RNA Design
Authors: Roman Klypa
Alberto Bietti
Sergei Grudinin
Venue: ICML 2025 poster (2025)
Keywords: generative modeling
sequential data
autoregressive generation
RNA design
biological sequences
deep learning models
Primary area: deep_learning
TL;DR: BAnG is a  Bidirectional Anchored Generation method specifically tailored for conditioned RNA design

Abstract: Designing RNA molecules that interact with specific proteins is a critical challenge in experimental and computational biology. Existing computational approaches require a substantial amount of experimentally determined RNA sequences for each specific protein or a detailed knowledge of RNA structure, restricting their utility in practice. To address this limitation, we develop RNA-BAnG, a deep learning-based model designed to generate RNA sequences for protein interactions without these requirements. Central to our approach is a novel generative method, Bidirectional Anchored Generation (BAnG), which leverages the observation that protein-binding RNA sequences often contain functional binding motifs embedded within broader sequence contexts. We first validate our method on generic synthetic tasks involving similar localized motifs to those appearing in RNAs, demonstrating its benefits over existing generative approaches. We then evaluate our model on biological sequences, showing its effectiveness for conditional RNA sequence design given a binding protein.

BAnG: Bidirectional Anchored Generation for Conditional RNA Design
Roman Klypa' Alberto Bietti 2 Sergei Grudinin
Abstract
graham al., 2023). This remarkable progress has not only revolutionized protein design but also opened new opportu- nities for addressing other complex biomolecular challenges. One such area is RNA generation, where similar principles of leveraging deep learning can be applied to advance our understanding and design of functional RNA sequences.
Designing RNA molecules that interact with spe- cific proteins is a critical challenge in experimen- tal and computational biology. Existing compu- tational approaches require a substantial amount of previously known interacting RNA sequences for each specific protein or a detailed knowledge of RNA structure, restricting their utility in prac- tice. To address this limitation, we develop RNA- BAnG, a deep learning- bearning-based model designed to generate RNA sequences for protein interac- tions without these requirements. Central to our approach is a novel generative method, Bidirec- tional Anchored Generation (BAnG), which lever- ages the observation that protein-binding RNA sequences often contain functional binding mo- tifs embedded within broader sequence contexts. We first validate our method on generic synthetic tasks involving similar localized motifs to those appearing in RNAs, demonstrating its benefits over existing generative approaches. We then eval- uate our model on biological sequences, showing its effectiveness for conditional RNA sequence design given binding protein.
Among the many challenges in RNA design, generating RNA sequences capable of binding to specific proteins stands out as critical task with significant implications for understanding RNA-protein interactions (Li et al., 2024; Fasogbon et al., 2024). These interactions, central to essen- tial biological processes such as gene regulation, splicing, and translation (Hentze et al., 2018), highlight the need for precisely engineered RNA molecules. A notable example of such molecules are aptamers, short single-stranded RNA sequences that bind to specifie proteins with high affinity and specificity. By acting as molecular inhibitors, probes, or delivery agents, aptamers offer versatile applications in therapeutics and diagnostics (Guo et al., 2010; Thavarajah et al., 2021). Traditionally, aptamers are identified through SELEX (Systematic Evolution of Ligands by Exponential Enrichment), a labor-intensive experimental process. De- veloping computational methods to design aptamers could significantly accelerate and simplify their discovery, expand- ing their potential in biomedical applications.
Several studies have explored RNA generation in this do- main. More classical approaches exploited evolutionary signals and statistical models (Kim et al., 2007a;b; Aita & Husimi, 2010; Tseng et al., 201 1; Zhang et al., 2023), molec- ular modeling (Torkamanian-Afãah Carlo tree search (Lee et al., 2021; Wang et al., 2022; Shin et al., 2023; Obonyo et al., 2024). More recent works used conditional variation autoencoders (Chen et al., 2022; Iwano et al., 2022; Andress et al., 2023), long short-term memory models (Im et al., 2019; Park & Han, 2020), transformer- based architectures (Zhao al., 2024: Zhang et al., 2024), and adversarial approach (Ozden et al., 2023). Most recent studies, e.g., AptaDiff (Wang et al., 2024), or RNAFLOW (Nori & Jin, 2024) also explored diffusion processes and flow matching. Almost all of the aforementioned approaches depend on a vast collection of nucleotide sequences known to interact with proteins to generate new ones, limiting their applicability to proteins for which extensive experimental data is available. To the best of our knowledge, RNAFLOW
1. Introduction
Deep learning has significantly advanced bioinformaties and structural biology, particularly in predicting the struc- tures, interactions, and functions of biomolecules (Callaway, 2024). It has also improved the efficiency of macromolec- ular design, facilitating applications in drug discovery and synthetic biology. In particular, significant progress has been made in protein sequence design. Some illustrative ex- amples include ESM3 (Hayes al., 2024) and Chroma (In-
et al., 2021), and Monte
Univ. Grenoble Alpes, CNRS, Grenoble INP, LJK, 38000 Grenoble, France. Center for Computational Mathematics, Flat- iron Institute. 162 5th Ave, New York, NY 10010, USA. Cor- respondence to: Roman Klypa < <<ooman.Kee KlypaannKKlyaeaa @ univ-grenoble, alpes.fr>, Alberto Bietti <Alberto. S @ comil.com>, Sergei Grudinin Eeegeee.Grudiiiiii @ univ- sirtii-greoble-aae alpe..
Proceedings of the nnnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

BAnG; Bidirectional Anchored Generation for Conditional RNA Design
Target Protein
RNA Sequences
<anc)<<ancr>
<ancB<ancrPA
G<ancl<<AncPA
ACUGGUUGU
ACACOUACGGUAC
CAGUAA
GAGGCAGGUACU
GranndaaanepAcees>
UG-andbxaanerAccesp
RNA-BAnG
ACACCUACGGUAC
CAGUAAUUCUBGG GAGGCAAGUACU
<eas>UGGanne>xaeerrrkeeess
UGA
Figure 2. A step-by-step example process of RNA sequence 'UGA' generation.
Figure 1. Schematic illustration of the RNA-BAnG generative pro- cess and its conditioning on the input protein 3D structure. The protein model was generated by AlphaFold2. RNA sequences are colored by nucleotides.
perimentally. The resulting combination of the RNA-BAnG model and the generative method, schematically illustrated in Figure 1, produces RNA sequences that interact with a given protein, utilizing both its sequence and structural information. Our main contributions can be summarized as follows:
stands out as the only method that has not been trained on RNA affinity experimental data. However, it relies on RNA structure prediction tools to guide RNA design. Since these tools often lack the accuracy needed for precise RNA struc- tures (Rhiju et al., 2024), they ultimately reduce the model's effectiveness in many practical scenarios.
1. We propose a new bidirectional generation method, BAnG, along with a transformer-based architecture, RNA-BAnG, that are well-suited for RNA generation conditioned on a binding protein.
In this work, we present novel generative method com- bined with deep-learning model that operates without rely- ing on specific experimental data for the target protein and does not depend on RNA structural information. This ap- proach enables a broader applicability and greater efficiency in RNA sequence generation than the ones mentioned above.
We thoroughly validate the effectiveness of our method on relevant synthetic tasks and compare with other widely used sequence generation methods.
The motivation for the design of the proposed generative method stems from two key observations. First, the total length of the RNA sequence to be generated is often un- known. Therefore, the method adopts autoregressive approach. Second, RNA sequences that interact with pro- teins typically contain functional binding motifs specific regions that mediate interaction by forming molecular con- tacts with the protein. These binding motif's are embedded within larger sequence contexts, where the surrounding non- binding regions exert lesser influence on binding specificity (Ray et al., 2013). This makes it more effective to initiate sequence generation from the binding motif, rather than from the sequence's ' ends, as is commonly done in current state-of-the-art NI.P autoregressive models. These are the core ideas behind our method, Bidirectional ANchored Gen- eration (BAnG).
3. We evaluate our approach on experimental RNA- protein interaction data, showing promising results that outperform previous methods.
2. Bidirectional Anchored Generation for RNA
In this section, we present our generative modeling ap- proach, BAnG, and its application to conditional RNA se- quence prediction through the RNA-BAnG model.
2.1. Description of the BAnG Generative Approach
Our approach models conditional distributions of each token given previously generated ones, similar autoregressive modeling, but with a different factorization of the joint distribution over the sequence, which leads to different order of generation. In the BAnG framework, we leverage the following factorization of the joint distribution over a sequence x (z-mm To. .o.......00 W:n):
The model, named RNA-BAnC based on transformer architecture with geometric attention. The latter allows for the incorporation of protein structural information, which is crucial for predicting RNA-protein interactions. By utilizing AlphaFold2 (Jumper et al., 2021), a state-of-the-an protein structure prediction tool, we can obtain highly accurate structural data, even if the target protein is not solved ex-
P(x) P(a)) II P II P())))) (1)
left tokens
right tokens
2

---

BAnG; Bidirectional Anchored Generation for Conditional RNA Design
<eos>
<andl> <ancr>
<eos>
<eoS>
Token
Figure 4. Schematic illustration of the masked attention mecha- nism and token prohabilities derivation. Bold lines indicate the token embeddings from which the probabilities for each token are derived, while the dotted lines represent the tokens to which the given token's attention is directed.
<andl>
<ancr>
that any representation of given token cannot depend on tokens beyond those appearing in the corresponding condi- tional in Eq. 1. This simple modification allows the model to learn the dependencies needed for our generation strategy. while ensuring efficient parallelization of the forward and backward passes across all tokens in a sequence during train- ing. During model training and inference, the probabilities for the next token in each direction are derived from the embedding of the most recently generated token in the same direction, shown schematically in Figure 4.
-3
-2
-1
-1
0
Taken index
Figure BAnG atlention mask. Anchor tokens are indexed in way preserve relative distances in sequence after anchors insertion.
where CL. = (a-1+1..... r;) and C = (x-i+1,... I1-1) denote left and right contexts. This formulation enables generation to proceed outward from the token To, condi- tioning each new token on a growing window of previ- ously generated tokens. While the factorization cen- tered on Tu, still need some initial tokens to begin generation. To this end, we insert two special anchor tokens, <ancl> and <ancr>, immediately before oo! These to- kens provide a fixed starting context, making it possible to assign probability to Ta and begin outward generation. They remain in the contexts Ch and CØ for all subsequent tokens.
We shall emphasize that the single pass training with the conditional factorization Eq. is only possible thanks to the introduction of two anchor tokens. Indeed, if only single anchor token was used, it would be responsible for predicting the first tokens in both directions, right and left. To prevent information leakage, its attention would be restricted to itself, which would make the prediction of the left token independent of the right one. This lack of conditioning could lead to the generation of incompatible token pairs.
x (m-m...., <ancl>, <ancr>, Io, ...,....... Vn..
With this setup in place, generation proceeds token by to- ken, alternating between the right and left directions: we first sample token on the right, then a token on the left, and repeat this process to progressively extend the sequence outward. At each step, the distribution of the next token is conditioned already generated ones, following Eq. 1. If an end-of-sequence (<eos>) token is generated in either direction, no further tokens are produced along that axis. The generation process stops when <eos> tokens are pro- duced for both boundaries or when a predefined maximum sequence length is reached (see Figure 2).
2.2. Model
The RNA-BAnG architecture consists of two main com- ponents: a protein module and a nuclcotide module. The protein module derives a representation from the protein's 's sequence and structure, while the nucleotide module gener- ates nucleotide sequence conditioned this representa- tion. Our model's modules comprise several main blocks Embedder, Self Attention, Geometric Attention, and Cross Attention, schematically illustrated in Figure 5. The Embedder block generates token embeddings from protein, RNA, and DNA sequences. The inclusion RNA and DNA sequences in the training data serves to augment the dataset, as can help the model learn the shared patterns. Protein sequences tokenized using the 20 standard amino acids and a padding token. RNA sequences are tokenized using the four canonical nucleotides, along with padding and end- of sequeeee tokens. DNA sequences are tokenized with the same set, treating DNA residues as their RNA equivalents. To allow the model distinguish between RNA and DNA,
BAnG enables training deep learning model to estimate the conditional distributions in Eq. order to perform bidirec- tional generation in the described manner, using a process analogous to autoregressive training. The key difference in the architecture compared to the standard autoregressive case is the replacement of the conventional lower triangular attention mask with a specifically designed bidirectional at- tention mask, shown Figure This custom mask ensures
3

---

BAnG; Bidirectional Anchored Generation for Conditional RNA Design
tion function in the Feed Forward layers. For normalization, we employed RMSNorm (Zhang & Sennrich, 2019). The number of attention heads is consistent across all attention blocks, and the dimension of each head is set independently of other model parameters. In both self-attention and cross- attention mechanisms, keys and queries are normalized prior to multiplication to prevent the attention-logit growth in- stability (Dehghani et al., 2023). The specific choice of hyperparameters is described in Appendix A.
Protein embedder
Nucleotide embedder
Nacelotia Module, ispeated times
Protein Module, repeated N times
Self Attention
Self Attention
Geometric Attention
Cross Attention
3. Validating BAnG on a Synthetic Task
Feedforward
Feedforward
In this section, we evaluate the effectiveness of our BAnG strategy through a thorough analysis on a relevant synthetic task involving distinct subsequences.
Prediction head
Figure 5. Schematic illustration of the RNA-BAnG architecture. The protein component is on the left, nucleotide component is on the right. More details can be found in Appendix A.
Description of the task. To evaluate the BAnG method, we first consider synthetic task that emulates a real-world conditional generation scenario. As mentioned above, RNA sequences often include functional binding motif within broader sequence context, where non-binding regions con- tribute minimally to binding specificity. The binding mo- tifs are typically short and linear, with some being highly conserved and others exhibiting probabilistic patterns (Ray et al., 2013). Based on this observation, the task's 's objec- tive is set generate sequences that contain a predefined short subsequence, synthetic motif. Although these sim- plifications do not fully capture the nuances of real-world scenarios, they offer baseline for understanding the method before applying it to more challenging tasks.
additional sequence type information is encoded separately (more details in Appendix A).
Self-attention in the model was implemented in a classical way (Vaswani et al., 2023) using Rotary Position Embed- ding (RoPE) (Su et al., 2023). While RoPE has recently been predominantly used in autoregressive models, it was originally introduced for bidirectional transformers, making it particularly suitable for our case. Self-attention for protein sequences uses unmasked attention, while for nucleotide sequences, the BAnG mask is applied.
Cross-attention is implemented similarly to the self- attention mechanism, with one key difference: instead of using RoPE, sinusoidal positional embeddings (Vaswani et al., 2023) are applied to the nucleotide sequence. Specifi- cally, we encoded the positions of nucleotide tokens relative to the anchor ones, where the anchor token <ancl> is as- signed an index of zero. This choice was made because residues in different chains do not have relative sequential distances, but we still want to include nucleotide position information in the calculation of its attention to the protein.
Synthetic data. The synthetic data consists of nucleotide sequences, each 50 residues long, with a synthetic motif placed at random position. The remaining residues are uniformly distributed. We fixed two different random sub- sequences of length 6 as the synthetic motifs. The exact content of these motifs is not crucial to the task, as any possible subsequences of this length equally likely be uniformly sampled. The length of six was chosen because it closely matches the size of real binding motif' (Ray et al., 2013) and reduces the likelihood of random occurrences of such subsequences. As the anchor point for BAnG, we chose the center of the synthetic motif.
Geometric Attention aims to incorporate protein structural information. To achieve this, we adopted the protein repre- sentation introduced AlphaFold2 (Jumper et al., 2021), 1). Specifically, for each protein residue, rigid frame T is constructed based the coordinates of its C, Ca, and N backbone atoms. Attention between these frames is then calculated using the geometric part from AlphaFold2's In- variant Point Attention (IPA) mechanism (more details in Appendix A).
In the Single training setup, sequences contained only the first synthetic motif, while in the DoubleBind setup, sequences could contain either the first the second syn- thetic motif with equal probability. The objective of the SingleBind setup was to compare generative methods on a simpler task, while the goal of DoubleBind was to assess their performance under more realistic, thus more uncer- tain conditions.
We use GELU (Hendrycks& Gimpel, 2016) as the activa-
4

---

BAnG; Bidirectional Anchored Generation for Conditional RNA Design
Table Comparison of generative methods on the SingleBind and DoubleBind tasks. Values represent the proportion of sequences that contain correct synthetic motif.
Table 2. Detailed statistics for the DoubleRind task: proportion of sequences containing either one of the synthetic motif's or both.
GENERATIVE METHOD
FIRST 1
SECOND t
BOTH
BANG
0.43
0.53
0.01
GENERATIVE МETHOD
SINGLEBIND f
DOUBLEBIND f
AUTOREGRESSIVE
0.17
0.25
0.11
BANG
0.98
0.97
IANG LOGIT MAX
0.09
0.44
0.01
AUTOREGRESSIVE
0.94
0.53
IANG ENTROPY
0.10
0.44
0.01
IANG ENTROPY
0,91
0,54
ITERATIVE ENTROPY
0.03
0.02
0.01
IANG LOGIT MAX
0.89
0.54
ITERATIVE LOGIT MAX
0.01
0.04
0.01
ITERATIVE ENTROPY
0.06
0.04
RANDOM SEQUENCES
0.01
0.01
0
ITERATIVE LOGIT MAX
0.04
0.05
RANDOM SEQUENCES
0.01
0.02
can be found Appendix B.3.
Reference methods. We compare our method to exist- ing generative approaches, including autoregressive genera- tion, which is commonly used in natural language process- ing (Bengio et al., 2000), and iterative generation methods based masked language modeling, as implemented in ESM3 (Hayes et al., 2024). Also, as the simplest base- line, we include in the comparison set of random sequences, where each token sampled from a uniform distribution. For the autoregressive approach, we trained the model using lower triangular attention mask. In the iterative approach, the model trained with demasking objective and high masking rate of 50%, which, according to the ESM3 authors, has been shown to yield effective generation results. Additional model training details for each tested method can be found in Appendix B.I.
The tables show that BAnG outperforms other methods, with a particularly notable margin on the DoubleBind task. BAnG also generates fewer sequences containing mixed syn- thetic motifs. We have also conducted experiments demon- strating BAnG's robustness to both larger fixed sequence lengths and varying sequence lengths, showing consistent performance across different configurations (Appendix B.4).
The very low performance of ESM3's iterative methods is expected, as the lack of absolute positional dependencies in the data causes the model to assign nearly uniform proba- bilities across tokens. This reasoning is further supported by the significant performance improvement observed when positional information introduced through the anchor to- ken in LAnG. Nonetheless, both methods suffer from key limitation: discrepancy between the demasking process during training and inference, which undermines their over- all effectiveness.
For the autoregressive approach, tokens are generated se- quentially, one at time, starting from the start-of-sequence (<sos>) token and continuing until the <eos> token is produced. In the iterative approach, however, all tokens but <sos> and <eos> are initially masked. Tokens are then unmasked one by one, with the next token to unmask chosen based on either the largest logit value (max logit decoding) or the smallest entropy (entropy decoding). Sampling details may be found in Appendix B.2.
Iterative methods may demonstrate improved performance in modality translation scenarios, such as structure-to- sequence generation, which is key focus ESM3. How- ever, they are unlikely match the effectiveness of regres- sive approaches in purely generative tasks.
4. Conditional RNA Generation
We also introduced modification of iterative methods, better suited for the task Iterative Anchored Generation (IAnG). This approach merges BAnG with iterative methods by incorporating an anchor token (<anc>) placed in the middle of the synthetic motif. In this method, the anchor token remains unmasked during training. At the start of the inference, the anchor token is positioned at random loca- tion within the sequence and remains unmasked throughout the process.
In this section, we evaluate our RNA-BAnG modeling strat- egy for protein-conditioned generation of RNA sequences based on experimental biological data.
4.1. Data
We collected our -ecletieee interaction data from the Protein Data Bank (PDB) (Berman, 2000), utilizing in- formation provided in the PPI3D database (Dapkūnas et al., 2024). However, we conducted independent postprocessing of the data, distinct from PPI3D, as they focus on structural RNA and DNA information, while we are concerned solely with their sequences. The postprocessing steps involved verifying chain interactions, discarding ambiguous protein
Evaluation results. With each tested approach, we gener- ated 1,000 sequences. Table I summarizes the performance of each method. Additional statistics on the frequency of each synthetic motif in the generated data for DoudleBind task are listed in Table 2. Examples of generated sequences
5

---

BAnG; Bidirectional Anchored Generation for Conditional RNA Design
structures, and excluding chains containing non-standard residues, as explained in more detail in Appendix C.1. Dur- ing training and validation, each time sample was encoun- tered, its anchor point was randomly sampled from the in- teracting mucleotides. The anchor tokens were then inserted
1.0
Proportion Above Threshold
0.8
0.6
right after the selected nucleotide. Additionally, to diver- sify RNA sequence information, we collected non-coding sequences from RNAcentral (release 24), a comprehensive database integrating RNA sequences from multiple expert sources (The RNAcentral Consortium et al., 2019) (more details in Appendix C.2). Since interaction data for these sequences is unavailable, their anchor points were selected randomly.
0.4
0.2
0.0
0.0
0.2
0.6 Threshold Value
0.4
0.8
1.0
Figure 6. Proportion of sequences above the threshold: generated by RNA-BAnG (green) and randomly (yellow). the positive (blue) and negative (red) experimental sets, The values here represent the averages for the entire test set.
4.2. RNA-BANA Training
We designed the training process to consist of two steps: the first for the model to learn general information about RNA sequences, and the second for the model to learn con- ditioning on proteins. In both steps, the training objective is the cross-entropy loss between the predicted and ground- truth nucleotide token probabilities (Bengio et al., 2000). As the first step we train RNA-BAnG nucleotide module without cross-attention block on standalone RNA sequence data from RNAccenrrl.. During training, the loss values for the four tokens closest to each anchor on either side are weighted at 0.01. This weighting scheme is applied because the model lacks the context needed accurately predict these residues. Next, we train the full model, using weights from the previous step, on the combined protein- nucleotide sequence data, this time without applying any loss weighting. Additional training details can be found in Appendix D.
Test set. We built our test set using data from RNAcom- pete experiments (Ray et al., 2009), conducted by the au- thors of the RNA Compendium (Ray et al., 2013). Each sam- ple consists of protein sequence paired with set of RNA sequences, each assigned score reflecting its binding affin- ity to the protein. We trained separate DeepCLIP model for each sample and excluded those where the scoring model did not meet our performance criteria (see Appendix E.
Due to the lack of experimentally solved protein structures for the test set and in pursuit of more robust method, we used only AlphaFold2 protein models for RNA-BAnC in- ference during evaluation. For each sample in our test set, we generated the corresponding three-dimensional protein model and retained only those with a predicted local dis- tance difference test (pLDDT) score greater than 70%. This threshold, widely accepted in the community, ensures the structural reliability of the produced protein models, at least at the domain level. The final test set included 71 samples, representing 67 unique protein sequences.
4.3. Evaluation Protocol
Evaluating generative models is difficult because direct com- parison with ground truth is not possible. To the best of our knowledge, there are no established computational protocols for measuring RNA affinity to proteins. Therefore, we use predictive model as main evaluation tool, following common practice in machine learning, such as FID scores in image generation or refoldability metrics in protein design.
To evaluate the generative models in a relative way, we built positive and negative sets of RNA sequences for each sam- ple. The positive set includes sequences with the highest experimental affinity scores, while the negative set includes those with the lowest. We excluded these sets from Deep- CLIP training and used them as an independent benchmark to assess both DeepCLIP predictions and the quality of the generated RNA sequences (see Appendix E. for details). In addition to generating sequences with the evaluated model, we also created random sequences to serve as a baseline (see Appendix E.2). Each mentioned set contains 1,000 sequences, unless stated otherwise.
Scoring model. As the scoring model, we adopted Deep- CLIP (Grønning et al., 2020), following the approach of the authors of GenerRNA (Zhao et al., 2024) and others (Im et al., 2019). DeepCLIP is Mate-of-theean predictive model that, after being trained on examples of interacting and non-interacting RNA sequences for a given protein, can assign binding probabilities with the same protein to any RNA sequence. These probabilities serve as a proxy for eval- uating the quality of sequences generated by RNA-BAnt and other methods.
4.4. Experiment Results
To estimate RNA-BAnG's performance, we use the pro- portion of sequences with DeepCLIP scores above a given
6

---

BAnG; Bidirectional Anchored Generation for Conditional RNA Design
Proportion of haRNA
1.0
0.8
0.6
0.4
0.2
0.0
138
1
500
0
0
284
28
000
00
060
100
260
100
000
205
075
036
093
077
Sample ID
Figure 7. Proportions of high-afiinity RNA sequences for each test sample, generated by RNA-BAnG (green) and randomly (yellow).
These are compared with the proportions in the positive (bluc) and negative (red) experimental sets. Test samples are ordered by RNA-BAnG performance. performacc Sample IDs mapping to RNAcompete IDs mentioned in Appendix E.
threshold, rather than relying the mean score for each test sample. This approach aligns better with practical ap- plications, where the goal is often to maximize the number of sequences that meet specific performance criterion, rather than optimizing for average performance across the entire set.
the test samples, more than half of the sequences generated by RNA-BAnG are the high-affinity ones. Additionally, model generated less than 20% high-affinity sequences for only 14 out of 71 test samples. The high success rate of randomly generated sequences for some samples may be attributed to the low complexity captured protein binding motifs (Ray et al., 2013).
Statisties of generated sequences. Figure 6 presents the hhreshold-dependett performance curve, showing the rela- tionship between sequences with DeepCLIP scores above certain threshold and the threshold value. The area under the threshold-dependee performance curves in Figure 6 cap- tures the method's ability to generate affine sequences. A larger area indicates better performance, reflecting a higher proportion of sequences that exceed the threshold at dif- ferent points. A high area value of 0.88 for the positive experimental set indicates that DeepCLIP is good distin- guishing high-affinity sequences from others. RNA-BAnC showed the value of 0.57, indicating moderate success in generating sequences that somewhat align with the positive set. The random set, with the area value of 0.39, suggests that our model is outperforming random generation. The negative set's very low area value of 0.11 further under- scores the DeepCLIP good performance, also suggesting that ENA-BAnG has some ability avoid producing low- affinity outputs.
The performance of our model shows significant corre- lation with the predicted quality of the protein 3D model (pl DDDT) or with the sequence similarity of protein tar- get to the training data (Figure F.2 in Appendix F). This analysis suggests that the model does not only memorize the information, but exhibits degree of generalization. We can also conclude that RNAABAnCC performance is robust to the quality of protein structure prediction, provided the pLDDT score exceeds 70%.
In addition to the deep learning beanning-based scoring model, we explored several computational approaches to evaluate the generated sequences. Among these, one method yielded results that showed a meaningful correlation with the deep learning scores, supporting the validity of the model's out- put. Detailed descriptions of the computational analyses, along with examples of generated sequences for representa- tive samples and individual DeepCLIP score distributions, are provided in Appendix F.
To complement the ftreshold-depender performance anal- ysis and capture variability concealed by averaging across samples, we conducted cample-by-sample evaluation. Us- ing a fixed threshold, we classified sequences as high-affinity RNA (haRNA) if their assigned score exceeded this value. For each sample in the test set, we then quantified the propor- tion of sequences meeting the haRNA criterion, presented in Figure 7. The fixed threshold value of 0.75 was chosen as detailed in the Appendix F. As one can see from Figure 7, generates more high-affimit sequences than a random generator for 56 out of 71 test samples. For 33 of
Novelty and diversity of generated sequences. For our model's goal of accelerating experimental RNA design, di- versity and novelty are essential metries. By generating a wide range of diverse sequences, the model increases the chances of identifying optimal binders with varying affini- ties for the target protein, ensuring more effective candidates for experimental validation. Furthermore, the generation of novel sequences allows the discovery of unique RNA- protein interactions, which can lead to innovative therapeu- tic applications.

---

BAnG; Bidirectional Anchored Generation for Conditional RNA Design
Table High-affiniyy RNA sequenee proportion generated by Gen- eIRNA and RNA-BAnG for each RNA Compendium sample. The protein sequences are identical for the same genes. Although un- derlined samples do not meet our test set selection criteria, they are included to enhance the diversity the comparison. Sample IDs mapping RNAcompete IDs mentioned in Appendix E.I.
Proportion of haRNA
0.6 0.4
SAMPLE ID
GENE
GENERRNA
106
SRSF1
0.42
0.49
258
256
178
069
008
258
027
037
208
091
107
SRSFI
0.59
0.74
108
SRSFI
0.56
0.77
Sample
109
SRSFI
0.62
0.85
110
SRSF1
0.43
0.66
Figure 8. Proportion of high affinity sequences generated by RNA- BAnG (green) and RNAFLOW (yellow). Samples are sorted by the performance of RNA BAAG. Sample IDs mapping to RNA- compete IDs mentioned in Appendix E.I.
121
ELAVLI
0.64
0.91
To quantify the diversity metric, we compute the ratio of the number of distinct clusters at threshold of 0.9 sequence identity the total number of generated sequences. A lected threshold of 0.9 is often considered as one of the lower limits used in RNA sequence clustering, especially when aiming to identify highly similar sequences or gene families (Edgar, 2018). The resulting average diversity across the test set is 0.93 0.13, indicating that the generated sequences are highly varied. The novelty defined by the proportion of RNA sequences in the generated set that are not similar to the training data. We identify sequences for each test sample that have no similarity (see Appendix G for more details) to the training set, and the novelty is calculated as the ratio of these sequences to the total number of generated sequences. The resulting average novelty across the test set 0.99 + 0.01, indicating that the generated sequences are highly novel and distinct from the training data, which is positive outcome for our model.
The second baseline model is RNAFLOW, which uses RNA structure prediction tools. Due to its much lower speed, approximately 50 times slower than RNA-BAnG, we only generated RNA sequences for proteins that had zero se- quence similarity with our training set, resulting in 100 sequences of length 50 for each protein. Unfortunately, RNAFLOW generated sequences with an unusually high frequences of G and C nucleotides spectively regardless of protein structure sequence. As a result, it produced high-affinity sequences for only couple of proteins, and none for the others. Consequently, RNA-BAnG outperformed RNAFLOW on most of the test samples (Figure 8).
62% and 32%, re-
The low performance of RNAFLOW can be explained by discrepancy between its training process and the infer- ence conducted. Concretely, RNAFLOW was trained on protein sequences and structures truncated to 50 residues, ensuring the inclusion of the protein binding site. However, we tested it on proteins of varying lengths, ranging from 177 to 589 amino acids, without highlighting their binding sites through truncation (since this information is unavailable). Moreover, the proteins we tested had zero sequence similar- ity those used to train GNA-BAnCC As both RNA-BAnG and RNAFLOW are trained on PDB-derived data, it is rea- sonable to assume low similarity to RNAFLOW'S training set as well. This could make prediction of their structure in complex with RNA more challenging. These limitations significantly restrict the applicability of RNAFLOW to RNA sequence design, especially when compared RNA-BAnG.
Comparison with similar models. We compare our model, RNA-BAnG, with two other methods, GenerRNA (Zhao et al., 2024) and RNAFLOW (Nori & Jin, 2024), as they are the most relevant existing methods for gener- ating RNA sequences for a diverse set of proteins. The first model, GenerRNA, leverages substantial collection of RNA sequences known interact with specific pro- tein during its fine-tuning process, allowing it to generate additional sequences with similar binding properties. Conse- quently, our comparison is limited to the proteins for which GenerRNA was fine-tuned by its authors. We used their published inference results, removing generated RNA larger than 50 nucleotides, leaving 921 and 909 sequences for the ELAVLI and SRSFI proteins, respectively. These proteins already present in the RNA Compendium samples. As shown in Table 3, our model generates more high-affinity RNA sequences than GenerRNA. It is important to note that BAnG generates these sequences without relying on any additional information, whereas GenerRNA required extensive data mining from multiple experimental studies.
5. Conclusion
This study introduces a novel deep-learning model, RNA- BAnG, and a sequence generation method, BAnG, for de- signing RNA sequences that bind to a given protein. Unlike previous approaches, our model demonstrates remarkable
8

---

BAnG; Bidirectional Anchored Generation for Conditional RNA Design
flexibility by eliminating the need for extensive structural or interaction data during inference. Although our model relies on protein structure, AlphaFold predictions make sequence information sufficient. This innovation significantly broad- ens the applicability of our method, making it a versatile tool for RNA- -roteint interaction studies.
thankful to Yoann Fauconnet and Jessica Andreani from 12BC, CEA Paris-Saclay for their support and advise on RNA scoring models. The authors also acknowledge the support from the PHC MAIMONIDE 2023 program, project number 49285YJ.
Our method is based on the observation that RNA sequences contain functional binding motif's while the surrounding sequence context is less critical for interaction. When evalu- ated on synthetic task against other sequence generation approaches, BAnG demonstrated superior performance. Im- portantly, the method's design makes it applicable beyond RNA-protein interactions, extending to any scenario where the focus is on optimizing functional subsequences within larger sequence, According to the state-of-the-an Deep- CLIP scoring, RNA-BAnG outperforms existing methods, generating higher proportion of sequences with strong predicted binding affinity. The generated sequences exhibit both diversity and novelty, expanding the range of potential RNA candidates for further experimental validation.
References
Aita, T. and Husimi, Y. Biomolecular information gained through in vitro evolution. Biophysical reviews, 2:1-11, 2010. Publisher: Springer.
Altschul, S. F., Gish, W. Miller, W. Myers, E. W., and Lipman, J. J. Basic local alignment search tool. Jour- nal of Molecular Biology. 215(3):403-410, October 1990. ISSN 00222836. doi: 10.1016/50022-28366055 80360-2. URL TRRLtttse://LLniiggtueeelsss com Andress, C., Kappel, K., Villena, M. E., Cuperlovic- Culf. M., Yan, H., and Li, Y. DAPTEV: Deep ap- tamer evolutionary modelling for COVID-19 drug de- sign. PLOS Computational Biology. 19(7)ee010774, July 2023. ISSN 1553-7358. doi: 10.1371/journal. 1011.1010774. URL https: http/xpplos...... aelos.org/10 1371/ .cb1.1000774.
Future work could focus on integrating experimental feed- back further refine the model, optimizing its architecture for enhanced performance, and improving its usability for broader applications. Aationn....iiii Additionally, experimental validation of the generated sequences would provide further insights into the practical applicability of our method. In summary, our work represents a significant step forward in RNA se- quence generation, offering a powerful and flexible tool for researchers in the field of RNA-protein interactions.
Bengio, Y., Ducharme, R., and Vincent, P. A Neural Probabilistic Language Model. In Leen, T., Dietterich, T., and Tresp. V. (eds.), Advances in Neural Infor- mation Processing Systems, volume 13. MIT Press, 2000. URL htw.httss/prroccedngg..ooo nerrips. cc/paper_ he//apper_filessappre////////// 77872062200155722544007888888888 pdf.
Software and Data. The code and the model, along with the model weights, are available at https: / /giit thub. com/ s/rrsskkpppp BNAGG The code allows for run- ning the RNA sequences generation process for any correct protein 3D structure as input.
Berman, H. M. The Protein Data Bank. Nucleic Acids Research, 28(1):235-242, January 2000. ISSN 13624962. doi: 10.10933nar/28.1.22 1.235. URL https:// a academi c. oup http://///////aattcllleooooooooo aoi/10.1033/rar/236
Impact Statement
The main purpose of this work is to advance the field of gen- erative models. However, the applications of this method may have social and industrial benefits. Potential applica- tions include in-silico SELEX approaches, RNA vaccine design, the development of novel drugs, and some other pharmaceutical tasks.
Callaway, E. Chemistry Nobel goes to develop- ers of AlphaFold AI that predicts protein struc- tures. Nature, 634(8034):525-52 ISSN 0028-0836, 1476-4687.
October 2024. doi: 10.1038/
041586-024-0321147. URL htttt:/ttss://w/t nature con/aarticlees/d158----1-1
Acknowledgments
Chen, J. C., Chen, P.. Shen, M. W., Wornow, M., Bac, M., Yeh, W.H.., Hsu, A., and Liu, D. R. Generat- ing experimentally unrelated target molecule-binding highly functionalized nucleic-acid polymers using ma- chine learning. Nature Communications, 13(1):4541, August 2022. ISSN 2041-1723. doi: 10.1038/
This work was granted access to the HPC resources of IDRIS under the allocation 2024-AD01 1015647 made by GENCI. A substantial part of the computations presented in this paper was performed using the GRICAD infrastructure inttps://gricad.umi univ grenoblc-alpes.ff which is supported by Grenoble research communities. The authors are very
$41467-022-31955 URL https: /wwww nature.
com/articles/ wtt.cclleeeeeeeeeeee 31955-4.

---

BAnG; Bidirectional Anchored Generation for Conditional RNA Design
Dapkunas, J., Timinskas, A., Olechnovič, K., Tomku- viené, M., and Venclovas, C. PP13D: web server for searching, analyzing and modeling protein-protein, protein protein-peptide and protein-nucleic
gert, M., Badkundri, R., Shafkat, I., Gong, J., Derry, A., Molina, R. S., Thomas, N., Khan, Y., Mishra, C., Kim, C., Bartie, L. J., Nemeth, M., Hsu, P. D., Sercu, T., Candido, S., and Rives, A. Simulating 500 mil- lion years of evolution with a language model, July 2024. URL httttttt///btt /bioorix 10. 1111112024..1 07 01. 600583.
Nucleic Acids Research, nucleic acid interactions. July 2024. ISSN 0305-1048, 1362-4962. doi: 10.1093/ nar/gkae278. URL httttttpps//accaamiiiiiee wwp..c...p..ooooo nar
hagg/looku//di..
Hendrycks, D. and Gimpel, K. Gaussian Error Linear Units (GELUs), 2016. URL https: /aapss///axxvv.o/// htt/ixxxvvvorr/ /bs/ 1606. 08415. Version Number:
Dehghani, M., Djolonga, J., Mustafa, B., Padlewski, P., Heek, J., Gilmer, J., Steiner, A., Caron, M., Geirhos, R., Alabdulmohsin, I,, Jenatton, R., Beyer, L., Tschannen, M., Arna,, A., Wang, X., Riquelme, C., Minderer, M., Puigcerver, J., Evci, U., Kumar, M.. Steenkiste, S. v., Elsayed, G. F., Mahendran, A., Yu, F., Oliver, A., Huot, F. Bastings, J., Collier, M. P., Gritsenko, A., Birodkar, V., Vasconcelos, C., Tay, Y., Mensink, T., Kolesnikov, A., Pavetić, F., Tran, D., Kipf, T., Lučić, M., Zhai, X., Keysers, D., Harmsen, J., and Houlsby, N. Scaling Vision Transformers 22 Billion Parameters, Febru- ary 2023, URL http:// 05442. arXiv:2302.05442 [cs].
Hentze, M. W., Castello, A., Schwarzl, T., and Preiss, T. A brave world of RNA-binding proteins. Nature Reviews Molecular Cell Biology, 19(5):327-341, May 2018. ISSN 1471-0072, 147 1471-0080.
doi: 10.1038/ nrm nrm.2017.130. URL ht https: / www.nature.com articles/ nrm. 2017.1213
Im, J., Park, B., and Han, K. A generative model for constructing nucleic acid sequences binding to pro- tein. BMC genomics, 20(Suppl 13):967, 2019. Publisher: Springer.
Edgar, R. C. Updating the 97% identity threshold for 16S ribosomal RNA OTUs. Bioinformatics, 34(14):
Ingraham, J. B., Baranov, M., Costello, Z., Barber, K. W., Wang, W., Ismail, A., Frappier, V., Lord, D. M., Ng- Thow-Hing, C., Van Vlack, E. R., Tie, S., Xue, V., Cowles, S. C., Leung, A., Rodrigues, J. V., Morales- Perez, C. L., Ayoub, A. M., Green, R., Puentes, K., Oplinger, F., Panwar, N. V., Obermeyer, F., Root, A. R., Beam, L.. Poelwijk, F. J., and Grigoryan, G. Il- luminating protein space with programmable gener- ative model. Nature, 623(7989):1070-1000 ber 2023. ISSN 0028-0836, 1476-4687. doi: 10.1038/ $41586-003-06622--8 URL http:///ww...........ooo/////aa. http::/:///www.aau....... com/articcees////e
2371-2375, July 2018. ISSN 1367-4803, 1367-4811. doi: 10.1099/bionnoomaattstttyyy //academic oup http:///i////oo//aaaaaooommmoete article/3//1423711/
URL https:
Fasogbon, I. V. Ondari, E. N., Tusubira, D., Rangasamy, L., Venkatesan, J., Musyoka, A. M., and Aja, P. M. RE- CENT FOCUS IN NON-SELEX- NON-SELEX-COMPPTTAAII COMPUTAAINAA APPROACH FOR DE NOVO APTAMER DESIGN: A
Novem-
MINI REVIEW. Analytical Biochemistry, pp. 115756, 2024. Publisher: Elsevier.
Iwano, N., Adachi, T., Aoki, K., Nakamura, Y., and Hamada, M. Generative aptamer discovery using
Grønning, A., Doktor, T. K., Larsen, S., Petersen, U.,
Holm, L. L., Bruun, G., Hansen, M. B., Hartung, A.-M., Baumbach. J., and Andresen, B. S. DeepCLIP: predicting the effect of mutations on RNoAein-RNA binding with deep learning. Nucleic Acids Research, pp. gkaa530, June 2020. ISSN 0305-1048, 1362-4962, doi: URL
RaptGen. Nature Computational Science, 2(6):378-
386, June 2022. ISSN 2662-8457 doi: 10.1038/
$43588-022 02200222-0-222 URL https: :tttpp::/we/ http//.wwau........m com con/articlesse s435ce///ss////// s43588 022-00249-6
10. 10.10933/ar/gkka553. oup. 1093
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M.,
Ronneberger, O., Tunyasuvunakool, K., Bates, R., Zidek, A., Potapenko. A., Bridgland, A., Meyer, C., Kohl, S. A. A., Ballard, A. J., Cowie, A., Romera-Paredes, B., Nikolov, S., Jain, R., Adler, J., Back, T., Petersen, S., Reiman, D., Clancy, E., Zielinski, M., Steinegger, M.. Pacholska, M., Berghammer, T., Bodenstein, S., Silver, D., Vinyals, O., Senior, A. W., Kavukcuoglu, K., Kohli, P., and Hassabis, D. Highly accurate protein structure prediction with AlphaFold. Nature, 596(7873):583-58) August 2021. ISSN 0028-0836 1476-4687, doi: 10310 s41586- 11886002--3819--2 URL ht https: wwww.wwwwommoom..... nat ure.. com/articles/ 4184415585800
Guo, P., Coban, O., Snead, N. M., Trebley, J., Hoeprich, S., Guo, S., and Shu, Y. Engineering RNA for Tar- geted siRNA Delivery and Medical Application. Ad- vanced Drug Delivery Reviews, 62(6):650-66) 666, April 2010. ISSN 0169409X. doi: ad1016/j.ddd::0111 03.008. URL
sevier.
com/ comp:/etriiove//l////88840000000
Hayes, T., Rao, R., Akin, H., Sofroniew, N. J., Oktay, D., Lin, Z.,, Verkuil, R. Tran, Q., Deaton, J., Wig-
10

---

BAnG; Bidirectional Anchored Generation for Conditional RNA Design
Katoh, K. and Standley, D. M. MAFFT Multiple Sequence Alignment Software Version 7: Improvements in Perfor- mance and Usability, Molecular Biology and Evolution, 30(4):772-780, April 2013. ISSN 0737-4038, 1537- 1719. doi: 10. 10.100//mollee//mss00000 URL httpt://///// pcademic.oup com wwwp///////.er...eeeeeeeeeeeeeee http//atiiilllc -ttptiooieeeeoeou////oooo doi/10.100 /mwp////m/sooll/o
Obonyo, S., Jouandeau, N., and Owuor, D. RNA Generative Modeling With Tree Search. In 2024 IEEE Conference on Computational Intelligence in Bioinformatics and Com- putational Biology (CIBCB), Pp. 1-9. IEEE, 2024.
Olechnovič, K. and Venclovas, C. Voronota: A fast and reliable tool for computing the vertices of the Voronoi diagram of atomic balls. Journal of Com- putatiional Chemistry, 35(8):672-681, March 2014. ISSN 0192-8651, 1096-987X. doi: 10. 10.1002/jec. 23538. URL https: com/ doi /10 1002/ jcc 23538.
Kim, N., Gan, H. H., and Schlick, T. A computational proposal for designing structured RNA pools for in vitro selection of RNAs. RNA, 13(4):478 492, 2007a. Pub- lisher: Cold Spring Harbor Lab.
Olechnovië, and Venclovas, C. VoroContacts: tool for the analysis of interatomic contacts in macromolec-
Kim, N., Shin, J. S., Elmetwaly, S., Gan, H. H., and Schlick, T. RagPools: ANA-As-Graph--Pon
a web server for as- sisting the design of structured RNA pools for in vitro se- lection. Bioinformatics, 23(21):2959-2966 2007b. Pub- lisher: Oxford University Press.
ular structures. Bioinformatics, 37(24):48734477
December 2021. ISSN 1367-4803, 1367-4811.
doi: 10. 10.109//ioonfomaatissstauaaubl / //academic c. wwtp:pp...ommo.ooiooooaaaaaaaaaa (Tcrccle/37722/4873/555
URL https:
Kingma, D. P. and Ba. J. Adam: A Method for Stochastic Optimization, January 2017. URL http: / arxiv. org/abs/ 412. 6980. arXiv:1412.6980 [es].
Ozden, F., Barazandeh, S., Akboga, D., Tabrizi, S. S., Seker, U. O. S., and Cicek, A. E. RNAGEN: A generative adversarial network-based model to gener- ate synthetic RNA sequences to target proteins, July 2023. URL httl: htt.:htttp///ixiiiii http:///boooiviiiio org/ wttglookkg//oooooooooooooooooooo 10. 10.1101/2023 07 .17.1.1.1.. 548246.
Korhonen, J. H., Palin, K., Taipale, J., and Ukko- nen, E. Fast motif matching revisited: high-order PWMs, SNPs and indels. Bioinformatics, 33(4):
514-521, February 2017. ISSN 1367-4803, 1367-4811. doi: 10.109//bionformaics/wwwwww
URL https:
Park, B. and Han, K. Discovering protein-binding RNA tifs with generative model of RNA sequences. Computa- tional Biology and Chemistry, 84:107171, February 2020. ISSN 14769271. doi: 10. 10.1016/..ompbbiochem..22
//academic oup hwtp://oooo/ooooooooccccccomco.. axticle/33/4/51///
Lee, G., Jang, G. H., Kang, H. Y., and Song, G. Predicting aptamer sequences that interact with target proteins us- ing an aptamer-protein interaction classifier and Monte Carlo tree search approach. PloS one, 16(6))e0253660, 2021. Publisher: Public Library of Science San Francisco, CA USA.
107171. URL htt.:/tttt./////iinnnn...eeeeeee com htnm:/etcieeeeeppa//111//1222220
Ray, D., Kazan, H., Chan, E. T., Castillo, L. P., Chaudhry, S., Talukder, S., Blencowe, J., Morris, Q., and Hughes, T. R. Rapid and systematic analysis of the RNA recog- nition specificities of RNA-binding proteins. Nature Biotechnology, 27(7))667-677 July 2009. ISSN 1087- 0156, 1454-166966 doi: 10.1038/nb..150.. URL https: / / //www. nature. com/ art articcesss..ie..... /nbt.. 1550.
Li, D., Huang, R., Cui, C., Towey, D., Zhou, L., Tian, J., and Zou, B. RNA-Protein Interaction Prediction Based Deep Learning: A Comprehensive Survey. arXiv preprint axXiv:2410.00077, 2024.
Ray, D., Kazan, H., Cook, K. B., Weirauch, M. T., Na- jafabadi. H. S., Li, X., Gueroussov, S., Albu, M., Zheng, H., Yang. A., Na, H., Irimia, M., Matzat, L. H., Dale, R. K., Smith, S. A., Yarosh, C. A., Kelly, S. M., Na- bet, B., Mecenas, D., Li, W., Laishram, R. S., Qiao, M., Lipshitz, H. D., Piano, F., Corbett, A. H., Carstens, P., Frey, B. J., Anderson, R. A., Lynch, K. W., Pe- nalva, L. O. F., Lei, E. P. Fraser, A. G., Blencowe, B. J., Morris, Q. D., and Hughes, T. R. A compendium of RNA-binding motifs for decoding gene regulation. Na- ture, 499(7457) July 2013. ISSN 0028-0836, 1476-4687. doi: 10.1033mmaturee233111 URL https: / //www.nature.cmm www..raauure.ccm//attlleeeeeeeee
Li, W. and Godzik, A. Cd-hit: fast program for clustering and comparing large sets of protein or nucleotide sequences. Bioinformatics, 22(13):1658- 1659, July 2006. ISSN 1367-4811, 1367-4803. doi: 10. 10.109//hainnoomaiiccchlll&& URL https: / //academic oup wttp://pww....iiiooooooot.aaiiaa atticlee22213/1655/22
Nori, D. and Jin, W. RNAFlow: RNA Structure & Se-
quence Design via Inverse Folding- Flow Matching,
June 2024. URL http::/arxiv......//////// org. abs / abb//405.. 2405. 18768. arXiv:2405.1 18768 [q-bio]

---

BAnG; Bidirectional Anchored Generation for Conditional RNA Design
Reddi, J., Kale, S., and Kumar, S. On the Convergence of Adam and Beyond, April 2019. URL http://att /xxvvv. org/ hrg/abos abs/1904 09237. arXiv:1904.09237 [es].
J. A., Bult, C. J., Chen, R., Zhao, Y., Wood, V., Ruther- ford, K., Rivas, E.,, Cole, J., Laulederkind, S. J. F,, Shi- moyama, M., Gillespie, M. E., Orlic-Milacic, M., Kalvari, I., Nawrocki, E., Engel, S. R., Cherry, M., Team, S., Be- rardini, T. Z., Hatzigeorgiou, A., Karagkouni, D., Howe, K., Davis, P., Dinger, M., He, S.. Yoshihama, M., Ken- mochi, N., Stadler, F,, and Williams, K. P. RNA- central: a hub of information for non-coding RNA se- quences. Nucleic Acids Research, ((DD):D221-D223) January 2019. ISSN 0305- 1362-4962. doi: 10. 1093nar/gkyy103.. URL hwwps::ttps... /twp///caaaccccccmmmm coup.
Rhiju, D., Shujun, H. Alissa, H., and Rachael, K. Nucleic Acid Assessment CASPI6, December 2024. URL https://predicttioneee httpc//pppo/cccppeeeeetttteeeeee
Day Day-3/Day3 3/Day3-01 1ay-3/Dayy-11-rresesee -ree--rrhhhhhsePeeee Assessement PRressmmett_Puttaaaa_RRRRRL pptx.
Schneider, T. D. and Stephens, R. Sequence logos: new way to display consensus sequences. Nucleic
Acids Research, 18(20):09776100, 1990. ISSN 0305-1048, 1362-4962. doi: 10.1093/nar/1..20.00 2..0..097
Torkamanian-Afshat M., Nematzadeh, S., Tabarzad, M., Najafi, A., Lanjanian, H., and Masoudi-Nejad, A. In silico design of novel aptamers utilizing hybrid method of machine learning and genetic algorithm. Molecular diversity, 25:1395-1407, 2021. Publisher: Springer.
URL https: /llps://academic.. oup. wwm/na..cccoaaaaaaooaoaaoooo
6097.
Shin, I., Kang. K., Kim, J., Sel, S., Choi, J., Lee, J.-W., Kang, H. Y., and Song. G. AptaTrans: deep neural network for predicting aptamer-protein interaction using pretrained encoders. BMC bioinformatics, 14(1):447, 2023. Publisher: Springer.
Tseng, C.-Y., Ashrafuzzaman, M., Mane, J. Y., Kapty, J., Mercer, J. R., and Tuszynski, J. A. Entropic Fragment- Based Approach to Aptamer Design. Chemical Biology & Drug Design, 78(1):1-13, 2011. Publisher: Wiley Online Library.
Stormo, G. D., Schneider, T. D.. Gold, L., and Ehrenfeucht, A. Use of the 'Perceptron' algorithm to distinguish trans- lational initiation sites in E. coli. Nucleic Acids Research, 1019):9997-3011, 1982. ISSN 0305-1048, 1362- 4962. doi: 10. 10.1093/mar/1009922999 academic oup. wttp://arraraieeeoaloaaeeeee doi/10 1wt.:/0000//a11//0
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. Attention Is All You Need, August 2023. URL http: arriv. org/ prg/abs/// 1706. 037 62. trXiv:1706.0333 [es].
URL https://
2997.
Wang. Y., Mistry, B. A., and Chou, T. Discrete stochastic models of SELEX: Aptamer capture probabilities and protocol optimization. The Journal of Chemical Physics, 156(24), 2022. Publisher: AIP Publishing.
Su, J., Lu, Y., Pan, S., Murtadha, A., Wen, B., and Liu, Y. RoFormer: Enhanced Transformer with Rotary Position Embedding, November 2023. URL http: arxiv. org/abs/ abs/ 2004. 09864. arXiv:2104.09864 [es].
Wang. Z., Liu, Z., Zhang, W., Li, Y., Feng, Y., Lv, S., Diao, H., Luo. Z., Yan, P., He, M., and others. AptaDiff: de novo design and optimization of aptamers based on diffusion models. Briefings in Bioinformatics, 25(6): bbae517, 2024. Publisher: Oxford University Press.
Thavarajah, W., Hertz, L. M., Bushhouse, D. Z., Archuleta, C. M., and Lucks, B. RNA Engineering for Pub- lic Health: Innovations in RNA-Based Diagnostics and Therapeutics. Annual Review of Chemical and Biomolecular Engineering, 12(1):263-286, June
2021. ISSN 1947-5438, 1947-5446. doi: 10.1146/
Zhang, B. and Sennrich, R. Root Mean Square Layer Normalization, October 2019. URL http: //t::///xxvv../ org brg/abs/// 1at...//99..0000 07467. a:Xiv:1910.07467 [cs].
umnueeycheembieeng-000000--00000 URL https:
//www //tp:/aannualrevvessss..///////a annureerrchhetb..eeng--1101-----
1146/
Zhang, Y., Jiang, Y., Kuster, D., Ye, Q. Huang, W., Fürbacher, S., Zhang, J., Tang, Z., Ibberson, D., Wild, K., and others. Single-step discovery of high-affinity RNA ligands by UltraSelex. 2023.
The RNAcentral Consortium, Sweeney, B. A., Petrov, A. I., Burkov, B., Finn, R. D., Bateman, A., Szymanski, M., Karlowski, W. M., Gorodkin, J., Seemann, S. E., Can- none, J. J., Gutell, R. R., Fey, P., Basu, S., Kay, S., Cochrane, G., Billis, K., Emmert, D., Marygold, S. J.,
Zhang, 7., Chao, L.,, Jin, R., Zhang, Y., Zhou, G., Yang, Y., Yang, Y., Huang, K., Yang, Q., Xu, Z., and others. RNA- Genesis: Foundation Model for Enhanced RNA Sequence Generation and Structural Insights. bioRxiv, pp. 2024-12, 2024. Publisher: Cold Spring Harbor Laboratory.
Huntley, R. P., Lovering, R. C., Frankish, A., Chan, P. P., Lowe, T. M., Bruford, E., Seal, R., Vandesompele, J., Volders, P.-J., Paraskevopoulou, M., Ma, L., Zhang, Z., Griffiths-Jome S., Bujnicki, J. M., Boccaletto, P., Blake,
12

---

BAnG; Bidirectional Anchored Generation for Conditional RNA Design
Zhao, Y., Oono, K., Takizawa, H., and Kotera, M. Gen- erRNA: A generative pre-trained language model for de novo RNA design. PLOS ONE, 19(10):e03 10814, Oc- tober 2024. ISSN 1932-6203. doi: 10.1371/journal. pone.0310814. URL https: //tp:///xxxpppp www.pppll..o///..1..//... org/10. 1371/journal pone. 0310814.
13

---

BAnG; Bidirectional Anchored Generation for Conditional RNA Design
Appendix
A. RNA-BAnG Architecture
RNA-BAnG architecture is schematically presented Figure of the main text and is composed of protein and nucleotide modules, detailed in Figure A.I. The latent dimension the model C2 128, all heads dimensions are set ch 64. Feedforward blocks have scaling factor 2. We set the number of protein and nucleotide modules 10 each. While reducing this number led to poorer performance, increasing it did not yield any noticeable improvements. We adjusted the rest of the hyperparameters by selecting the smallest model size combination that ensured stable convergence. Resulting model contains 14,5 million parameters.
21] Protein sequerce
[Fa
[rui
[[a.8]
[ra
Prosein indices
Protein framnes
Nuclestecindees
Niclost soquence
Naclondiee tyee
One-hot Linear
One-hot Linear
One -bot Linear
Embedder
Embeddar
[1a
[Tra
Sequesce reprasaaton
RMSNorm
RMSNorm
RoPE Self
RoPE Self Attention
Repentable nucleotide module
Repestable protein
Geometric Self Attention
Cross Attention
MLP+GELU
GELU
Protein rngresemationn
Masiccttlee
Figure A.I. Schematic illustration of the protein (on the left) and nuclentide (on the right) modules in the RNA-BAnG architecture. Here, Tha the number of nucleotides, Faa is the number protein residues, X stands for concatenation.
Algorithm A. describes the Geometric Attention block. It takes frames T; and single representation s; of every protein residue in the chain as inputs. The number of attention heads is 12, the number of query points is Nquery paints = 4, the number of value points is Nvalue points = 8. The weight per head 7h E IR is the softplus of a learnable scalar, w is weighting factor. We adjusted the block's s hyperparameters to align with the AlphaFold2 IPA choices.
Algorithm A.1 Geometric Attention
input
8; TV
q.
hp h.pp < LinearNoBias(s.)
p [1, N Nquery points
f + I LinearNoBias($
hp
p E [1, Npoint values
w
+
9.uun points
4/ - softmax 2/w Ep TT c.. T, k)p1(1)
hp I E, a/ (T)
0,
output Linear (concath.p(o hp (|o,
14

---

BAnG; Bidirectional Anchored Generation for Conditional RNA Design
B. Synthetic Task
B.1. Toy Model
To avoid direct memorization on the synthetic task, we opted for a compact model, same for each tested method. Its final configuration is a two-block transformer (with a hidden dimension of 64) with the RoPE positional attention and two attention heads, resulting in 17k parameters. We trained the model for each method for 80k steps with batch size of 8, using the Adam optimizer (Kingma & Ba, 2017) with the learning rate of 0.0001. During the inference, the length was fixed to 50 tokens for iterative methods.
B.2. Synthetic Sampling
During the inference, the sequence length was fixed 50 tokens for iterative methods. For all the methods tested on the synthetic task, sample tokens from their distributions using the top-k strategy with k 4.
B.3. Generated Synthetic Sequences
Table B.1 lists examples of sequences generated for the DoubleBind task with each tested approach.
Table B.I. Example of generated sequences for the synthetic task. Expected synthetic motifs are underlined.
GENERATIVE METHOD
SEQUENCES
BANG
AUTOREGRESSIVE
IANG ENTROPY
ITERATIVE ENTROPY
AANAUUGGGGUUUUCCCCCCCCUAAAAAAACC
B.4. Additional Synthetic Results
To assess the robustness of BAnG to sequence length, we conducted additional synthetic experiments. Our method was tested on the SingleBind task under three scenarios: sequence lengths of 100 tokens, 200 tokens, and uniformly sampled between 40 and 50 tokens. The results, presented in Table B.2, show that BAnG performs consistently across these variations. The average length of the generated sequences was 46 tokens when trained on sequences sampled from the 40-50 token range.
Table B.2. BAnG performance SingleBind task with various sequence length. Scores represent the proportion of sequences that contain the correct synthetic motif.
LENGTH
SCORE 1
50
0.98
100
0.98
200
0.95
4(400, 50)
0.98
We also investigated BAnG's behavior when multiple motifs are present simultaneously in sequence. DualBindMix is variation of the DualBind task in which two motifs, separated by three tokens, appear in every training sequence, with anchor points placed randomly in the middle of either motif. The task, where motif separation is random, is referred to as DualBindMix Random. The results of these experiments are listed in Table B.3. In the DualBindMix BAnG generated the sequences containing two motifs simultancously, while in the DualBindMix Random most the time it generated only one motif. As expected, only the increased uncertainty in motif positioning negatively impacts the performance of our method.
15

---

BAnG; Bidirectional Anchored Generation for Conditional RNA Design
Table B.3. BAnG performance on tasks with several synthetic motifs. Values represent the proportion of sequences containing either of the synthetic motifs or both.
TASK
FIRST
SECOND
BOTH
DUALBIND
0.43
0.53
0.01
DUALBINDMIX
0.01
0.01
0.97
DUALBINDMIX RANDOM
0.47
0.46
0.05
C. Training Data
C.1. Protein Coupled Nucleotide Sequences
We defined two protein-RNA or protein-DNA chains as interacting at least one interaction occurred between their residues. We defined an interaction between two residues if they share at least one atom-atom contact, calculated using the VoroContacts software (Olechnovič & Venclovas, 2021). The VoroContacts method identifies contacts based on the Voronoi tessellation of atomic balls, constrained within the volvent-accessible surface (Olechnovië & Venclovas, 2014).
We excluded all samples where protein atom coordinates are ambiguous (had alternative locations). where the protein chain contains non-canonical residues (though we substitute 'SCE' with 'CYS' and 'MSE' with 'MET' beforehand), or where the nucleotide chain contains non-standard residues or a mixture of standard RNA and DNA residues. We also excluded samples with the nucleotide sequence length of less than 10 residues. To avoid potential computational resource problems, included in the training only samples with protein length of fewer than 500 residues.
During training, we split the data by protein sequence homology. We measured it using the clustering provided by PPI3D (Dapkūnas et al., 2024), where proteins within the same cluster share less than 40% sequence similarity with those in other clusters. We allocated samples from 95% of randomly selected clusters to the train set and the rest we used for validation. This approach allows tracking the model generalization across the protein space. To enhance our potential test set, we removed from the training data clusters containing proteins from the PDB samples 5ITH (chain A), 7CRE (chain A), 6QW6 (chain R), 8OPS (chain B), and 1CVJ (chain A).
Some proteins and nucleotide sequences are overrepresented in the data, which may lead to training imbalance. To address this, we introduced an additional level of sample clustering. First, we clustered nucleotide sequences (DNAs and RNAs separately) based on sequence similarity, grouping those with 90% identity using CD-HIT-EST (Li & Godzik, 2006). Then, we clustered the samples based on the combination of protein and nucleotide sequences clusters. At each epoch during training, we selected eight random samples (the mean cluster population) to represent each of the latter clusters, repeating the samples when necessary.
The resulting dataset consisted of I 123,043 samples, distributed across 3,580 protein sequence clusters, 2,807 nucleotide sequence clusters (915 RNA and 1.892 DNA), and 12,667 combined clusters. The protein sequences in the dataset have a mean length of 155 residues with a standard deviation of 90. For nucleotide sequences, RNA lengths average 1,834 nucleotides (= 1, 1,564), while DNA lengths average 76 nucleotides (+ 78).
To prevent potential computational resource issues and focus the model on the binding motifs, we truncated nucleotide sequences exceeding 300 residues during training and validation. This truncation limited sequences to 300 residues centered around the anchor point, which was selected as described prior to the truncation.
C.2. Standalone RNA Sequences
From RNAcentral database we selected sequences containing only standard residues and with lengths between 10 and 500 nucleotides. The selection was then deduplicated using CD-HIT-EST with 90% similarity threshold, resulting in a final set of 3 million sequences.
D. Training Details
As explained in the main text, we conducted the RNA-BAnG training in two steps. In the first step, we used a batch size of 64 and trained for 255k steps. In the second step, we used batch size of and trained for 216k steps. We stopped
16

---

BAnG; Bidirectional Anchored Generation for Conditional RNA Design
Table F.I. Example of sequences generated by RNA-BAnG. Samples are ordered by decreasing proportion of high-afinity sequences. First two generated nucleotides are in bold and marked by red color.
RNACOMPETE ID
SEQUENCES
AUUUUUAAAUAAUUUUUAAAAAAAA
RNCMPT00077
UAUUAUUUAAUUUUUAAAA
GUAUGUAUUUUAUCCUU
CICGAACGCHAAAAACCUUUUUACUUUAUUCC
RNCMPT00173
AUUGAAACCCUUUUNGGCUUUUUUUAAAAAAA
GAACCAAGOGAGGAAAAGGAAAAAAAAAAAAA
CUUGUGUGAAAAGG
RNCMPT00167
CCCCUUCCCCCAACCCCCCCCCCCCCCCCCCC
RNCMPT00133
CCAAGGAAATTTQUUTAAAAAAAAAAAAAAAC
training when the validation loss decline became visually negligible. In both training steps, used the ASMGrad (Reddi al., 2019) variation of the Adam optimizer (Kingma & Ba, 2017) with default parameters and the learning rate of 0.0001. Learning rate was warmed up linearly for the first 1k steps and then decayed exponentially with 7 = 0.99 and period of 1k steps. Complete training took 4 days on a single MI120 AMD GPU.
E. Evaluation Protocol
E.1. Test Data
The RNA Compendium study provides 244 samples, each comprising a protein sequence paired with approximately 200,000 RNA sequences and their corresponding experimental binding scores. These samples are designated by the authors as RNCMPTO0XXX, where XXX represents numerical identifier. For simplicity, we occasionally refer to these samples solely by their numerical identifiers XXX.
We processed the data as follows. For each sample, RNA sequences were ranked by their binding scores. The top 2,000 sequences were labeled as the positive (interacting) class, while the bottom 2,000 were labeled as the negative (non- interacting) class. To eliminate sequence redundancy and prevent data leakage, duplicate RNA sequences were removed using CD-HIT-EST with sequence identity threshold of 90%. The remaining sequences were then randomly split into training and testing sets, that each test set would have 1,000 sequences. Those test sets referenced as positive and negative sets in the main manuscript. We then trained individual DeepCI.IP models for each sample, following the protocol outlined by the authors. The DeepCLIP model contains approximately 3,000 trainable parameters, and its training protocol - designed by the authors- automatically selects the best-perforning weights when provided with sets of positive and negative sequences. Only models achieving an area under the receiver operating characteristic curve (AUROC) of 0.95 or higher on the corresponding test sets were selected for further analysis.
E.2. RNA-BAnG Sampling
For RNA- MAA-BAnG, we sample tokens from their distributions using the top-k k strategy with = 4. Maximum sequence length set to 50 nucleotides. Average sampling time 20 minutes for 1,000 RNA sequences. To generate a random nucleotide sequence, we first choose a sequence length at random between 40 and 50 (inclusive). Then, we create a sequence of that length by selecting each nucleotide independently and uniformly at random. The length choice ensures that the mean sequence length of random samples matches that of SaA--BAnG-geneeee sequences (45.6 nucleotides).
F. Evaluation Results and Comparative Approaches
F.1. Additional Results
Table F.1 lists randomly selected examples of generated sequences for the two best and two worst performance test samples.
17

---

BAnG; Bidirectional Anchored Generation for Conditional RNA Design
We have also compared under the threshold-depeedeett performance curves for GenerRNA and RNAFLOW corresponding comparison sets (curves shown in the Figure F.1). RNA-BAnG demonstrated area values of 0.81 and 0.53,
surpassing GenerRNA with 0.66 and RNAFLOW with 0.13.
Proportion Above Threshald
1.0-
Proportion Above Threshold
1.0
0.8
0.8
0.6
0.6
0.4
0.4
0.2-
0.2
0.0
0.2
0.4 Threshald Value
0.8
1.0
0.2
0.6 Threshold Value
0.4
1.0
Figure F.I. Proportion of sequences above the threshold: The left panel shows results for GenerRNA (ycllow), RNA BAnG (green), and the corresponding positive (bluc) and negative (red) experimental sets used in the GenerRNA vs. RNA-BAnG comparison. The right panel presents RNAFLOW (yellow). ANA-BAnt (green), and the positive (blue) and negative (red) experimental sets used the RNAFLOW RNA-BAnG comparison. The values represent averages the entire comparison in each
Figure F.2 shows the proportion of high-affinity generated sequences as a function of AlphaFold PLDDT scores and protein sequence similarity with the train set.
1.0
Proportion of haRNA
-90
Protein PLDDT
85
0.6
0.4
80
0.0
0
20
40
60
80
100
Protein Sequence Similarity
Figure F.2. Proportion generated high-affinity sequenees function of protein sequence similarity to the training data and the AlphaFold PLDDT scores of the predicted protein structures.
Appendix details protein sequence similarity caleulations. Figure shows kernel density plots calculated over the whole test set of 71 samples. We assessed four different DeepCLIP score threshold values (0.65, 0.75, 0.85, 0.95, from left right in the Figure). We can conclude that the value of 0.95 is too stringent, the mean of the positive experimental sets approaches the value of 0.5. Subjectively, the most visually appealing threshold value is 0.75. Nonetheless, in the main text (Figure 6) we continuously assess all the values by plotting the mean proportions above certain threshold value.
Distributions of DeepCLIP scores for each of 71 test samples are depicted in Figure G. and Figure G.2.
F.2. A Ablation Studies
To better understand the contributions of individual components in our model, we conducted ablation studies focusing on Geometric Attention and the use of relative position encodings in Cross Attention. Removing the Geometric Attention block resulted in a relatively small change the number of model parameters (approximately 10%). while removing relative position encodings led change of less than 1
18

---

structure contains BAnG: Bidirectional Anchored Generation for Conditional RNA Design
0.65
0.75
0.85
0.95
5
4-
Density
2
1-
0.0
0.5
1.0 0.0
0.5
1.0 0.0
0.5
1.0 0.0
0.5
1.0
Proportions of haRNA
Figure E.3. Density plots of the proportion of high-affintyy sequences for several threshold values (0.65, 0.75, 0.85, and 0.95, listed on top) across the whole test set. RNA-BAnG is shown in green, random sequences in yellow, positive experimental sequences in blue and negative experimental sequences in red.
As shown in Figure F.4, Geometric Attention significantly enhances the performance of RNA-BAnG. While the contribution of relative position encodings in Cross Attention is more modest, it still provides measurable improvement.
1.0
1.0
Proportion Above Threshald
Proportion Above Threshold
0.8
0.8
0.6
0.6
0.4
0.4
0.2
1.0
0.2
0.6 Threshold Value
0.4
1.0
Threshold Value
Figure E4. Proportion sequences ahove the threshold: generated by RNA-BAnG (green) and randomly (yellow), the positive (blue) negative (red) experimental Dashed green line corresponds results of BAnG generation without Geometric Attention (on the left) or without relative position encodings in Cross Attention (on the right). The values here represent the averages for the entire comparison
F.3. Protein Preprocessing
We also investigated the impact of protein data preprocessing on RNA-BAnG performance. Our findings suggest that segmenting the protein structure into individual domains or removing intrinsically disordered regions (IDRs) can, in some cases, enhance the quality of generated sequences (Figure F.5). By domain here, we mean a compact, spatially distinct region of the protein with an ordered structure - recognized visually, by simply inspecting the structure and seeing how can be split into distinct rigid blocks.
For instance, in the case of the protein RNCMPT00133 (RNAcompete code), where RNA-BAnG performs the worst, removing IDRs (according to AlphaFold2 pLDDDT scores) from the structure led to tenfold increase in the number of generated high-affinity RNAs. Another example is protein RNCMPT00033, whose AlphaFold2-predicee multiple domains running the model on individual domains revealed that some domains yield significantly better results than using the full-length protein. However, such preprocessing is generally only practical when meta-in intaainforration about protein -NA interactions available, as RNAs may interact with multiple domains IDRs simultaneously.
F.4. Alternative Sequence Similarity Scoring Approach
We calculated the proportions of highly similar sequences (hsRNA) between the generated and experimental sets. sequence is considered highly similar if identified as such by blastn with -task blastn parameter. The resulting values
19

---

BAnG; Bidirectional Anchored Generation for Conditional RNA Design
RNCMPT00133
Domain A
p(haRNA) 0.03
p(haRNA) 0.28
RNCMPT00033
Domain
Domain B
CITTER
p(haRNA) = 0.18
p(haRNA) = 0.68
Domain
Domain D
p(haRNA) 0.52
P (haRNA) 0.70
p(haRNA) 0.72
Figure F.5. Proteins before (on the left) and after (on the right) the preprocessing. The proportion of haRNA generated by RNA-BAnG reported below the proteins or their domains. The domains were split using visual inspection guided by the pLDDTT confidence scores.
for each test sample are shown in Figure F.6. Across different samples, the proportions of generated sequences similar to the positive set are higher than those of similar to the negative set, averaging 29% and 23%, respectively. However, is important to note the limited applicability of blastn cases where sequences are intentionally designed without evolutionary signals and are experimentally selected based on the presence of relatively short binding motifs (Ray et al., 2013).
Proportion of haRNA
0.4
0.3
0.2
0.1
Sample
Figure F... Proportions of high-similarity RNA sequences, generated by RNA BAnG, to positive (blue) and negative (red) experimental sets for cach test sample. Test samples are ordered proportion of haRNA sequences, in Figure 7. Sample IDs mapping RNAcompete IDs mentioned in Appendix E.1
20

---

BAnG: Bidirectional Anchored Generation for Conditional RNA Design
E.5. Alternative Motif Search Scoring Approach
Motif searching tools in bioinformatics are specialized software applications designed identify recurring patterns, or motifs, within biological sequences such DNA, RNA, proteins. Motifs usually represented as Position Weight Matrices (PWMs), which capture the variability and conservation of cach nucleotide (or amino acid) at every position within the motif (Stormo et al., 1982). PWMs are obtained by aligning multiple sequences that share the motif and calculating the frequency of each symbol at each position, often followed by converting these frequencies into probabilities or log-odds scores (Schneider & Stephens, 1990). Once constructed, PWMs are then used to scan larger sequences or entire genomes to predict new sites with similar patterns.
As mentioned previously, the sequences in experimental are intentionally designed without evolutionary signals, making them inherently random in nature. This randomness makes aligning the sequences problematic. Therefore, followed the approach used by the RNA Compendium authors, considering only the top-10 scoring 7-mers (scoring data provided by the authors). We then aligned these 7-mers using MAFFT (Katoh & Standley. 2013), removing columns with gaps in more than half of the rows. This removal occurred only at the beginning or end of the alignment, ensuring no changes to the core motifs. The resulting aligned 7-mers were used to construct PWMs for each sample where the data was available in our experimental set.
Proportion Above Threshold
1.0
0.8
0.6
0.4
0.2
0.0
0.0
0.2
0.4
0.6
1.0
Threshold Value
Figure F.7. Proportion of sequences above the threshold: generated by RNA-BAnC (green) and randomly (yellow), the positive (blue) and negative (red) experimental The values here represent the averages for the entire test set.
Next, used MOODS (Korhonen al., 2017) the motif search tool. We selected it for its ease of use and its ability to incorporate dinucleotide frequencies, which improves search accuracy. We mapped its scores to 0-1 range using sigmoid function. Similar to the approach with the DeepCLIP scores, Figure Figure F.7 presents threshold-dependentt performance curve, showing the relationship between sequences with MOODS (sigmoid) scores above certain threshold and the threshold value itself. A high area value of 0.74 for the positive experimental set, coupled with a very low area value of 0.26 for the negative set, indicates that motif-based scoring is effective at distinguishing high-affinity sequences from others. RNA-BAnG achieved value of 0.49, indicating moderate success generating sequences that partially align with the positive set and demonstrating some ability avoid producing low-affinity outputs. The random set, with an area value of 0.40, suggests that model outperforms random generation. To provide more detailed analysis, defined haRNAs as those in which the highest motif score exceeds 0.4, a threshold selected using the elbow method from Figure F.7. The resulting sample-by-sample statistics are shown in Figure F.8.
This scoring approach correlates with the one based on DeepCLIP scores, showing a Pearson correlation coefficient of 0.64 between the proportion of haRNA sequences defined by DeepCLIP and those defined by motif-based scoring for each sample. Additional details on the usage MAFFT and MOODS can be found in Appendix G.
21

---

BAnG; Bidirectional Anchored Generation for Conditional RNA Design
Proportion of haRNA
0.4
0.2
0.0
100
100
Sample ID
Figure F.8. Proportions of high-an high-affinity RNA sequences for each test sample, generated by RNA-BAnG (green) and randomly (yellow). These are compared with the proportions in the positive (blue) and negative (red) experimental sets. Test samples are ordered by RNA-BAnG performance. Sample IDs mapping RNAcompete IDs mentioned in Appendix E.1.
G. A Alignment Tools Parameters
Clustering of nucleotide sequences was always performed using CD-HIT-EST (Li & Godzik, 2006) with a sequence identity threshold 90%. We used CD-HIT-EST with the following parameters: -c 0.9 -n -d -T 10 -U 10 -1 For protein sequence identity the train set calculations used blastp (Altschul al., 1990) with default parameters. For the nucleotide sequences similarity to the train set caleulations used blastp (Altschul et al., 1990) with default parameters.
For the 7-mers alignment we used MAFFT (Katoh & Standley, 2013) with the --nuc parameter. Motif search was performed with MOODS (Korhonen et al., 2017) using these parameters: -R -p 1.0 - batch. Motifs were represented in adm format. We have also set search window size equal to the motif size.
22

---

BAnG: Bidirectional Anchored Generation for Conditional RNA Design
ANCHIPT00145
RNCMPTOD00R
RNCHPTB0156
ANCHPT001
ANCMPTO01
TO0022
5
ANCHPT00023
RNCMPT00024
RNCMPT00245
RNCHPT00161 67
RNCMPT00027
RNCMPT00091
15-
10-
RNCMPTO0095
RNCMPT00117
20
Denaity
ANCMPT00136
RNCMPT00032
RNCMPT00274
RNCMPT00033
RNCHPT00169
ANCMPT00062
10
20
RNCMPT00139
RNCMPT00043
RHCMPT00171
RNCMPT00157
RNCMPT00246
RNCMPTD0206
10
5
0.00
0.25
0.50
0.75
1.00
9.30
0.25
059
0.75
1:00
0.00
0.25
0.50
0.75
1.00
0.00
0.25
059
6.5
1.00
0.00
0.25
0.50
0.75
1.00
0.03
8.25
3.30
0.75
1.00
Scores
Figure G.I. Distribution of DecpCLIP scores for the first part of the test samples (RNAcompete sample IDs on top). Scores of RNA-BAnG in green. Scores positive and negative experimental sequenees in bluc and red, respectively.
23

---

BAnG: Bidirectional Anchored Generation for Conditional RNA Design
ANCMPT00229
RNCMPT90249
RNCMPT00268
RNCMPT00269
RNCMPT00241
RNCMPT00184
5
ANCHPT00285
RNCMPT80049
RNCMPT80283
ANCMPT00052
RNCMPT0024]
RNCMPT00173
15-
10
ANCHPTO0205
RNCMPT00108
20
Denaity
ANCMPT00109
RNCMPT800T2
RNCMPT00076
RNCHPT00217
ANCMPT00229
RNCMPT00252
10
20
ANCMPT00226
RNCMPTD0236 36
RNCMPT90257
ANCMPTO02 58
RNCMPT00088
10
5
0.00
0.25
0.50
0.75
1.00
0.00
6.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
0.00
0.25
5.50
8.3
1.00
0.09
9.25
9.30
0.75
1.00
Scores
Figure G.2. Distribution of DecpCLIP scores for the second part of test samples (RNAcompete sample IDs on top). Scores of RNA-BAnC in green. Scores positive and negative experimental sequenees in bluc and red, respectively.
24


Reviews:
[{"review_id": "7oKmHuaJyC", "reviewer": "Reviewer_e4Br", "summary": "The paper introduces RNA-BAnG, a model for designing RNA sequences that bind to specific proteins, which does not require to be trained on extensive experimental data of RNA sequences known to interact with target proteins or detailed RNA structural information. The model combines a novel generative method. BAnG. with a transformer based architecture enhanced by geometric attention mechanisms that incorporate protein structural information. The authors base their approach around the observation that RNA sequences binding to proteins often contain functional binding motifs, making it more effective to anchor sequence generation around the motif (i.e. initiating from binding sites) rather than sequence ends. \n\nThe authors validate their approach first on synthetic tasks and show that the proposed method preserves motifs more effectively than conventional autoregressive and masked iterative generation approaches. They then evaluate the model on real biological RNA-protein interaction datasets, where again RNA-BAnG outperforms existing models in generating high affinity sequences and is shown to generate sequences of high diversity and novelty.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer’s thoughtful feedback and suggestions. We will add the necessary citations and work on further clarifying our methods and biological explanations to ensure the content is more accessible.\n\nIn response to the reviewer’s request, we have conducted **ablation studies** to assess the impact of non-traditional architecture choices, such as positional encodings in Cross Attention and Geometrical Attention. These studies have demonstrated the valuable contribution of these components to the model’s performance, and the results will be included in the Appendix for further clarity. \n\nWe agree that an **explanation of RNA-binding motifs** would strengthen the biological motivation for our approach. Aptamer RNA binding motifs are short, conserved sequence patterns that serve as recognition sites for RNA-binding proteins (RBPs). Many RBPs specifically recognize distinct nucleotide sequences, such as AU-rich or GU-rich elements, while others rely on secondary structures like hairpins and stem-loops for recognition, though these cases are relatively rare. We will incorporate a brief discussion of these motifs in the revised manuscript, referencing doi.org/10.1038/nature12311, which also provides the evaluation data used in our study.\n\nWe have additionally explored the **effects of protein data preprocessing on model performance**. Our observations indicate that segmenting the protein structure into domains or removing intrinsically disordered regions (IDRs) can improve generation results in certain cases. However, such preprocessing is only feasible if meta-information about protein-RNA interactions is available, as RNAs may interact with multiple domains or IDRs simultaneously. We will add this information into the revision.\n\nLastly, regarding **BAnG’s broader applicability**, we plan to extend its use to other biological tasks, such as protein-peptide interactions. However, this would require substantial additional effort and is beyond the scope of the current paper.\n\nWe greatly appreciate the reviewer’s valuable comments, which have helped us improve the clarity of our explanations. We hope the added details and planned additions will address the concerns raised by the reviewer and enhance the overall quality of our paper. As the requested revisions were focused on improving clarity, we hope the reviewer will consider increasing their score. If there are any additional specific changes or analyses you would like us to address, we are more than happy to make further revisions.", "claims_and_evidence": "The authors make several specific claims in the paper that warrant careful examination.\n\nFirst, they claim that their method is particularly suited for RNA sequence generation compared to other generation methods. based on, as mentioned above, the observation that RNA sequences binding to proteins contain localized functional binding motifs. This is strongly supported by synthetic experiments such as SingleBind and DoubleBind, where RNA-BAnG achieves significantly higher motif preservation (97% in DoubleBind) compared to autoregressive (53%) and iterative generation methods (4-5%). This provides robust empirical validation of the claim in controlled settings. Then, they claim that their method can generate RNA sequences that bind to specific proteins without requiring extensive experimental data of RNA sequences known to interact with target proteins or detailed RNA structural information. This is also strongly supported, since the model is trained only using protein sequence, and its structure as predicted from AlphaFold, without experimentally validated RNA binding sequences. Furthermore, they do some analysis that suggests that the model generalizes beyond the training proteins and has learnt something more fundamental. While the analysis provided is not in depth, this is compelling evidence.\nThe authors also claim that the geometric attention mechanism, which integrates protein structure information, is essential for their model to converge. However, they don't provide comparative experiments with ablations. This is a claim that would benefit from stronger evidence. finally, the authors claim that their method produces diverse and novel RNA sequences with high predicted binding affinity to target proteins. They provide quantitative metrics: high sequence diversity (0.93 ± 0.13) and novelty (0.99 ± 0.01), calculated as the proportion of generated sequences that do not appear in the training set. The results show that the model consistently generates sequences with higher predicted affinity scores than GenerRNA, and achieves superior performance to RNAFLOW across most test proteins (although the authors acknowledge possible methodological differences here). \n\nA notable limitation in the evidence is the reliance on DeepCLIP scores as a proxy for binding affinity rather than experimental validation. The authors acknowledge this limitation but demonstrate DeepCLIP's reliability by showing its clear discrimination between experimentally derived positive and negative binding sequences (area values of 0.88 vs. 0.11). This suggest that it could be a reasonable proxy for this evaluation, though as the authors acknowledge in the conclusion, experimental validation would strengthen the practical applicability of the generated sequences.", "methods_and_evaluation_criteria": "The paper introduces the BAnG generative approach and the RNA-BAnG model architecture. BAnG introduces a novel factorization of the joint distribution over an RNA sequence which enables bidirectional sequence generation from a central anchor point (rather than sequentially from sequence ends), which as explained before, exploits the biological reality that RNA sequences binding to proteins contain embedded functional motifs. Specifically, it defines two special anchor tokens that represent the left and right boundaries of the binding region, and generates sequence tokens alternately in both directions, extending outward from the anchors, using a custom bidirectional attention mask to correctly model dependencies. The RNA-BAnG architecture itself is an architecture with two modules reminiscent of a traditional transformer encoder decoder, with a few key changes. First there is a protein module that encodes protein sequence and structure, using standard self attention and a geometric attention based on the invariant point attention mechanism from AF2 to process the structure information. Then there is a nucleotide module which generates RNA sequences conditioned on protein representations. First, it encodes RNA sequences with self attention (with special handling for anchor tokes), and then it incorporates the protein representations from the protein module with a cross attention block. The training procedure consists of two phases (1) pretraining on non coding RNA sequences to learn general RNA properties, and (2) finetuning on protein-RNA interaction datasets to learn protein conditioned sequence generation.\n\nEvaluation: the model is evaluated across different tasks - synthetic motif reconstruction tasks (SingleBind and DoubleBind) and biological evaluation based on DeepCLIP.  \n\nOverall, there are no real comments to be made in this section. The methodology seems sound and well argumented. The evaluation criteria also seem well suited to this problem, with the obvious caveat that DeepCLIP is used as a proxy for binding affinity. The authors support this choice by showing that DeepCLIP reliably distinguishes positive and negative experimental binding sequences. The additional diversity and novelty metrics are critical since in RNA design it's useful to generate a broad range of candidates instead of just optimizing affinity. The benchmark datasets are appropriate for the task also. Indeed, the only thing that requires some attention would be the lack of experimental validation which the authors include as further work in the conclusion.", "theoretical_claims": "not applicable", "experimental_designs_or_analyses": "As explained previously, the experimental design is generally sound, with well-defined synthetic and biological tasks, and comprehensive baseline comparison. The synthetic experiments effectively isolate the impact of bidirectional generation on motif preservation, and comparisons with autoregressive and iterative generative models are appropriate. The biological experiments leverage real interaction datasets and AlphaFold2-derived protein structures to test RNA sequence generation. However, while the model avoids direct training on experimentally derived RNA-protein interaction data, it still depends on AlphaFold2 predictions, which may introduce biases.\n\nSome things to highlight: \n1. For DeepCLIP, the performance gap between positive and negative experimental sets (area values of 0.88 vs. 0.11) validates DeepCLIP as a reasonable proxy.\n2. For the comparison with GenerRNA, the authors match the test conditions by using the same proteins and filtering sequences to match length constraints. \n3. For RNAFLOW comparisons, they selected proteins with zero sequence similarity to the train set to ensure a fair assessment of generalization.\n\nThus, considerable efforts are made to set up fair comparisons in their experiments.", "relation_to_broader_scientific_literature": "The work connects a few different areas of the literature and extends recent advances in biomolecular design. The BAnG method itself represents an innovation in sequence generation that differs from standard autoregressive models like those commonly used in NLP, and from iterative generation methods like those employed in ESM3. The approach is particularly novel in its focus on bidirectional generation from a central anchoring point. The work relates to the broader literature on RNA-protein interactions, particularly studies focusing on the identification and characterization of RNA binding motifs. the learnings from which are directly incorporated into the model and training design. The geometric attention builds directly on invariant point attention from AF2 and the paper demonstrates its effectiveness in this domain.", "essential_references_not_discussed": "The paper briefly mentions RNA structure prediction tools but could reference specific methods like RNAFold or SPOT-RNA or more recent deep learning approaches for RNA structure prediction to provide context for why existing structural information might be insufficient. Also, the work on protein LMs that has become of increased relevance in the literature lately can give additional references for how to incorporate structure information into these attention based models. Finally, potentially some advances in multimodal approaches for biomolecular interactions could be discussed as relevant literature.", "other_strengths_and_weaknesses": "The paper is well written and the illustrations are very clear and helpful. There is lots of content for the reader. The code and model weights are open sourced.\n\nSome points to note that would make the paper stronger: \n1. The explanation of the BAnG method could benefit from a more intuitive description or an illustration before diving into the mathematical formulation, to help readers unfamiliar with sequence generation methods.\n2. The authors mention that RNA sequences often contain functional binding motifs embedded within larger sequence contexts but don't provide much detail on how these motifs typically manifest in RNA-protein interactions. A brief explanation of the typical characteristics of these motifs would strengthen the biological motivation for the approach.", "other_comments_or_suggestions": "not applicable", "questions_for_authors": "1. Are there certain protein structural features or binding patterns that affect generation in a certain way e.g. make it more challenging? \n2. BAnG seems potentially applicable to other biomolecular design problems. Have other applications/domains been considered e.g. PPI or small molecule design?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QTerDLj2oH", "reviewer": "Reviewer_W2v8", "summary": "This paper proposes RNA-BAnG, a deep learning framework for conditional RNA design, focusing on generating RNA sequences that can bind to specific proteins. The core contribution is the introduction of the Bidirectional Anchored Generation (BAnG) method, which generates RNA sequences bidirectionally starting from anchor tokens placed within functional binding motifs, rather than from sequence ends as in standard autoregressive approaches.\n\n## update after rebuttal\n\nThanks for your additional experiments. I would be happy to raise my score, and I suggest that the authors include these results in the camera-ready version.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer’s insightful comments. Let us now address the specific questions and concerns raised in the review regarding our evaluation choices and methodological decisions.\n\n**Expanding the comparison** to additional baseline models is challenging because most existing models (cited in the paper) rely on large training sets of RNA sequences known to interact with a specific protein to generate new interacting sequences. However, such datasets are often difficult to obtain or may not exist at all, making our model—which operates without this requirement—more versatile and practical. This same constraint also prevents us from expanding the GenerRNA comparison set. Regarding RNAFLOW, its assessment on proteins with no sequence similarity to our training data was intended to make the comparison more rigorous for our model and should not have negatively impacted RNAFLOW’s performance. The decision to restrict the comparison set in this way was due to RNAFLOW’s long generation time and its clear limitations in the given settings, particularly its inability to operate on non-truncated proteins. Since truncation is not technically possible—binding sites are unknown, just as they typically are in real-world applications—our evaluation setup naturally reflects this constraint.\n\nBoth **comparisons with baseline models** are quantified using statistical measures. Each sample consists of 1,000 generated RNA sequences, and we evaluate performance by calculating the proportion of sequences that achieve high affinity scores (section 4.4). This ensures a robust statistical basis for our comparisons and allows for a meaningful assessment of model performance. To provide a more quantitative comparison, we computed the proportion of sequences above a threshold, similarly to Figure 6, and calculated areas under threshold-dependent performance curves for both RNAFLOW and GenerRNA. The results further strengthen our claim and can be found [here]( https://imgur.com/a/dO2tVXx).\n\n**Evaluating generative models** is inherently challenging, as direct comparison with ground truth is not possible. Additionally, to the best of our knowledge, there are no established computational protocols for assessing RNA affinity to proteins. DeepCLIP is experimentally validated, does not rely on RNA structure (which is unavailable in our case), and has demonstrated strong performance in benchmark studies (doi:10.1093/bib/bbad307). To address evaluation challenges, we have added a more traditional approach by calculating sequence similarity using BLASTN. Results for this evaluation can be found [here](https://imgur.com/a/uuwVgxG).\n\nRegarding **BAnG’s broader applicability**, its extension to alternative biological tasks, such as protein-peptide interactions, is planned for future work as it requires substantial additional effort, particularly in data mining and preparation, making it beyond the scope of this paper.\n\nWhile the effectiveness of BAnG for multiple mutually exclusive motifs was verified with the DoubleBind task (Table 2), its performance when **multiple motifs** appear simultaneously in a sequence depends on the uncertainty of their relative positioning. If motif placement is completely random, BAnG will face similar challenges to autoregressive models. However, if relative positions are fixed, BAnG’s performance remains stable. These points are supported by the additional experiments we have conducted, their results can be found [here](https://imgur.com/a/LPugL4K). Ultimately, BAnG is designed to reduce uncertainty in the target probability distribution, and increased noise (uncertainty) in the data makes learning more difficult for any model.\n\n**Anchor point selection** plays a key role in minimizing uncertainty in the resulting factorization distribution. In synthetic tasks, any residue with a fixed distance from the synthetic motif could serve as an anchor point. However, in real-world scenarios, binding motifs vary in size and content, making random selection among interacting residues a practical approach to reduce uncertainty and diversify training data. In general, anchor point selection is driven by the goal of uncertainty reduction, but the optimal choice remains case-dependent.\n\nFinally, regarding decoding strategies, **ProteinMPNN’s sampling method** is equivalent to the Iterative Max Logit strategy tested in our manuscript. However, extending BAnG to allow decoding in any order contradicts its core probability factorization and is therefore not feasible.\n\nWe are grateful for the reviewer’s thoughtful feedback, which have helped clarify key aspects of our approach and provided valuable ideas for additional experiments and evaluation metrics. We hope our responses address the reviewer's concerns and the reviewer will consider increasing their score. If there are any additional specific changes or analyses you would like us to address, we are more than happy to make further revisions.", "claims_and_evidence": "Claim about outperforming RNAFLOW and GenerRNA broadly: While RNA-BAnG shows better performance in the provided case studies, RNAFLOW and GenerRNA were evaluated on a relatively limited test set. The RNAFLOW evaluation is constrained to proteins with no sequence similarity to the training data and without manually truncating proteins to binding regions, which might have impacted its performance.", "methods_and_evaluation_criteria": "The methodology and evaluation framework are well-aligned with the stated goals of conditional RNA design and provide a strong foundation for assessing model performance.", "theoretical_claims": "The paper does not present formal theoretical claims or proofs.", "experimental_designs_or_analyses": "The experimental design is largely sound and well-aligned with the goals of conditional RNA generation. The analyses are thorough, but the biological evaluation would be further strengthened with experimental validations or a broader baseline comparison scope.", "relation_to_broader_scientific_literature": "By training the model to design RNA starting from the binding area, BAnG outperforms other models. The findings may also be useful for other bio-molecule designs.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "Strengths:\n- The proposed BAnG outperforms previous works significantly.\n- The idea of generating the import area of RNA first makes sense.\nWeaknesses:\n- The proposed BAnG is only compared with RNAFLOW on several samples. A statistical metric comparison is required, e.g. binding affinity distribution.\n- The effect of anchor point position selection is not discussed.", "other_comments_or_suggestions": "- >\"Importantly, the method’s design makes it applicable beyond RNA-protein interactions, extending to any scenario where the focus is on optimizing functional subsequences within a larger sequence.\"\nI suggest the authors conduct more experiments on protein-protein or other generation tasks to demonstrate the application\n- The authors use DeepCLIP to access the RNA-protein binding affinity. Can any other models or traditional computational approaches be used? I suggest the authors use more methods to access the binding affinity since there is not a commonly used model to access the affinity.\n- I will raise my rating if more evaluation and comparison is conducted.", "questions_for_authors": "- The bidirectional direction decoding approach is similar to ProteinMPNN. Can this method be extended to decode in any order, like proteinMPNN? \n- Besides, BAnG should be effective for one motif. But how about multiple motifs? The important areas are also multiple.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RNtnInkMzg", "reviewer": "Reviewer_KUjk", "summary": "This manuscript presents RNA-BAnG, a deep learning model for generating RNA sequences that bind to specific proteins. The method involves a novel Bidirectional Anchored Generation (BAnG) technique, which generates RNA sequences by conditioning on protein sequence and structure. RNA-BAnG utilizes geometric attention to incorporate protein structural information, enabling effective RNA sequence design without requiring experimental data for the target protein. The model is validated on synthetic tasks and experimental RNA-protein interaction data, demonstrating superior performance compared to existing methods.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your helpful feedback. We appreciate your suggestions and have made adjustments accordingly. Below, we address the points you raised regarding our methods and analysis. Additionally, we are exploring ways to provide a more detailed **analysis of the binding motifs**, as suggested, to further improve the clarity and depth of our evaluation.\n\nIn response to your suggestion, we have conducted **ablation studies** to evaluate the impact of non-traditional architecture choices, such as positional encodings in Cross Attention and Geometrical Attention. These studies have demonstrated the significant contribution of these components to the model’s performance, and the results will be included in the Appendix of the revised manuscript for further clarity.\n\nWe hope that these changes address the concerns raised and improve the overall quality of our manuscript. Thank you again for your insightful comments and suggestions. If there are any additional specific changes or analyses you would like us to address, we are more than happy to make further revisions.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "Yes", "experimental_designs_or_analyses": "Yes", "relation_to_broader_scientific_literature": "All good", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "This manuscript has several notable strengths:\n\n1.Introducing RNA-BAnG, a novel deep learning-based model that generates RNA sequences for protein interactions without requiring experimental data or RNA structural information, making it widely applicable in various biological contexts.\n2.Innovative BAnG method, which allows RNA sequence generation by conditioning on protein sequence and structure, addressing the challenge of protein-RNA interaction design more efficiently than existing approaches.\n3.Demonstrating strong performance through comprehensive validation on synthetic tasks and experimental data, showing that RNA-BAnG outperforms other sequence generation methods in terms of both novelty and binding affinity.\n\nWeaknesses:\n\n1.Although the manuscript demonstrates the superiority of RNA-BAnG by comparing its performance with different methods, it does not directly evaluate the impact of the model's different modules or mechanisms through ablation experiments. It would be beneficial to include ablation studies to assess the specific impact of each component on the model's generation performance.\n2.While the sequences generated by the model show good performance in terms of DeepCLIP scores, the actual biological functions and mechanisms of these binding motifs have not been thoroughly validated. It is recommended that the authors include a detailed analysis of the binding motifs in the manuscript to enhance the biological interpretability of the model.\n3.In the section “2.1. Description of the BAnG generative approach”, it is suggested to change the reference format for the equations to \"Eq. (1)\".", "other_comments_or_suggestions": "Same to Other Strengths And Weaknesses", "questions_for_authors": "Same to Other Strengths And Weaknesses", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "No."}}, {"review_id": "xlvl0dcgrV", "reviewer": "Reviewer_o8nd", "summary": "The paper proposes a generative model which takes as input a protein structure and generates a potential RNA binder to that protein. The main methods contribution is a modification to usual left to right autoregressive generation which better fits the given task.\n\n---\n\n# Post-rebuttal, based on authors' last comment\n\n> FID scores in image generation, refoldability metrics in protein design\n\nFID is based on Inception (top model on ImageNet at the time) and re-foldability is based on AlphaFold2 (Nobel Prize). In my opinion, individually (per test set sample) trained DeepCLIP models are not similar/analogous to these models used in other ML domains. I'm not convinced that this manner of evaluation is appropriate or meaningful.\n\n> using RNA structure: we are confused by what the reviewer finds “not sufficiently convincing”\n\nI'm just not convinced by claims and statements in the main paper PDF which say that the method does not use structural data.\n\nI wrote previously: \"Introduction claims that the proposed method does not rely on RNA structural data. Yes, that's technically true once the model is trained, but all the training data requires using RNA structures from the PDB based on my understanding, so this claim is not supported.\"\n\nAnd the authors wrote back confirming my understanding to be correct - they do need the PDB and 3D structures in order to prepare their training datasets, which I think is one of the most important parts of applied ML models. So the claims are misleading in my opinion.\n\n> While we are happy to elaborate on data processing and other procedural details, the emphasis appears to have been placed on peripheral rather than essential elements. \n\nI think evaluation metrics and training datasets are super important details of applied ML papers for scientific domains.\n\n> Additionally, some comments seem to focus more on wording than on technical substance. Although we provided detailed and direct responses to all raised points, these were not reflected in the final evaluation, as the original score was maintained.\n\nI think one of our major jobs as reviewers is to carefully scrutinize the major claims that the authors make, which includes details/nuances of how they are worded. That's what I am doing in my review.\n\n> Based on the points outlined above, we believe the current score is not well supported by the substance of the review and does not accurately reflect the core contributions of the paper, particularly in the context of machine learning methods for sequence generation. We hope the reviewer might reconsider their evaluation in light of the clarifications and arguments provided.\n\nI believe the score reflects my assessment of the paper at present. I am sorry if the authors are disappointed by this.", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your feedback and for bringing attention to key aspects of our study. Below, we address your concerns and clarify key points regarding our methodology and evaluation.\n\n**RNA structural information** from PDB was used only during data preprocessing to identify interacting nucleotides; however, it was not incorporated into the training process. Our model does not require RNA structure information for inference, nor does it internally predict or rely on it in any way. In contrast, some other methods depend on structural data for training, inference, or both, making our approach more broadly applicable, especially in cases where such information is unavailable. We acknowledge that certain explanations may have been challenging to interpret and will take this into account in our text revisions.\n\nFor training, we filtered proteins to include only those shorter than 500 residues, while RNA and DNA sequences were not subject to length restrictions (Appendix C.1). Consequently, proteins in our dataset tend to be shorter. Nucleotide sequences were cropped during training, leading to an effective training length of 205±114 residues, primarily to reduce computational memory requirements. Long RNA sequences are typically derived from the ribosome. Given that our ultimate goal is to generate aptamers—short RNA sequences—the cropping is appropriate, as our model doesn't need to learn long-range dependencies. We acknowledge that a **sequence-based split** is not ideal; however, it was applied solely during training to enhance data diversity and facilitate optimal weight selection, rather than to support any specific claims (Appendix C.1). Our test set contains proteins with less than 25% sequence similarity to the training data (Appendix F). Given that proteins with such low similarity are estimated to have less than a 10% chance of sharing a similar fold (doi.org/10.1093/protein/12.2.85), this suggests that our model demonstrates a degree of generalization across both structural and sequential spaces.\n\n**Evaluating generative models** is inherently challenging, as direct comparison with ground truth is not possible. Additionally, to the best of our knowledge, there are no established computational protocols for assessing RNA affinity to proteins. Given these challenges, we opted to use a deep learning method for scoring, similar to practices in other domains like computer vision. Among available methods, we selected DeepCLIP due to several key advantages: it has undergone experimental validation, is retrainable for individual proteins, has demonstrated strong performance in benchmark studies (doi:10.1093/bib/bbad307), and does not rely on RNA structural information. We took extensive measures to verify its suitability (detailed in Appendix E.1), including rigorous training and testing of DeepCLIP models for each evaluation sample and selecting only those with exceptionally high performance metrics (AUROC > 0.95). While the reviewer suggests that such models may have a tendency for false positives and negatives, this would only imply that RNA-BAnG performs even better than our current estimates indicate. Nonetheless, we ensured that our DeepCLIP training and testing datasets contained an equal number of positive and negative samples to mitigate any potential biases (Appendix E.1).\n\nThe claim that **aptamer binding motifs** are the primary determinants of protein-RNA interactions, while the rest of the RNA sequence plays a lesser role, is supported by the findings of a prior study (doi.org/10.1038/nature12311). This paper, which provided the experimental data used in our evaluation, also states that RNA structure is not a critical factor in protein interactions for most of the analyzed samples.\n\nTargeting specific **protein binding sites** with our model is currently possible through protein truncation. We did not incorporate a more direct method, since the primary protein-aptamer interaction experiment, SELEX, is not binding site specific as well. It has not been a priority for the current study, but it could be explored in our future work.\n\nWe appreciate the reviewer’s insights, which have helped us refine our explanations. We will also ensure that our model's limitations are properly discussed in the conclusion. Given the additional clarifications provided, we hope the reviewer will consider reassessing their score. If there are additional aspects you would like us to address or specific analyses you would like to see, we are open to further suggestions.", "claims_and_evidence": "I felt like I disagreed or did not find myself being sufficiently convinced by several claims here:\n\nIntroduction claims that the proposed method does not rely on RNA structural data. Yes, that's technically true once the model is trained, but all the training data requires using RNA structures from the PDB based on my understanding, so this claim is not supported.\n\nThere is the related claim (repeated throghout) that all other methods other than this paper require structures as inputs. The conclusion states \"eliminating the need for extensive structural or interaction data...innovation significantly broadens the applicability of our method\" -- but yet again, the model is trained based on PDB structural data on interacting proteins and RNAs, so I find this statement confusing.\n\nNext, the claim (repeated throughout) that aptamer binding motifs are the only thing important for protein-RNA interactions and the rest of the RNA sequence does not matter/is less important -- this claim is presented as if universally true -- but firstly this should be supported by some sort of citation from basic science, and if this is not the case universally, some caveats should be presented here.\n\nIntroduction ends with a claim that experimental RNA-protein interaction data is used for evaluation. I am not sure that's what happens, as all the evaluation uses another ML model (DeepCLIP), trained on experimental data, to compute in-silico evaluation metrics. So the phrasing of this claim feels vague and imprecise. On a related note, all performance-related claims in the abstract/introduction feel purposely not quantitative (eg. simply saying \"shows promising results\", \"outperform previous methods\").", "methods_and_evaluation_criteria": "I don't think that the data is prepared in a sufficiently rigorous manner/I have questions I'd like clarified:\n\nData is split based on protein sequence homology. However, there can be a few known issues with such a split when working on biomolecule interactions. Firstly, while overall homology may be low, there may be cases where the homology of the interacting residues/positions is still high - and it seems like best practice to prioritise interacting positions instead when preparing splits. Additionally, structural homology may also be highly relevant here as you define interactions from a structural perspective/rely on the PDB. There have been several papers now pointing to such issues in biomolecule interaction data splits, and this one is perhaps the most recent resource: https://www.pinder.sh/\n\nThe appendix states the following about the data: \"... protein mean length of 155 residues with a standard deviation of 90; RNA lengths average 1,834 nucleotides (± 1,564); DNA lengths average 76 nucleotides (± 78).\" \n\nCan you provide further information as to why the proteins have shorter lengths than RNAs in your dataset? Are many of these interactions from the ribosome? Are these super long RNAs from the ribosome?\n\nNext, its stated: \"To prevent potential computational resource issues and to focus the model on the binding motifs, we truncated nucleotide\nsequences exceeding 300 residues during training and validation.\"\n\nWhy do you think this is a biologically sensible choice/is there a scientific justification to doing this, or is it a purely ML-driven decision?", "theoretical_claims": "Not applicable.", "experimental_designs_or_analyses": "I feel very skeptical of the evaluation metric used here -- another machine learning model which is trained to predict protein-RNA binding affinity -- my intuition from reading literature is that such models almost never generalize to new datapoints beyond what's close to their training set, as they don't really learn the physics of protein-RNA interaction but are rather based on co-occurence statistics. Additionally, these models performance often looks good when presenting aggregate numbers (avg. across a test set), but in reality they often become very good at ranking/predicting for poor binders/negative fitness, while being almost random for the very, very few positive binders/positive fitness that are present in the train as well as test data. However, practically, we mostly care about their performance at identifying the positives, which they don't really tend to do in my experience.\n\nI would really like the authors to provide significantly more justification of this most important experimental design choice rather than just referencing another recent paper.", "relation_to_broader_scientific_literature": "I think the paper is contributing to an area with growing recent interest. Generally designing biomolecular interactions and specifically RNA aptamers are a growing topic. But I think the paper and broader community need to do a far better job of dataset and evaluations being rigorous and biologically relevant.", "essential_references_not_discussed": "Perhaps discussing recent advances in 3D and pseudoknotted RNA inverse design, eg. gRNAde (ICLR'25), RhoDesign (Nature Computational Science). I think a lot of interactions with binding partners are mediated via complex RNA tertiary structural elements like pseudoknots - and it may be worth connecting to how our ability to design RNA from a tertiary perspective is also improving (and is perhaps complementary to this work which takes a more implicit sequence-only approach).", "other_strengths_and_weaknesses": "The synthetic experiments were well designed to highlight why the methodological contribution in interesting. I liked how they eased the reader into the much more complex experiments afterwards and how they connected with the methods section.\n\nThe paper should really be discussing the limitations of the method in more depth. The conclusion reads a bit too positive and makes no attempt to provide caveats (this is a matter of taste/opinion, but I think the study has some limitations esp. around how the data has been prepared and what evaluation metrics are being used).", "other_comments_or_suggestions": "Generally, we may want to target specific epitopes/binding sites on a protein. It may be worth discussing how the current method is limited in that respect.", "questions_for_authors": "Stated in the rest of the review.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iSu4AxVGab", "reviewer": "Reviewer_AqSw", "summary": "This paper proposes a deep learning-based model, RNA-BAnG, for designing RNA sequences that interact with specific proteins without requiring extensive experimental data or structural knowledge. Its core innovation, Bidirectional Anchored Generation, exploits the presence of functional binding motifs within broader sequence contexts. The model is validated on synthetic tasks with localized motifs and biological sequences to demonstrate its effectiveness for conditional RNA sequence design.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the reviewer’s thoughtful comments and the ideas provided for additional experiments. \n\nWe understand that the **synthetic task** serves as a controlled environment to test the model’s performance, and we agree that the simplifications involved may not fully reflect the complexity of real-world scenarios. However, our intention was to provide a baseline evaluation to better understand the model’s behavior in a controlled setting before applying it to more complex, real-world tasks. We will clarify this in the revised manuscript to ensure that the purpose of the synthetic task is better understood.\n\nThe length of 50 was selected, as our main application goal is the design of aptamers, which are typically short (20-100 bases). While **experiments with longer sequences** will likely degrade the performance of other methods, this will not affect BAnG. ROPE (or an alternative positional encoding) is required to incorporate nucleotide order and relative distance in chain information. Furthermore, while **variable sequence length** may negatively impact the performance of other methods, it should have no influence on BAnG. These points are supported by the additional experiments we have conducted, the results will be provided in the revised manuscript and can be found [here](https://imgur.com/a/LPugL4K).\n\nWe are grateful for the reviewer’s suggestions, which have helped guide us in exploring these additional experiments. We hope the clarifications and new results will address the concerns raised and the reviewer will consider increasing their score. If there are any additional specific changes or analyses you would like us to address, we are more than happy to make further revisions.", "claims_and_evidence": "- Authors provided supporting experimental evidence to support their major claims.\n\n- The method is indeed novel. Specifically, the factorization of the joint distribution in Eqn. 1 and how the attention mask has been designed to execute the training steps in one-go is quite interesting. The authors claim on this has been properly justified in Section 2.1 \n\n- My main concern is regarding the synthetic task for evaluation and comparison with other methods. The synthetic task designed here is interesting, however it is not clear if doing better on this task will directly transfer to more complex real-world tasks.\n\n- In the synthetic task, only sequences of length 50 are considered, which is quite short. Often sequence generative models perform much worse with longer sequences. I do not see any benefits of usig ROPE in this case since all the sequences are quite short.\n\n- It seems like all the sequences are 50 residues long (as mentioned in Section 3: \"The synthetic data consists of nucleotide sequences, each 50 residues long\"). If that is the case, then this is also a point of concern. Real-world sequences have a high variability of sequence lengths, which affect the evaluation of model's generalizability.", "methods_and_evaluation_criteria": "The method has novelty and the evaluation criteria were chosen properly for the application at hand.", "theoretical_claims": "The authors introduce a factorization of the joint distribution in Eqn. 1 and a way to execute that with a particular attention masking strategy, that enables parallelization during training similar to autoregression. This is correctly provided and discussed properly.", "experimental_designs_or_analyses": "The experimental design and analyses in this paper are mostly valid. However, the evaluation on synthetic data has some limitations as discussed above.", "relation_to_broader_scientific_literature": "The method introduced in this paper is related to the RNA sequence design literature as well as machine learning on RNA molecular data.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "I appreciate that the authors provided anonymized code for their method.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper has a wide range of reviews, from five reviewers; one "reject", three "weak accept", one "accept". Two reviewers raised their scores following the rebuttal period, during which the authors provided additional ablation experiments that strengthen the paper.

The reviewer arguing to reject had some concerns, which I do think are quite valid and shouldn't be dismissed. One of these could largely be addressed by writing; RNA structure data is not an input to the model, but is required as part of preparation of the training data and the paper should be honest about this. 

The second concern is regarding evaluation using DeepCLIP. I think this is a fair criticism, and it was shared by some of the other reviewers as well: unlike some of the other settings the authors cite, where e.g. AlphaFold has been well shown to provide good performance and generalisation in a wide variety of contexts, the DeepCLIP models are trained on only a handful of binding partners and as such we do not know or necessarily expect them to be accurate in predicting binding for the (possibly o.o.d.) samples from the proposed model.

As such, I think it is fair to say that the DeepCLIP evaluation metric is unreliable here, or at least unproven. However, I would still recommend acceptance, given the novelty of the method and its appropriateness to the problem at hand, and the fact that the majority of the reviewers are happy to accept the paper despite these issues. Additionally, there was not a clear alternative computational approach suggested by the reviewers that would be an appropriate alternative.
Decision: Accept (poster)