KinPFN: Bayesian Approximation of RNA Folding Kinetics using Prior-Data Fitted Networks
Authors: Dominik Scheuer
Frederic Runge
Jörg K.H. Franke
Michael T. Wolfinger
Christoph Flamm
Frank Hutter
Venue: ICLR 2025 Poster (2025)
Keywords: RNA Folding Kinetics
Prior-Data Fitted Networks
Deep Learning
Synthetic Data
Transformer
Bayesian Inference
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: We use a synthetic prior of RNA folding time distributions to train a prior-data fitted network to accurately approximate RNA folding time distributions in a single forward pass, given only a small set of initial folding time examples.

Abstract: RNA is a dynamic biomolecule crucial for cellular regulation, with its function largely determined by its folding into complex structures, while misfolding can lead to multifaceted biological sequelae. During the folding process, RNA traverses through a series of intermediate structural states, with each transition occurring at variable rates that collectively influence the time required to reach the functional form. Understanding these folding kinetics is vital for predicting RNA behavior and optimizing applications in synthetic biology and drug discovery. While in silico kinetic RNA folding simulators are often computationally intensive and time-consuming, accurate approximations of the folding times can already be very informative to assess the efficiency of the folding process. In this work, we present KinPFN, a novel approach that leverages prior-data fitted networks to directly model the posterior predictive distribution of RNA folding times. By training on synthetic data representing arbitrary prior folding times, KinPFN efficiently approximates the cumulative distribution function of RNA folding times in a single forward pass, given only a few initial folding time examples. Our method offers a modular extension to existing RNA kinetics algorithms, promising significant computational speed-ups orders of magnitude faster, while achieving comparable results. We showcase the effectiveness of KinPFN through extensive evaluations and real-world case studies, demonstrating its potential for RNA folding kinetics analysis, its practical relevance, and generalization to other biological data.

Published as a conference paper at ICLR 2025
KINPFN: BAYESIAN APPROXIMATION OF RNA FOLD- ING KINETICS USING PRIOR-DATA FITTED NETWORKS
Dominik Scheuer' Frederic Runge' Jörg K.H. Franke'
Michael T. Wolfinger Christoph Flamm² 2 Frank Hutter
'University of Freiburg, Germany University of Vienna, Austria
aRNA Forecast e.U., Vienna, Austria *ELLIS Institute Tübingen, Germany Correspondence to rungetêcs ceciessc.ii.ii wnwuunii-eeeeee cii--frebbuue de or dom. scheuer@gmail com.
ABSTRACT
RNA is a dynamic biomolecule crucial for cellular regulation, with its function largely determined by its folding into complex structures, while misfolding can lead to multifaceted biological sequelae. During the folding process, RNA tra- verses through a series of intermediate structural states, with each transition occur- ring at variable rates that collectively influence the time required to reach the func- tional form. Understanding these folding kinetics is vital for predicting RNA be- havior and optimizing applications in synthetic biology and drug discovery. While silico kinetic RNA folding simulators are often computationally intensive and tome-consuming, accurate approximations of the folding times already be very informative to assess the efficiency of the folding process. In this work, we present KinPFN, novel approach that leverages prior-data fitted networks directly model the posterior predictive distribution of RNA folding times. By training on synthetic data representing arbitrary prior folding times, KinPFN efficiently ap- proximates the cumulative distribution function of RNA folding times in single forward pass, given only a few initial folding time examples. Our method offers a modular extension to existing RNA kinetics algorithms, promising significant computational speed-ups orders of magnitude faster, while achieving comparable results. We showcase the effectiveness of KinPFN through extensive evaluations and real-world case studies, demonstrating its potential for RNA folding kinetics analysis, its practical relevance, and generalization to other biological data.
INTRODUCTION
Ribonucleic acid (RNA) plays a pivotal role in various biological processes, serving as a crucial intermediary between DNA and proteins while exerting significant regulatory functions through di- verse mechanisms (uu 2014). Composed of four nucleotides Adenine (A), Cytosine (C), Guanine (G), and Uracil (U) the functionality of RNA is closely tied to its structure (Lodish et al. 2005):
An RNA molecule adopts one or more native conformations that are essential for its biological ac- tivity (Fang et al. 2015). The dynamic process of how RNAs acquire their functional structure is known as the kinetic folding of RNA. During this process, the RNA strand transitions through sev- eral intermediate structural states, driven by intra-molecular interactions (Flamm et al.; 2000 Yu ct al. 2018). Since misfolding can lead to significant dysfunctions and diseases (Conlon & Manley 2017 the study of RNA folding kinetics is highly relevant for biomedical applications.
An important aspect of folding dynamics is the study of the rates and pathways through which RNA molecules achieve their native structures (Chen 2008). A common measure to quantify these processes are first passage times (FPTs), i.c. the time required to acquire certain structure for the first time, and their cumulative distribution functions (CDFs) (Flamm et al. 2000 Wollinger et al., 2004). These functions are derived from extensive simulations, requiring thousands of folding iterations to capture the probabilistic behavior of RNA molecules. While essential for understanding
RNA dynamics, calculating FPT CDFs is computationally expensive (Wolfinger et al.) 2004 Badell]
ct al. 2023), posing a significant barrier to real-time applications such as kinetic RNA design, which
"Equal Contribution.

---

Published as a conference paper at ICLR 2025
critical for drug discovery. While deep learning methods could improve the state of the art in RNA folding (Fu et al. 2022 Franke et al., 2024) and RNA design (Runge et al. 2024 Patil et al. 2024). they are not yet used in modeling RNA kinetics.
In this work, we present KinPFN, a novel deep learning-based approach that dramatically accel- erates the computation of RNA first passage times. KinPFN leverages prior-data fitted networks (PFNs) (Müller et al., 2022) trained on synthetic datasets of RNA folding times to predict the en- tire CDF of folding times from just few context examples fast and accurate distribution approximations, KinPFN can be integrated with existing RNA kinetics simulators, offering comparable performance at a fraction of the computational cost. These speed- ups make KinPFN a valuable tool for the study of RNA folding kinetics, offering novel routes for applications in kinetic RNA design, which was previously intractable due to exponential runtimes of kinetic folding simulators, and promising fast analysis of RNA folding behaviors across multiple applications in drug discovery, medicine, biotechnology and synthetic biology.
single forward pass. By providing
Our main contributions are summarized as follows:
We propose new synthetic prior to sample datasets of RNA folding times. We use this syn- thetic data to train prior-data fitted network to learn to predict the distribution of RNA first passage times, conditioned on a small set of context examples (Section 4.1).
We introduce KinPFN, a new deep learning model for RNA kinetics. KinPPN provides accurate predictions of RNA first passage time distributions, accelerating kinetic simulations by orders of magnitude (Section 4.2)
We evaluate KinPFN's performance synthetic and real-world RNA data (Section 5 demon- strating its practical utility through two case studies: an analysis of eukaryotic RNAs (Sec- tion (5.2p and a study RNA folding efficiency (Section[5.3).
In addition to its application to RNA folding kinetics, we assess KinPFN's ability to generalize to different biological data sources by approximating gene expression data obtained from previous smFISH (Femino et al. 1998 Raj et al., (2008) wet-lab analysis (Bagnall et al. 2020). demonstrating its potential to accelerate experimental protocols (Section 5.4).
We provide an overview of KinPFN in Figure [ publicly available at htttps://githbb.comt http:///..c.ooomummm//////raaa
Our source code, data, and trained models are
Training on Synthetic Prior
Synthotic Datasets Drawn from Multi-Modal Gaussians to Mimic RNA Folding Time Distributions
KinPEN Leverages In-Context Learning to Approximate Given Distribution from Few Example "Times"
KiPPFN is Trained on -5M Distributions by Minimizing the Negative Log-Likelihood
KinPFN
NLL
Context
Riemaun Distribution
b Application
RNA Kinetirs Simulators are Costly Scale Exponentially with the RNA Length
KinPFN Approxinates the Distribution Folding Times Using Kinetics Simulatious data Coutext
KinPFN Achieves Comparable
Results While Reducing
Simulator Costs by 95%
Simulator Data KinPEN Approximation
Runtime
Runtime
Length
Folding
Time
Simulator KinPFN
Figure/ Graphical abstract. a: KinPFN is trained on synthetic RNA folding time distributions drawn from parameterized multi- Gaussians by minimizing the negative logelikelihood (NLL). b: KinPFN accelerates RNA kinetics simulators by predicting the RNA folding time distribution in single forward pass, given few folding times as context.
2

---

Published as a conference paper at ICLR 2025
2 BACKGROUND
RNA First Passage Times Kinetic RNA folding is typically approximated by Monte-Carlo sim- ulation techniques (Flamm & Hofacker 2008). However, this is computationally expensive since enough stochastic simulations need to be accumulated to get a statistically representative time evo- lution of the state probabilities. Depending on the number of different structural states, which is typically huge, the path during folding, and the energy barriers between the states, the time to reach a certain structure for the first time, i.e., the first passage time, can differ across multiple kinetic sim- ulations for a given RNA. By comparing the first passage time CDFs of different RNA molecules or under varying conditions, differences the folding dynamics can be revealed and better under- stood. This comparison provides insights into the efficiency and stability of the different folding processes, examining the impact of various modifications, such as chemical alternations or evolu- tionary changes (Flamm et al. 2000). In this work, we use the term folding time synonym for first passage time and focus on RNA folding kinetics based on secondary structure information.
Prior-Data Fitted Networks Prior-data fitted networks (PFNs) (Müller et al. 2022) use transformer- bassed model to perform approximate Bayesian inference. PFNs are trained to predict an output y E R, conditioned on an input and a training set Durain of input-output pairs. Dur- ing training, these samples are drawn from a prior distribution over datasets p(D), optimizing the Cross-Entropy loss for a PFN qo with parameters 0,
la = ((z.))))Puiia (pD) |- log go(y) 22, D.
(1)
for predicting the label y. given x and Durain- As shown by Müller et al. 2022). this approach directly minimizes the Kullback-Leibler (KL) divergence between the prediction of the PFN and the true posterior predictive distribution when training on many samples of the form (x. y) U Dirain In this work, we adapt this strategy to tackle the prediction of RNA first passage time distributions, accounting for the specific challenges of the probabilistic behavior of RNA molecules that is also reflected in kinetic simulators by renouncing quantile information. For more details, see Section
RELATED WORK
In silico analysis of RNA folding kinetics can be divided into nucleotide-resolution and coarse- grained approaches. While the first yields a high level of simulation details, the latter typically allows studying larger systems, i.e. longer RNA chain lengths. The first publicly available tool for comput- ing RNA folding kinetics at nucleotide resolution is Kinfold (Flamm et al. 2000). Markov-chain Monte Carlo (MCMC) method that is still considered one of the most accurate approaches avail- able (Fukunaga & Hamada 2019). This accuracy. however, comes at the cost of runtime as Kinfold MCMC simulations typically require large number of trajectories to obtain reliable results. While it is possible to simulate the folding kinetics of RNA chains of several hundreds of nucleotides, such calculations require substantial compute (Fukunaga & Hamada 2019). This limitation inspired ac- celerating techniques like memoization and parallelization (Aviram al. (0122). shorteuts for the energy calculations of RNA secondary structures as implemented in Kfold (Dykeman) 2015). In con- trast, we develop KinPFN speed up every kinetic simulator that produces first passage times.
extension to existing kinetic RNA folding simulators massively
An Alternative to KinPFN are probabilistic density estimators like kernel density estimation (KDE) (Bishop) 2006). Gaussian Mixture Models (GMM) (Bishop) (006) or Bayesian Gaussian Mixture Models, also known as Dirichlet Process GMMs (DP-GMM), which utilize Variational Bayesian estimation of Gaussian mixtures (Blei & Jordan 2006). Similar to KinPFN, GMM and DP-GMM aim to model the posterior predictive distribution as a multi-modal Gaussian distribution. While GMMs struggle with complex data structures, especially when the number of modes is un- known, Bayesian approaches like DP-GMM can dynamically adjust the number of mixture compo- nents (McLachlan et al. 2019 Neal 2000). Alternatively, kernel density estimation (KDE) offers a non-parametric approach by estimating probability densities through the summation of kernels, like Gaussians, over data points (Bishop) 2006). From a deep learning perspective, methods based on normalizing flows (Rezende & Mohamed] 2015). variational autoencoders (VAEs) (Kingmal 2013), or a probabilistic transformer as proposed in Franke et al. (2022), would be well suited for proba- bility density estimation of RNA folding kinetics. However, these methods typically require large amounts of training data which is not available for RNA folding kinetics. Instead, we approach the
3

---

Published as a conference paper at ICLR 2025
problem of folding time prediction using synthetic prior to train PFN for direct approximation of the CDF of folding time distributions.
While, to the best of our knowledge, KinPFN is the first deep learning approach for RNA folding ki- netics, PFNs were previously applied to multiple problems like few shot image classification Muüller
al. )022). classification for small tabular datasets (Müller et al. 2022 Hollmann et al. 2023
trapolation of learning curves (Adriaensen et al. (023), Bayesian optimization and hyperparameter
optimization (Müller et al. 2023] Rakotoarison et al. 2024), and time series forecasting (Dooley
et al. 2024).
For more discussions on related work, please see Appendix
4 APPROXIMATION OF RNA FOLDING TIME DISTRIBUTIONS
We consider the problem of learning the posterior predictive distribution (PPD) of first passage times for an RNA molecule & E (A, G, C, Up of length I, conditioned on a small set of initial examples, to approximate the cumulative distribution function (CDF). Formally, the first passage time is the time required for the RNA & to fold from an initial structure Wistatt into a stop structure whsop while transitioning through arbitrary intermediate structural states. Running M folding simulations under the same conditions (for RNA sequence 6, Wstart and wstop) yields distinct first passage times i L.. By aggregating these times, we compute the fraction of molecules e folded by time T', denoted (T), where F. (T) = P(I S T) represents the CDF of the stochastic variable 1.
The problem we consider in this work can be formulated as follows: Given N < M observed first passage times f1,...; fvy and prior distribution over first passage times from which we can generate samples, aim to approximate the PPD q(t .t....,tN) With approximated PPD, can compute the predicted CDF (T)), which approximates the true CDF (T))); the fraction of molecules folded by time T.
In the following sections, we describe approach define synthetic prior of first passage time distributions that allows us to approximate the PPD of folding times (Section 4.1) and explain the development of KinPPN in detail (Section 4.2
4.1 SYNTHETIC PRIOR FOR RNA FOLDING TIME DISTRIBUTIONS
Obtaining large amounts of prior RNA kinetics data to train a deep learning model, particularly for longer RNAs, currently infeasible due to the exponential runtime of accurate kinetic simulators (see Figure Appendix B). This hinders us from using traditional Bayesian approaches for the ap- proximation of RNA first passage times, e.g., by training a variational autoencoder (VAE) (Kingmal 2013). Therefore, we take an alternative approach, training a PFN solely on synthetic prior of RNA first passage time distributions. However, developing a synthetic prior for molecular problems is challenging since it seems impossible to generate meaningful synthetic combinations of molecule features with posterior information from a process depending on these features. We. therefore, de- velop KinPFN independent of molecular features and restrict its input to first passage times only. This offers the advantage that we can apply KinPFN to predict first passage time distributions at test time, independent of the underlying eittagenerating process.
For the development of our synthetic FPT prior, we leverage the observation that RNA first pas- sage time distributions often exhibit CDFs with regions of slower growth interspersed with steeper transitions, leading to distinct plateaus and multiple changes between convex and concave sections representing inefficiencies in the corresponding folding pathway (Flamm al.) 2000, Wolfinger et al. 2004).. These patterns make multi-modal distributions a natural choice to model the complex- ity of such processes synthetically, as they are designed to capture data with multiple local maxima or modes (Hartigan & Hartigan| 1985). We thus construct a prior distribution over RNA first pas- sage times as a family of multi-modal Gaussian distributions ( (P | E (2, 4, 5), v/ E ) Each multi-modal distribution in this family comprises & Gaussian components, each character- ized by its own mean (4; and standard deviation o,, = 1,...,k. k. The parameter space V thus defines the family of distributions, with each specific distribution parameterized by vector 2/x = ((u, a.), (Mz oz).... . (Jlg, ox)) within gg. We illustrate a synthetic bi-modal PDF along- side its corresponding CDF and examples of synthetic first passage time CDFs in Figure g
4

---

Published as a conference paper at ICLR 2025
Since cannot make any further assumptions about the distribution of folding times, especially when generating synthetic data, I and y of a prior distribution p(ta)) are considered completely independent. Consequently, we decide to assign a value of zero to all variables x, representing no prior information, while the variables are ultimately sampled from the aforementioned multi- modal distributions. As the targets y represent synthetic first passage times, they will be referred to as & from this point forward. We set the range of possible first passage time values ~ p(tix) to [10-6, 6, _103], J. range that covers a large fraction of possible folding processes based on observations from preliminary kinetic simulations. To mimic realistic first passage time distributions, we choose bounded uniform base means hasse U(-5,16) and uniformly distributed standard deviations 0; ~ 100.1,,47 4.2) based on preliminary experiments. To increase the variability of the prior, we introduce a uniformly distributed shifting parameter 4(-6, 15), which sampled only once and fixed for all i 1....,k. k. The final means μ4 are then given by:
with the probability density function (PDF) of the mollil Gaussian distribution parameterized by the expressed as
(log x 144)
p(vk, 2) = Σ exp
(3)
20?
for a value x.
To sample first passage times (FPTs) from these PDFs, we generate the PDF over logarithmically spaced range of x-values within the provided FPT bounds and employ the inverse transformation method, known the Smirnov transformation. The required series of calculations to derive the
CDF, its quantile function CDF different normalizations to properly scale the functions, and logarithmic transformations are detailed in Appendix C.I The prior distribution over synthetic RÑA first passage times used in this work is then represented by the log-encoded samples from multi-modal Gaussian distribution p(tb)) E Pow:
Y logı0 ((CDF (Wx)) 1)) (((W))).
(4)
4.2 PFNS FOR THE APPROXIMATION OF RNA FOLDING TIME DISTRIBUTIONS
We propose to use PFNs (Müller et al. 2022) to accelerate kinetic simulations for RNA first pas- sage time distributions. During training. the PFN with model parameters P is presented with M synthetic first passage times, {(,,, t4) M sampled from the prior distribution p(vx). To enable the model to generalize across varying amounts of training data instead of a fixed number of con- text folding times, this example set is split at a random cutoff point N ~ 24(0, M - 1). resulting in training subset Dusin {0,,, t )=)]== while the remaining first passage times are held out via masking. These held-out times, test = [tN+1..... } are then used as targets for prediction by minimizing the prior-data negative likeliieldoood (NLL)) according to Equation]] 1
Lo - [- log qo (Trest 1Oeett Danin)
(5)
Figure Sasurre@schemttccall illustrates this training process of KinPFN for single batch of size B, along with its application in approximating the posterior predictive distribution (PPD) of RNA first passage times using N real folding times as context obtained from kinetic simulator.
KinPFN Architecture and Hyperparameters We adopt the transformer-based (Vaswani et al., 2017 PFN architecture as proposed by Müller al.) (2022) and treat each pair (0, t) as a separate loken. To learn the distribution of the targets rather than their specific ordering, we deliberately omit positional encoding to maintain permutation invariance according to Müller et al. (2022). Since the first passage times t have already been log-encoded to the range 6, 15] in the prior distribu- tion p(t*)) (see Section 1). we encode the input with a linear layer after normalizing the data to zero mean and standard deviation of one while preserving the distributional properties. Follow- ing Müller et al.] (2022). ) we mask the attention matrix s.t. each position only attends to the training positions. This ensures that only training examples influence each other while test samples remain independent. We use the Adam optimizer (Kingma & Bal 2013) with a cosine decay (Loshchilov & Hutter 2017) and linear learning rate warm-up over 25% of the training steps previously
5

---

Published as a conference paper at ICLR 2025
Sample syathetic finst passage pUI p(at))
1yb [(0,,
((0,.))) (00,,4))
12 (102)) (21)
Triin KinPEN by minimizing 2 log log aill D
Di (.0,
((0, u (00, (i) =
Dua KO06 ()
KinPEN with parameters
. real first passage times and lest imput for inference ceriain lolding simultiions
Bayesian inference via the trained KinPFN, with the actual training
((0,, 41) u. (00))) (Dean Ose)
data points ingut:
ge Doee Duas) is pifseolosa, (huse)
Figure 2: A schematic visualization of KinPFN. Diagram based on Müller et al. 2022
proposed Müller al. 2022 Adriaensen et al. 2023 KinPFN outputs discretized distribu- tion go (t 0. Durin) (Riemann distribution; see Müller et al. 2022 ) using finite number of buckets with equal likelihood of containing t; a hyperparameter that included in our hyperparameter op- timization (HPO) procedure leading to final number of 1,000 buckets for KinPFN, initialized on a batch of 100,000 prior samples. A visualization of the discretized distribution qe can be found in Appendix H.4 Further hyperparameterss, like the number of layers, the embedding size, or the learning rate are inherited from the Transformer architecture. Given the infinite nature of synthetic training data, we the dropout rate and the weight decay to zero. We tune hyperparameters in two separate runs using Neural Pipeline search (NePS) (Stoll et al., 2023). More details regarding hyperparameters, hyperparameter optimization, and the final configuration of KinPFN can be found in Appendix D
The final model of KinPFN was trained for roughly five hours on a single A40 GPU.
5 EXPERIMENTS
KinPFN was trained on synthetic datasets of RNA folding times to learn to predict the distribution of first passage times, conditioned on few examples. Therefore, the predictions only depend on example folding times for a given RNA but not on other features, e.g., its length, sequence com- position, structure, or energy parameters. In this section, show that this feature of KinPPN is a main contributor to its practical relevance. First, we confirm its ability to transfer from the syn- thetic prior data to realistic scenarios using a test set of simulations for randomly generated RNAs (Section 5.1). Then, we demonstrate the practical importance of KinPFN in two case studies: We show that KinPFN capable of approximating first passage time distributions of natural RNAs (Section 5.2) and analyze the folding efficiency of different RNA sequences (Section 5.3). Finally, we assess KimPFN's ability to generalize to different biological data by approximating gene expres- sion data from previous study (Bagnall et al. 2020) (Section 5.4). Preliminary evaluations for the predictions on samples from the synthetic prior are shown in Appendix H.I We report per- formance in terms of prior-data negative log-likeliie likelihood (NLL) between the approximated posterior predictive distribution (PPD) and the true first passage time distribution and mean absolute error (MAE) between the CDF of the approximated PPD F(tt) and the true target CDF F()). More infor- mation about these measures can be found in Appendix F All experiments analyzing runtimes were benchmarked on single AMD Milan EPYC 7513 CPU with 2,6 GHz.
5.1 KinPFN TRANSFERS TO REAL- WORLD SCENARIOS
We assess the general capabilities KinPEN to transfer from synthetic data to data obtained from kinetic simulators. In particular, we analyze the robustness of KinPFN to changes in the sequence length of the RNA, the start and stop structure, and different kinetic simulators. To do so, we create novel test set of 635 randomly generated RNA sequences with lengths between 15 and 147 nucleotides, run Kinfold (Flamm et al., 2000) for 1,000 simulations on each of the test samples and extract first passage times (FPT') from the simulations. We compare KinP to GMMs and DP-

---

Published as a conference paper at ICLR 2025
Moon NLL 993.
MAE kd. 5td
0.00
RMA Sequence Length Ranges
Congartve 2og-latio Prbbalee
1idF% mosin asd
Cunuratie Fop iop.lnniin Probarlitty
Kedfts Oresin stel
0.8
0.
06-
6-
0.4
0.4
02-
0.2
0.0
302
102
227
35
-87 127
107
0.
107
30"
132
127
132 19 107
Figure 3: KinPFN approximations of first passage time distributions for simulation data of random RNA sequences across different settings. a: KinPFN testing set PPD mean NLL losses along with the CDF MAES across RNA sequence length ranges. Error bars show the standard deviation of the losses. b: Example approximation for an alternative folding path of 75 nucleotide RNA sequence with ground truth data obtained from Kinfold simulations. c: Example approximation for a 56 nucleotide RNA using Kfold simulation data as ground truth. We use N 25 context first passage times for all experiments. Approximation examples show the mean and standard deviation around the mean for 20 predictions with different context examples sampled at random.
GMMs across multiple modes as well as KDE on this dataset, evaluating their performance across varying amounts of context using identical context first passage times. The modes GMMs align with our assumption in the synthetic prior for KinPFN (Section 4.1.. Information about hyperparameter optimization for the competitors can be found in Appendix E To analyze the
for the
performance of KinPFN for arbitrary folding paths that do not include the unfolded or minimum free energy structural states, we additionally run Kinfold on randomly generated RNA sequence of 75 nucleotides and predict the PPD of first passage times for alternative folding paths. For the evaluation of KinPFN's robustness changes of the simulator, we use the Kfold (Dykeman) 2015) kinetic simulator to obtain FPTs for a randomly generated RNA of length 56. We provide more details about our novel test set in Appendix G Predictions with different context lengths, more competitor evaluations, and results for additional RNAs are reported in Appendix and and
H.3
Results Table D provides comparison of KinPFN with the GMM, DP-GMM and KDE on our introduced test set with respect to NLL. Please find results for mean absolute eтror (MAE) and Kolmogorov-Sminn
(KS) statistic as well as explanations about these metrics Appendix H.2 [ respectively. Across all three metrics, KinPFN consistently demonstrates lower mean losses from a sample size of 25 onwards, outperforming the other approaches across various context first passage times (N E {25, 50, 75, 100)). Consistent with expectations, the performance of KinPFN constantly improves with more context FPTs. For a context size of ten, KinPFN performs slightly worse than KDE in terms of MAE and KS, achieving the second best per- formance while still outperforming KDE in terms NLL. However, while we do observe vi- sually strong approximations with a context size of ten in later experiments with KinPFN (see e.g. Section 5.3), we note that these results should be taken with care due to relatively large KS values, indicating that the predicted distributions do not strongly match with the ground truth distributions at context size of ten. As shown in Figure 3h. KinPFN performs well across all sequences of the test set independent of the sequence length, given only 25 context points.

---

Published as a conference paper at ICLR 2025
This is important finding since especially simulations for long RNAs could benefit from accelerations with KinPFN. observe very Similarly, approximation of
Table Evaluation of KinPFN, KDE, and multiple GMM, and DP- RPGMMM models with different initial modality assumptions & E 32,3, 5,5} on a newly introduced testing set comprising 635 real-world first passage time distributions in terms of prior-data negative log-likelihood loss (lower is better) with context first passage time cutoffs N {10, 25, 50, 75, 1003.
good fit of the the CDF of first passage times for folding paths between alter- native structures (Figure 3p) and the application of KinPFN to simulations obtained from Kfold instead of Kinfold (Figure 3). Our results thus indicate that KinPFN seems to general- ize across different sequence lengths, start and stop struc- tures, and different simulators. Notably, the approximations with KinPFN only require 2.5% of the compute budget of the original simulators to achieve comparable results. However, the accuracy of the KinPEN approximations across all experiments can be further improved as we observe better performance with an increasing number of context examples (see also Table g and in Appendix H.2 This, however, comes at the cost of additional simulator runtime.
Method
First Passage Times N
10
25
50
75
100
KinPFN
1.3739
1.2435
1.2047
1.1916
1.1858
GMM2
2.3122
1.3612
1.2355
1.2036
1.1933
GMM3
5.2469
1.5830
1.2838
1.2132
1.1910
GMM4
13.1325
1.9922
1.3676
1.2480
1,2119
GMM;
37.5845
2.7708
1.4957
1.2953
1.2374
DP- DP-GMM2
1.6285
1.3529
1.2618
1.2305
1,2150
DP-GMM,
1.6268
1.3549
1.2653
1,2323
1,2155
DP-GMMA
1.6294
1.3558
1.2663
1.2337
1,2169
DP-GMM3
1.6256
1.3572
1.2675
1.2337
1,2175
KDE
1.4370
1.2559
1.2133
1.2003
1.1957
5.2 KinPFN APPROXIMATES FIRST PASSAGE TIMES OF ECKARYOTIC RNAS
While we observed robust performance of KinPFN for randomly generated RNA sequences, pre- dictions for natural RNAs might be more challenging. In particular, highly structured RNAs like transfer RNAs (IRNA) or ribosomal RNAs (rRNA) might show different folding behavior com- pared to random RNA sequences due million years of evolutionary pressure (Vicens & Kiefi, 2022 Herschlag 1995). We, therefore, decide to evaluate KinPFN on a IRNA phe 76 nucleotides and 5S rRNA of 121 nucleotides (RNAcentral Id: URS000055688D 559292) from Saccharonyces cerevisiae, one of the most extensively studied eu- karyotic model organisms in molecular and cell biology, commonly known brewer's yeast. For our experiments, we again use 1,000 Kinfold simulations as the ground truth data.
(RNAcentral Id: URS000011107D..
Cumulative Popuiation Probability
KinFFN
Target
2500
0.8
Context First Passage Times
Kinfold CRU Time (Minutes)
1500
0.4
0.0
10
10'
10
10a
102
10"
10'
10
10"
1000
Time
Number of Folding Simulazians
Figure 4: KinPEN first passage time CDF approximations for Succharomyces cerevisiae tRNAphe We show the mean and standard deviation for 20 predictions of KinPFN, each using 50 randomly sampled context times (left). On the right side, we show the runtime Kinfold for 50 and 1,000 kinetic simulations for the phe
8

---

Published as a conference paper at ICLR 2025
3.5
Corulaaive Papulaiten Probebillty
0.0
10%
75% Foibod
0.5
2.4
0.4
25% faied
25% Fobled
6, Contedt Times
strumam
Figure 5: RNA folding efficiency analysis. The left plot shows the ground truth CDFs F(t) for three sequences On. @1 and Φ2, representing the fraction of molecules folded into the MFE conformation (shown in dot-bracket notation (Hofacker et al. 1994)) over time The right plot displays the KinPFN approximations F()) with Kinfold times as context.
Results Figure 4 shows the first passage time CDF approximations of KinPFN for the IRNA (left). We observe that KinPFN is capable of approximating the ground truth data nearly perfectly using only 50 context first passage times. The runtime plot in Figure (right) visualizes the decrease of the computational demands as the approximations of KinPFN result from using only 5% of the original compute budget, reducing the required CPU time from approximately 2,686 minutes (100) simulations) to 170 minutes (50 simulations) while achieving nearly identical results. More predic- tions with different context times, as well as similar results for the 5S rRNA and further RNA types, shown in Appendix H.S and H.6 We conclude that KinPFN is capable of accurately approxi- mating the CDFs of first passage times for real-world, structured RNAs like tRNA and rRNA.
5.3 CASE STUDY: RNA FOLDING EFFICIENCY ANALYSIS
To demonstrate the utility KinPFN, we conduct a case study focused on comparing the folding efficiency of three 43 nucleotide long RNA molecules (do. Φ1, Φ2) that are predicted to fold into the same minimum free energy (MFE) structure. Alterations in the RNA sequences, such as mutations have a significant effect the
modifications often driven by evolutionary optimization folding dynamics (Flamm et al. 2000). A comparison of the CDFs first passage times can distin- guish molecules that fold more less efficiently and provide information about how alternations in the molecules impact the folding behavior, an important aspect for RNA-based therapeutics (Mollica al. 2022). For our experiment, we simulate 1,000 folding trajectories from the open chain the MFE structure using Kinfold and calculate the ground truth first passage time CDFs shown in the left plot of Figure for each of the RNA molecules.
Results We find that KinPFN captures the general folding behavior of the RNAs accurately, as shown in 3 However, while it captures the saddle points the CDFs of Φ1 (orange) and d2 (green) arguably well, it is slightly less accurate for the most efficiently folding RNA, do (blue). Remarkably, the KinPFN approximations were obtained using only ten context times, mark- ing 100x speed-up compared to each of the three individual simulation trajectories. Results for more approximations using different context lengths are shown in Appendix H.7
5.4 KinPFN GENERALIZES TO GENE EXPRESSION DATA
Besides their usage in RNA folding kineties analysis, CDFs of different distributions are common tool for the analysis of biological data. For example, Bagnall et al. (020)) analyzed the messenger RNA (mRNA) expression interleukin-l-a (IL-la), interleukin-l-3 (IL-13), and tumor necrosis factor-alpha (TNF-a) to study inducible gene expression in the immune toll-like receptor (TLR) sys- tem. Using single-moles molecule fluorescence in situ briiiizattion (smFISH) (Femino et al. 1998 Raj et al. 2008) analysis of the cumulative probability distribution of IL. IL-13, and 7NF-a mRNA expression in two cell lines (established RAW 264.7 macrophage cells and bone- marrow-derive macrophages (BMDM)) stimulated with lipid A, Bagnall al. (200)) demonstrate conserved vari-

---

Published as a conference paper at ICLR 2025
RAM 264.7
BMDM
10-
Cumulativv Probalbility
624
log (MRNA 13
log (MRNA
Terget iLix
Target 1.1
larget INF
KiiffN 1.1
KnFFF INFo
Contost 1230
Contest 1..
Concent Thio
Figure 6: Approximation of mRNA expression of IL-la, IL-13 and TNF-a in RAW 264.7 and BMDM cells. We plot approximations using only 25 context data points per gene.
ability in the TLR system across cell types, suggesting different modes of regulation of IL-13 and TNF-a expression. We use this experiment analyze the capability of KinPFN generalize to dif- ferent biological data. Specifically, we use the raw count data of 447, 718, and 356 RAW 264.7 and 447, 732, and 322 BMDM cells for II.lo, HI. 33 and 7NF-a, respectively, to predict the cumulative probability functions of mRNA expression replicate the outcome of the smFISH experiment of Bagnall et al. (020 with KinPFN while using only a fraction of the data.
Results Figure 6 illustrates the approximations of the mRNA expression of IL-lo, IL-13, and TNF- a. We observe that KinPFN can approximate the gene expression with high accuracy, using only roughly 8% of the expression data. These results suggest that besides its application to RNA folding kinetics- KinPFN could be valuable tool for different types of analysis across biological questions, including the potential speed up even wet-lab experiments (see also Appendix H.8)
CONCLUSION, LIMITATIONS & FUTURE WORK
We present KinPFN, the first work that uses prior-data fitted networks for biological data. Trained on a synthetic prior, we show that our novel approach can accurately model RNA folding kinetics while accelerating RNA first passage time analysis by orders of magnitude. Moreover, we demonstrate that KinPFN generalizes to gene expression data obtained from wet-lab smFISH analysis, suggesting that KinPFN could be applicable to the analysis of a wide range of different biological questions.
Limitations While showing impressive accuracy across multiple tasks, KinPFN also has limita- tions. Since is purely trained on synthetic first passage time data, it depends on a data-generating approach like kinetic simulators during inference. Consequently, KinPFN's performance is bounded by the accuracy of the simulator. Incorporating other features, like the RNA sequence, structure, or energy information, could mitigate this issue. However, it is an open problem to implement the required information in a synthetic prior without using external data sources. Additionally, KinPFN would benefit from larger-scale evaluations, e.g., on longer RNAs, to confirm its independence of RNA features like sequence length. However, obtaining this kind of data is currently infeasible due to the large computing demands of available simulators and the problem's complexity. Further, KinPFN is limited to a bounded range of time values; however, so far, we have not experienced this limitation to be major problem, and the training time of KinPFN is moderate, allowing retraining on adapted ranges. Similar to GMMs and KDEs, the performance of KinPFN strongly depends on the provided context. We tried compensate for that by showing mean and standard deviation around the mean across 20 context inputs to quantify the variation in KinPPN approximations.
Future Work Using synthetic data for biological applications appears very promising. Unlike GMMs or standard KDEs, KimPFN is not limited to predefined kernels or Gaussian distributions; we consider the definition of synthetic priors using different distributions as future work. Generally, PFNs could play an important role in the field of structural biology, with the potential to substantially impact biological analysis, offering tremendous possibilities to accelerate scientific discovery.
10

---

Published as a conference paper at ICLR 2025
REPRODUCIBILITY STATEMENT
To ensure the reproducibility of our results, have made our source code, the trained model, and datasets publicly available at htttps://githun tains detailed instructions for setting up the required conda environment and package installs (see README.md), Model checkpoints of KinPFN are provided in the models directory. The valida- tion and test sets are stored in the neps va dattiiii on set and k kinpfn test ing set direc- tories, respectively. We provide notebooks (along with the required experiment data) to demonstrate the training and evaluation of KinPFN and for reproducing results the notebooks directory. We recommend using single GPU with least 48GB of memory for training KinPFN. However, for inference, single CPU should be sufficient. Following the provided instructions, it should be straightforward to reproduce environment, train and evaluate KinPFN, and replicate experi- ments with minimal effort.
com/ auttom//King
The repository con-
AUTHOR CONTRIBUTIONS
F.R. conceptualized the study and developed the methodology. D.S. and F.R. wrote the manuscript, designed figures, and were responsible for data curation. D.S. implemented the model and con- ducted all experiments. J.F. contributed to the experimental design and baseline selection. M.W. and C.F. provided expertise in RNA kinetics simulations and theory. J.F., M.W., and C.F. assisted with manuscript refinement and figure layout. F.H. provided project supervision and secured funding.
ACKNOWLEDGMENTS
Dominik Scheuer and Frederic Runge would like to thank Samuel Müller and Steven Adriaensen for helpful discussions and valuable comments. This work is supported in part by the European Union's Horizon Europe Doctoral Network programme under the Sorrie-Sktoowskaaaae Curie grant agreement No 101072930 (TACsy). the Novo Nordisk Foundation grant NNF210C0066551 (MATOMIC), and the Austrian Science Fund FWF grant I-6440 N. The authors further acknowledge funding by the German Research Foundation (DFG) under SFB 1597 (SmallData), grant no. 499552394, and through grant no. 417962828 as well as support by the state of Baden-W Württemberg through bwHPC and the German Research Foundation (DFG) through grant no INST 39/963-1 FUGG (bwForCluster NEMO) and grant INST 35/1597-1 FUGG (bwForCluster Helix). Frank Hutter acknowledges the financial support of the Hector Foundation. This research was funded by the European Union (via ERC Consolidator Grant DeepLearning 2.0, grant no. 101045765). Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Research Council. Neither the European Union nor the granting authority can be held responsible for them.
Funded by the European Union
REFERENCES
Steven Adriaensen, Herilalaina Rakotoarison, Samuel Müller, and Frank Hutter. Efficient bayesian learning curve extrapolation using prior-data fitted networks. In Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS 2023), 2023.
net/forum? wiwpmm/xd=xxxxmxxxxxxex
Iddo Aviram, Ilia Veltman, Alexander Churkin, and Danny Barash. Efficient procedures for the numerical simulation of mid-size ma kinetics. Algorithms for Molecular Biology, 7:1-11, 2012.
Stefan Badelt, Ronny Lorenz, and Ivo Hofacker. Drtransformer: heuristic cotranscriptional ma folding using the nearest neighbor energy model. Bioinformatics, 39ttadbt, 2023.
James Bagnall, William Rowe, Nissrin Alachkar, James Roberts, Hazel England, Christopher Clark, Mark Platt, Dean A Jackson, Mark Muldoon, and Pawel Paszek. Gene-specifie linear trends constrain transcriptional variability the toll-like receptor signaling. Cell Systems, 11(3))300- 314, 2020.
11

---

Published as a conference paper at ICLR 2025
Christopher Bishop. Pattern Recognition and Machine Learning, volume 16, pp. 01 2006. doi: 10.1117/1.2819119.
David M. Blei and Michael I. Jordan. Variational inference for Dirichlet process mixtures. Bayesian Analysis, 1(1):121 143, 2006.
doi: 10.1214/06-BAI URL https://doi.czq//
1144006-BA144
Shi-Jie Chen. Rna folding: conformational statistics, folding kinetics, and ion electrostatics. Annual Review of Biophysics, 37:197-214, 2008. doi: 10.1166aamrrc.bbborrr77777001170
Elizabeth G. Conlon and James I.. Manley. RNA-binding proteins in neurodegeneration: mecha- nisms in aggregate. Genes & Development, 31(15):150-1528, August 2017. doi: 10.1/0//
304055.117.
Ludmila V Danilova, Dmitri D Pervouchine, Alexander V Favorov, and Andrei A Mironov. Rnaki- netics: a web server that models secondary structure kinetics of an elongating ma. Journal of bioinformatics and compulational biology, 4(02):589-596, 2006.
Samuel Dooley, Gurmoor Singh Khurana, Chirag Mohapatra, Siddartha Naidu, and Colin White. Forecastpfn: Synthetically-trainee zero-shot forecasting. Advances in Neural Information Pro- cexsing Systems, 36, 2024.
Eric C.
Dykeman, An implementation of the Gillespie algorithm for RNA kinetics with logarithmic time update. Nucleic Acids Research, 43(12):5708-5715 05 2015. ISSN 0305-1048. doi: 10. 1093/nar/gkv480. URL https: /to:://dooi..o http://ooo..ogg/////// 00t.:/..0.0......0.............. 10w30/00/////////////m
Xianyang Fang, Jason R Stagno, Yuha R Bhandari, Xiaobing Zuo, and Yun-Xing Wang. Small- angle x-ray scattering: a bridge between rma secondary structures and three-dimensional topolog- ical structures. Current Opinion in Structural Biology, 30: 30:147-160, 2015. ISSN 0959-440X. doi: https://db..og//00000////.....0.
URL pttps://wwwt sciencedi
cenn/sciencer http://c..ooooee....ee.////////0 and their protein complexes.
Folding and binding/Nuullic acids
Andrea M Femino, Fredric S Fay, Kevin Fogarty, and Robert H Singer. Visualization single ma transcripts in situ. Science, 280(5363):585 1998.
Christoph Flamm and Ivo Ludwig Hofacker. Beyond energy minimization: Approaches to the kinetic folding of RNA. Chemical Monthly, 139(4):447-457, 2008. doi: 10.1007/ 600706-008-0895-1
Christoph Flamm, Walter Fontana, Ivo L. Hofacker, and Peter Schuster. RNA folding at elementary step resolution. RNA, 333):325-338, March 2000. doi: 10. 10.10117/115553822009992222
Christoph Flamm, Ivo I.. Hofacker, Peter F. Stadler, and Michael T. Wolfinger. Barrier trees of degenerate landscapes. Zeitschrift für Physikalische Chemie, 216(2):155, 2002. doi: doi: 2pch.2002..11.21155. URL http:htts://// //oii org.10. 1544 zpch 20...216.2.22.... 155
Jörg Franke, Frederic Runge, and Frank Hutter. Probabilistic transformer: Modelling ambiguities and distributions for rna folding and molecule design. Advances in Neural Information Processing
Systems, 35:26856-26873, 2022.
Jörg K.H. Franke, Frederic Runge, Ryan Köksal, Rolf Backofen, and Frank Hutter. Rnaformer: A simple yet effective deep learning model for ma secondary structure prediction. bioRxiv, 2024. doi: 10.1101/2024.02.22.57777
Laiyi Fu, Yingxin Cao, Jie Wu, Qinke Peng, Qing Nie, and Xiaohui Xie. Ufold: fast and accurate ma secondary structure prediction with deep learning. Nucleic acids research, 50())414-e14, 2022.
Xiang-Dong Fu. Non-coding RNA: a new frontier in regulatory biology. National Science Review, 1(2):190-204 2014.
12

---

Published as a conference paper at ICLR 2025
Tsukasa Fukunaga and Michiaki Hamada. Computational approaches for alternative and transient secondary structures of ribonucleic acids. Briefings Functional Genomics, 18(3):88-191, 2019.
Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In International conference on machine learning, pp. 1263-1272. PMLR, 2017.
J. A. Hartigan and P. M. Hartigan. The dip test of unimodality. The Ammals Statisticss 13(1):70-84, 1985. ISSN 00905364, 21688966. httth:tt://wett wettortorrsttttteeeeooeeeeoooeet wttgg//tale/e/eeeecoeo
Daniel Herschlag. Rna chaperones and the folding problem. Journal of Biological Chemistry, (00036))0871-20877) 1995.
Ivo L Hofacker, Walter Fontana, Peter F Stadler, L Sebastian Bonhoeffer, Manfred Tacker, Peter Schuster, et al. Fast folding and comparison of ma secondary structures. Monatshefte fur chemie, 125:167-167, 1994.
Noah Hollmann, Samuel Müller, Katharina Eggensperger, and Frank Hutter. Tabpfn: transformer that solves small tabular classification problems in second. In The Eleventh International Con- ference on Learning Representations (ICLR), 2023.
Diederik Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations (ICLR), 2015. URL https:
/arxiv. abs//412...... 6980
Lisha Li, Kevin Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, and Ameet Talwalkar. Hyperband: novel bandit-based approach to hyperparameter optimization. Mach. Learn. Res., 18(1): 0765-6816, 16,1, jan 2017. ISSN 1532-4435.
Harvey Lodish, Arnold Berk, Paul Matsudaira, Chris A. Kaiser, Monty Krieger, Matthew P. Scott, et al. Molecular Cell Biology. W.H. Freeman and Co., New York, 5th edition, 2005.
Iyya Loshchilov and Frank Hutter. SGDR: Stochastic gradient descent with warm restarts, 2017.
URL htttts:://axiiiii
al6/1/08.b/11.//../.// 03983
Geoffrey McLachlan, Sharon X. Lee, and Suren Rathnayake. Finite mixture mod- cls. Annual Review of Staristics and Its Application, 6(Volume 6, 2019):355-378 doi: htpp://ddii.rg//ii http:000166//mmrstttttttt---1111
2019. ISSN 8326-831X.
URL https:/ http://wwwwanwaaaaaaaamaaamamaaa wtw.reevsssaatstt.sss1111111111.
annua revi weweeeeweeee eww..wwwwe..... www.wwwww.......e..ee.eeeeeeeeee https//cccccooeee/ooooaaaa//////
Luca Mollica, Francesca Anna Cupaioli, Grazisa Rossetti, and Federica Chiappori. An overview of structural approaches to study therapeutic rmas. Frontiers in Moleculur Biosciences, 9:1044126,
2022.
Samuel Müller, Matthias Feurer, Noah Hollmann, and Frank Hutter. Pfns4bo: In-context learning for bayesian optimization. In International Conference on Machine Learning, pp. 2547000.70. PMLR, 2023.
Samuel Müller, Noah Hollmann, Sebastian Pineda Arango, Josif Grabocka, and Frank Hutter. Trans- formers can do bayesian inference. In International Conference on Learning Representations (ICLR), 2022. URL
net/ for.mmid- KSugKcbNF9 K
Radford M. Neal. Markov chain sampling methods for dirichlet process mixture models. Journal of Computational and Graphical Statistics, 9(2):449-253 2000. doi: 10. 10.1080/161880000000 2000. 10474879. URL https: //wp://wwww..// tandfon ine. www.coo.//oooooooocoooooooooo..t /asii//a///// /10 0tt.///0.000/10000 8600.
2000. 1047.00444999
Sharat Patil, Frederic Runge, Jörg K.H. Franke, and Frank Hutter. Towards generative RNA de- sign with tertiary interactions. In ICLR 2024 Workshop Generative and Experimental Per- spectives for Biomolecular Design, 2024, URL httt:s::/ppeeeeeieeeeeee net/forum?id
pi.zoHOceHN
13

---

Published as a conference paper at ICLR 2025
Eva Prašnikar, Martin Ljubič, Andrej Perdih, and Jure Borišek. Machine learning heralding development phase molecular dynamics simulations. Artificial intelligence review, 57)) 2024.
Arjun Raj. Patrick Van Den Bogaard, Scott A Rifkin, Alexander Van Oudenaarden, and Sanjay Tyagi. Imaging individual mrna molecules using multiple singly labeled probes. Nature methods, 5(10):877-877 2008.
Herilalaina Rakotoarison, Steven Adriaensen, Neeratyoy Mallik, Samir Garibov, Eddie Bergman, and Frank Hutter. In-context freeze-thaw bayesian optimization for hyperparameter optimization. In Forty-first International Conference on Machine Learning (ICML), 2024.
Danilo Rezende and Shakir Mohamed, Variational inference with normalizing flows. Interna- fional conference machine learning, pp. 1530-1538. PMLR, 2015.
Frederic Runge, Jörg Franke, Daniel Fertmann, Rolf Backofen, and Frank Hutter. Partial ma design. Bioinformatics, 40(Supplement 14t.pppemmeen.1)i445--445555 2024.
Danny Stoll, Neeratyoy Mallik, Simon Schrodi, Maciej Janowski, Samir Garibov, Tarek
Abou Chakra, Daniel Rogalla, Eddie Bergman, Carl Hvarfner, Binxin Ru, Nils Kober, Théophane Vallaeys, and Frank Hutter. Neural pipeline search (NePS), October 2023. URL https:
//github. com/ aut oml/neps
Richard S Sutton. Reinforcement learning: An introduction. A Bradford Book, 2018.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. CoRR, abs/1706.03762, 2017.
BtRp:
org/ abs/1706. 03762
Quentin Vicens and Jeffrey S Kieft. Thoughts on how to think (and talk) about ma structure. Pro- ceedings of the National Academy of Sciences, 119(17):e21 12677 119, 2022.
Michael T. Wolfinger, W. Andreas Svrcek-Seiler, Christoph Flamm, Ivo L. Hofacker, and Peter F. Stadler. Efficient computation of RNA folding dynamics. Journal of Physics A: Mathematical and General, 37(17)47331, April 2004. doi: 10. 1088/0005-444007777770000 URL https://
wx..ooi.oo woi..orgol.......o.oooo 10. 1088/0305 01115-4447733111/00
Alain Xayaphoummine, T Bucher, and Herve Isambert. Kinefold web server for ma/dna folding path and structure prediction including pseudoknots and knots. Nucleic acids research, 33(suppl_2): W605-W610, 2005.
Angela M Yu, Paul M. Gasper, Eric J. Strobel, Kyle E. Watters, Alan A. Chen, and Julius B. Lucks. Computationally reconstructing cotranscriptional ma folding pathways from experimental data reveals rearrangement of non-native folding intermediates. bioRxiv, 2018 doi: 10.1101/379222. http://cnnenneeeeee/////////////
URL https: /wwww b www.xii........................i wwwpxivoor//ooooo www..wwwwg//oootettttt.oo.......
14

---

Published as a conference paper at ICLR 2025
A FURTHER BACKGROUND & RELATED WORK
In the following, we outline further background information and related work on RNA folding dy- namics.
The folding dynamics of RNA can be described as a stochastic process in a state space, comprised of of structures or conformations given RNA sequence may assume, move that defines the allowed elementary transitions between conformations in the state space, and transition rates for all allowed transitions. Mathematically, this compiles into continuous time Markov process governed by the following master equation for the state probabilities (t((t) of observing state at time
(t) dt
[(oo(t)key - (t))kyy))
yf
where Key is the transition rate from state y to state æ. For RNA sequences of moderate length, the master equation becomes too high dimensional to be solved analytically: therefore, it is approxi- mated by Monte-Carke simulation techniques (Flamm Hofacker 2008). which is, however, very time-consuming since enough stochastic simulations need to be accumulated to get a statistically representative time evolution the state probabilities. Alternatively, acceleration has been proposed through a more macroscopic structural description of RNA by helix kinetics methods (Xayaphoum- mine ct al. 2005 Danilova et al. 2006
A different approach to simulating the dynamics of RNA folding is through analysis of the underly- ing folding landscape. Such a landscape can be constructed from complete suboptimal folding with barriers (Flamm et al. 2002), which provides exact partitioning of the RNA conformation space into basins of attraction, i.c., local optima of the energy landscape. These macro-states provide a nat- reduced
ural ctarseggriniing of the folding landscape and allow re-formulate the dynamics number states, resulting in a massive speedup of computation time at comparable levels of detail. This idea is implemented in the tool treekin, which models the complete folding dynamics of RNA molecules of length up to approximately 100 nucleotides as a continuous-time Markov process that solved by numerical integration (Wolfinger et al. 2004).
For molecular dynamics (MD) simulations, Al methods have already been applied in different parts of the MD pipeline. Deep learning methods like graph neural networks (GNNs) (Gilmer 2017) or variational autoencoders (VAEs) (Kingmal 2013). as well as reinforcement learning (RL) (Sut-)
ton 2018) are regularly used these scenarios to e.g. enhance the sampling techniques during MD simulations, replace quantum mechanical force field simulations,
analyze the MD trajecto- ries (Prašnikar et al. 2024). However, current approaches mainly focus on small molecule data due to the complexity of MD simulations for larger macromolecules and have the disadvantage that they require large amounts of simulation data for training (Prašnikar et al. 2024). For more information on Al-based methods in the field of MD simulations, we refer the interested reader to detailed review of the field by Prräšniarr et al. (2024).
15

---

Published as a conference paper at ICLR 2025
B EXPONENTIAL Kinfold RUNTIME
Figure shows the mean CPU times (in minutes), along with the upper bound standard deviations, for simulating 10, 25, 50, 75, and 1000 folding processes transitioning from an open chain the minimum free energy conformation - with the mean times calculated for different RNA sequence lengths based on 50 distinct artificial RNA molecules per length. Despite the logarithmic scale on the CPU time axis, the mean CPU time still shows a linear increase, highlighting the exponential growth the computational time required for these simulations. The calculations for Figure were performed on single core of an AMD Milan EPYC 7513 CPU with 2.6 GHz.
1000
Avg. Kinfold Time for
10 Simulations
100
Simulations
50 Simulations
Kinfold CPU Time (in Minutes)
75 Simulations
1000 Simulations
0.1
0.07
60
RNA Sequence Length (in Nucleotides)
Figure 7: Kinfold mean CPU times (in minutes), including the upper bound standard deviations for simulating 10, 25, 50, 75, and 1000 folding processes over different RNA sequence lengths, based on 50 distinct artificial RNA molecules per length.
16

---

16, and the standard deviations o; are uniformly distributed between Published as a conference paper at ICLR 2025
C SYNTHETIC FOLDING TIME DISTRIBUTION PRIOR DETAILS
In the following, we will describe our proposed synthetic prior and the method for sampling a single batch of synthetic first passage times from it in more detail. The synthetic first passage times t are sampled from a distribution p(t'k) generated from family of multi-modal distributions Pv. as introduced in Section 4.1 The possible first passage time values across all p(tx) range from 10 to 10 with a -6 and 3 = 15, thereby limiting T E [Tsme Tsop] by min(Tam)) 10-6 and max(Taop) = 10ts, as we observed that this time range covers a very high fraction of possible RNA folding processes.
Each distribution p(v)k) E Ps. is characterized by l Gaussian components, each with a mean μ4 and a standard deviation Gs for 1.....k k. The base means fume are uniformly distributed between - and 3 + 0.1 and В-a 4.2. Further, introduce shifting parameter ö, which is uniformly distributed between o and [, i.e. ~ 7(-6, 15) and is fixed for all I,...,k. The final means are then given by:
p4 p4 base + 6.
Given the parameters the and a value x, the probability density function (PDF) of the multi-modal Gaussian distribution is expressed as:
(log x - -1,)2
p(1R,,
exp
20²
1=1
C.I SAMPLING FROM THE SYNTHETIC PRIOR OF RNA FIRST PASSAGE TIMES
To sample a batch of synthetic first passage times of size with fixed number of times, i.e., number of simulations per training example of M from multi-modal distribution p( ((tx), we employ the inverse transformation method also known as the Smirnov transformation. To do so we generate PDF p(vx,.2 .) over logarithmically spaced sequence of length M from 10a to 10". Then, to normalize this PDF and therefore ensure a valid probability distribution, we calculate:
p( 1/kk.20 x)
(6)
S10- y' (vh; r) dr
Next, we compute the cumulative distribution function (CDF):
L p( T)
CDF(
10
To ensure the CDF ranges from 0 to 1, normalize it by dividing by the integral over the entire range from 10a to 102:
CDF( [ p(vE, T) dr
(8)
J p(vi, r) dr
This normalization ensures that the CDF is properly scaled, with CDF( 108) =.
By inverting the CDF, we obtain the quantile function CDF To generate samples, we draw uniform samples u, from a uniform distribution 2(0011 1) for i 1,..., M and transform these samples using the inverse CDF:
t = CDFF 1 uz).
where I; are the sampled values from the distribution. We then encode these samples by applying logarithmic transformation:
= logio(t4)
Finally, constructing the prior output, for batch of size B and a fixed number of first passage times per example M, we generate the independent variables and Y as follows:
X = =00xMxx
Yi, ....,,....................
1..,2,...., 8m]] for I,... B.
17

---

Published as a conference paper at ICLR 2025
1.0
Curuiaation Population Probability
Samples
20
0.0
Polabilis Dunsity Finction
0.6
0.4
Oumulativv
0.2
0.0
0.0
10
10-
10-3
107
101
101
107
109
105
Synthetic
Time
Figure 8: Examples the synthetic prior of RNA first passage times. We show an example of single CDF (red) and the corresponding multi-modal probability density function (PDF) (blue; dotted line) generated from the synthetic prior (left). The distribution bi-modal (k- = 2) with the parameters d/a ((10.86, 1.36), (2.38, 2.48)). The right plot visualizes example CDFs generated from the synthetic prior.
18

---

Published as a conference paper at ICLR 2025
D KinPFN DETAILS
D.1 KinPFN HYPERPARAMETER
All hyperparameters in the KinPFN model are inherited from the transformer-based architec- ture (Vaswani et al.] 2017) of prior-data fitted networks (PFNs) proposed by Müller et al. (2022) These include the number of layers (nlayers), attention heads (nheads), embedding size (emsize), ) the number of neurons in each hidden layer (nhidden), the learning rate for the Adam optimizer (Kingma & Ba 2015 fleaming-ratee the number of steps per epoch (steps), and the total number of epochs (epochs). However, is not entirely accurate refer to "epochs" in this context, as we are training on syntheric data sampled from prior, resulting in a single, infinite epoch. In the context of PFNs, the loss is updated after each step, which is why we describe these steps as hyperparameterizee steps per epoch. The term "epochs" is used here primarily because it serves as hyperparameter within the code, providing a mechanism to control the training process. Another crucial parameter is the sequence length (seq of the input, representing the number of folding simulations (i.e., first passage times M) fed into the Transformer. This sequence length indicates the number of samples drawn from a prior distribution p((h)) E Por. as defined in Section 4.1] Additionally, given the infi- nite nature of synthetic training data and the singular epoch, we set the dropout rate and the weight decay to zero
D.2 HYPERPARAMETER OPTIMIZATION
Given the uncertainty about the significance of each parameter in the final model's performance, we decided to utilize Neural Pipeline Search (NePS) (Stoll et al., 2023) for the hyperparameter op- timization (HPO) of the KinPFN architecture. NePS is an pourcs Python library that offers state-of-the-as art HPO methods, including Bayesian Optimization and multi-fidelity methods like Hy- perband (Li et al. 2017). In our setup, we chose Hyperband as our HPO technique. Hyperband optimizes the search process by dynamically allocating resources, enabling faster identification of the best configurations. It strikes an effective balance between exploration and exploitation. Initially, it explores wide range of configurations with minimal resources, then progressively concentrates resources on the most promising candidates while discarding poor-performing ones early through a process of successive halving (Li at. 2017).
As a performance metric for Hyperband to assess the quality of hyperparameter configurations, utilize the prior-data negative log-likelihood (NLL), outlined in Section 2 This approach is equivalent to calculating the -ulbback-Leibler divergence between the approximated posterior predictive distribution (PPD) and the true target PPD (Müller et al. 2022). Each configuration
trained by Hyperband is evaluated on a newly introduced validation set, discribed in Section
We conducted two final iterations of the NePS Hyperband process, evaluating a total of 261 con- figurations. After completing the first iteration, we made slight adjustments to the search space. Additionally, we set N 25 for the validation pipeline in the first iteration and N = 10 for the second iteration, representing the number of context first passage times for each approximation. To ensure comparability across the validation of different hyperparameter configurations. we fixed the indices of these N context first passage times within the available time points, which, in real-world scenario, would typically be randomized since first passage times are usually obtained without any order when rumning kinetic folding algorithms like Kinfold (Flamm et al. 2000p.
Table 2 and 3 outline the hyperparameter search space used for our optimization process in iteration one and two, respectively (differences are highlighted in blue). In the first iteration, we used fixed batch size of 50. However, in the second iteration, we reduced the batch size to 40 to accommodate the adjusted search space, which brought to our GPU memory limit. Since Hyperband requires fidelity parameter to represent resource usage - in this case, computing time- - we designate the epochs hyperparameter as the fidelity parameter, defining its range between 250 and 3000. This is directly related to the steps per epoch, as the model runs a specified number of steps during each epoch, with each step involving training on a single batch. By tuning both the number of epochs and steps per epoch, we control the amount of synthetic data sampled from the prior that our model sees during training. Additionally, we adjust the learning rate for the Adam optimizer (Kingma & Ba (015). setting a range between 10-5 and 10-3, This range is informed by preliminary training sessions, where we observed that higher learning rates resulted in highly irregular learning curves
19

---

Published as a conference paper at ICLR 2025
Table Hyperparameter search space for NePS Hyperband iteration 1. Differences to iteration 2 are highlighted in blue
Hyperparameter
Type
Values/Range
epochs
Integer
[250, 3000] (hyperband fidelity)
steps
Integer
[50, 100]
learning_rat
Float
[10-5, 10 10-3] (log scale)
seq_len
Categorical
{200, 300, 500. }00}
buckets
Categorical
1100, 1000, 10000}
emsize
Categorical
(256, 512}
nheads
Categorical
[4. 8}
nhidden
Categorical
[512. 1024]
nlayers
Categorical
{2, 3, 4, 6, 8, 12)
Table 3: Hyperparameter search space for NePS Hyperband iteration 2. Differences to iteration are highlighted in blue
Hyperparameter
Type
Values/Range
epochs
Integer
[250, 3000] (hyperband fidelity)
steps
Integer
[50, 100]
learning.rate
Float
[10-5, 10-3] (log scale)
seq_een
Categorical
{200. 300, 500, 700, 1000, 1400}
buckets
Categorical
1100. 1000. 5000, 10000}
emsize
Categorical
(256. 512}
nheads
Categorical
[4. 8)
nhidden
Categorical
(512, 1024)
nlayers
Categorical
{2, 3, 4, 6, 8)
and, consequently, poor performance. We also evaluate models using different Transformer input sequence lengths specifically 200, 300. 500, 700, 1000, and 1400 as this parameter represents the number of first passage time samples M drawn from each prior distribution p(th.). Furthermore, we assess the models with 100, 1000, 5000, and 10000 buckets over which we discretize the learned posterior predictive distribution. For the embedding size, we evaluate options of 256 and 512, and we asses 4 and Transformer attention heads, which split the embedded input into smaller segments for focused attention. We also explore various model complexities by varying the number of neurons per hidden layer (512 and 1024) and the total number of layers, considering a broad range from 2 to 12 layers.
After both Hyperband iterations we identified four highly promising KinPFN architectures (KinPFN KinP Among these, KinPFN, and KinPFN, demonstrated the minimal NL.I in the first and second NePS Hyperband iteration with 1.1761 (N = 25) and 1.2101 (N = 10), respectively. Table 4 shows the NLL performance metrics of the found configurations across vari- ous cutoffs N E {10, 25, 50, 75, 100}. For each distribution example in the proposed validation set, randomly selected the N context times from the pool of M 1000 available times, ensuring broader and more generalizable evaluation, as the Hyperband validation pipeline was only based on fixed N first passage times with fixed indices within M.
While KinPFN, was the configuration with the second-best mean NLL loss with 1.2102 (N = 10) after KinPFNg the second NePS iteration, KinPFN, adopted the configuration of KinPFN, but trained on a larger Transformer input sequence length of 1400.
In the model analysis, KinPFN, shows the best performance with N 10 context first passage times. However, for all other values of N (N E {25, 50, 75, 100)), KinPPN, surpasses it. Addition- ally, KinPFN, outperforms both models from the second NePS iteration, KinPFN, and KinPFN,, based the NLL losses, as demonstrated in Table Based these results, selected KinPFN, as our final KinPFN model that was utilized in all experiments, as it shows the best overall perfor- mance.
20

---

Published as a conference paper at ICLR 2025
Table 4: Comparison of four promising KinPFN hyperparameter configurations identified in two NePS (Stoll et al., 2023), i.e. Hyperband (Li et al. 2017) iterations in terms of prior-data negative log-likelihood loss (lower is better) with context first passage time cutoffs N E {10, 25, 50, 75, 100).
Configuration
Parameters
First Passage
Times
10
25
50
75
100
KinPFN,
sog len=700, epochs=1000, steps=86,
learn-
1.348
1.254
1.225
1.216
1.210
ing snggraaee2..55550 55887480288874
10
buckets=1000, emsize=256, nheads44, nhidden=512,
nlayers=8, batch.size=10
KinPPN2
soq_lem=14000 1400, ppochs=(000,
stops=86,
learn- 10
1.378
1.246
1.207
1.195
1.189
ing.rmm=22.5555788555522222 435574855558889
buckets=1000, emsize=256, aheads=4, shidden=512,
nlayers=8, hatch size=50
KinPFN3
seq_len=1400 1400, epachs=
steps=72,
1.384
1.255
1.219
1.208
1.202
ing.rate=3, (t0t:=3..0044001111000
10
backets=10000,
emsi2e=256,
abeads=4,
nhid-
den=1024, nlayers butch
KinPFN,
seq len=1400, epochs=333.
sleps=85,
1.418
1.259
1.215
1.202
1.194
ing rate=7 062252166 123585
10-1.
backets 10000, emsize=512.
nheadss4, den=1024, nlayers=2, batch_iia==40
Final KinPFN Configuration The final KinPFN model consists of 4.86 million parameters, fea- turing a total of layers, each with hidden size of 512, 4 attention heads, an embedding size of 256, learning rate of
x 10-5, and 1000 buckets. The model was trained for steps (with batch size of 50), resulting in total 4,300,000 seen examples (calculated as 1000 x 86 x 50). Each example comprised M 1400 (synthetic) first passage times from (thcoretical) folding simulations, which represent the Transformer input sequence length.
1000 epochs, each consisting
21

---

Published as a conference paper at ICLR 2025
E KDE AND DP-GMM DETAILS
To ensure an optimal comparison of KinPEN with Kernel Density Estimation (KDE) and the Dirich- let Process Gaussian Mixture Model (DP-GMM), we performed random search hyperparameter optimization (HPO). For KDE, we tuned the bandwidth hyperparameter over logarithmic search space ranging from 10-3 3 to 10', while for DP-GMM, we optimized the weight concentration prior within a logarithmic range of 10-4 to 102. Both methods were evaluated using 1,000 configurations, selecting the one with the lowest mean negative ligeliill on our validation set, consisting of 2,019 real RNA first passage time distributions (Appendix GG) using 25 context times for each ex- ample. Figure 9 illustrates the HPO results for KDE (left) and DP-GMM (right). As result of the HPO, selected bandwidth of 0.352 with Gaussian kernel for KDE and weight concentra- tion prior of 9.79e-4 for DP-GMM and allowed a maximum of 100,000 iterations of Expectation- Maximization (EM).
3.0
Mean NLL vs. Bandwidth
Mean NIL vs. Weight Concantration Prior Best Weight Concentration Prior (9.8e-4)
Best Eandwidth (0.352)
Mean Negatioe (NL1))
Mean Negattive Log- Loe-tihtthoo (NLL)
2.6
202
2.2-
10a
1.8-
20²
1255
1.4
3.359
10
10:2
10
107
10:
10
10
30
10:
10"
10:
10a
CDE Banowicth
DP GMM Weight Concentration Prior
Figure 9: Hyperparameter optimization for Kernel Density Estimation (KDE) on the bandwidth parameter (left) and for Dirichlet Process Gaussian Mixture Models (DP-GMM) on the weight con- centration prior parameter (right).
22

---

Published as a conference paper at ICLR 2025
F METRICS
In our experiments and evaluations, we rely on the prior-data negative log-likelihood (NLL) between the approximated posterior predictive distribution (PPD) and the true first passage time distribution as a primary performance metric, consistent with its use during training and hyperparameter opti- mization (HPO) (Section 4.2):
lo ( |- log qo (fiest (0est: D Derain)
(9)
When comparing KinPFN to other methods, such as Gaussian Mixture Models (GMM), Dirichlet Process Gaussian Mixture Models (DP-GMM), and Kernel Density Estimation (KDE), we consis- tently use the mean negative log-likelihood (NLL) as the evaluation metric. This choice is motivated by the fact that the mean NLL reflects how effectively each method has learned the underlying pos- terior predictive distributions (PPDs) of the first passage times. Minimizing the NLL aligns with minimizing the Kullback-Leibles (KL) divergence between the estimated PPD and the ground truth PPD (Müller et al., 2022), making it robust measure of model performance.
As our main objective is approximating the CDFs of the first passage times, we additionally evalu- ate the performance by measuring the mean absolute error (MAE) and Solinggrov--mminie (KS) statistic between the CDF of the approximated PPD F1)) and the true target CDF F(t). For single CDF approximation of KinPFN, the mean absolute error (MAE) is defined as the average of the ab- solute differences between the predicted CDF values and the ground truth CDF values for a specific sequence of folding times. Mathematically, it can be expressed as:
M
MAE
1 M
F(L) F(())
F(4))
(10)
where M is the number of available ground truth first passage time points for the particular example RNA sequence, F(4) is the predicted CDF value at the i-th first passage time 4, computed by KinPFN, F(4) is the ground truth CDF value at the i-th first passage time 4,-
Similarly, the KS statistic is calculated as the maximum absolute difference between the predicted and true CDFs:
KS Statistic = max ((t.) - F(t)) 4
where lower values indicate better fit of the model to the true distribution.
23

---

Published as a conference paper at ICLR 2025
G VALIDATION AND TEST DATA
We introduce two new datasets: a validation set and a test set, both consisting of real RNA first passage times. The validation set contains 2,016 randomly generated RNA sequences, while the set includes 635 sequences. The times were acquired by simulating the folding process of the RNAs, starting from an open-chain conformation and progressing the molecule's minimum free energy conformation with the kinetic folding simulator Kinfold (Flamm et al., 2000). Figure I0 illustrates the distribution of RNA sequence lengths across both datasets. The validation set, used throughout all NePS (Stoll et al. 2023) iterations (i.e., Hyperband (Li et al. (017)), is shown in dark blue, while the test set, shown in dark red, is reserved for final KinPEN model evaluations (see Section 5.1. Importantly, these two datasets are mutually independent in terms of RNA primary sequences and secondary structures.
Number of First Passage Time Distribution Examples
NePS Validation Set
400
KinPFN Testing Set
350
300
250
200
150
100
50
0
15-24
25-34
35-44
45-54
55-64
65-74
75-84
85-94
95-104
105-114
115-124
125-134
135-144
145-154
RNA Sequence Length Ranges
Figure 10: Number of examples by RNA sequence length ranges for the custom validation set used in all NePS (Stoll al., 2023) i.e., Hyperband (Li et 2017 iterations and the custom testing set used for final KinPFN model evaluations (Section Both sets are independent of each other with respect RNA primary sequence and secondary structures.
24

---

Published as a conference paper at ICLR 2025
H ADDITIONAL EVALUATIONS
H.1 SYNTHETIC PRIOR APPROXIMATIONS
We evaluate our proposed model using synthetic data generated from the same prior distribu- tion employed during training (see Section 4.1). We approximate 10,000 synthetic first pas- sage time distributions, varying the cutoff points for the number of context first passage times N E {10, 25, 50, 75, 100). This allows us to evaluate the model's performance as the number of context points provided to KinPFN increases. For each case, we sample M = 1000 first passage times from the prior distribution. The performance is measured in terms of the posterior predic- tive distribution (PPD) mean negative log-likelihoot (NLL) and the cumulative distribution function (CDF) mean absolute error (MAE), computed over all 10,000 examples at each cutoff N. Table 5 presents the results of this evaluation. We observe significant improvements MAE when increasing the number of context points from N 10 to N 25 and from N 25 to N = 50. Beyond N = 50, while the loss continues to decrease, the rate of improvement slows down the context size grows from N 75 N 100.
both the NLL and
Table 5: Performance evaluation of KinPFN on 10,000 synthetic first passage time distributions.
Metrics are shown for different context first passage time cutoffs N E {10, 25, 50, 75, 100), mea- sured in terms of negative log-likelihood (NIL)) and mean absolute error (MAE). Lower values indicate better performance.
Performance Metric
First Passage Times N
10
25
50
75
100
Mean Prior-Date NLL
2.4265
2.1364
2.0596
2.0388
2.0281
Mean Absolute Error
0.0878
0.0553
0.0388
0.0321
0.0275
H.2 COMPARISON KinPFN, GMM, DP-GMM AND KDE
To further evaluate our model, we compare KinPFN against multiple Gaussian Mixture Models (GMMs) and Dirichlet Process Gaussian Mixture Models (DP-GMMs) using various initial modal- ity assumptions. Specifically, we consider mixture models with modalities & E {2, 3, 4. 5), align- GMM2, and ing with the assumptions outlined in synthetic prior (Section 4.1). denoted
DP-GMM3- For all evaluations, the models were provided identical context first passage times. Both GMM and DP-GMM models were allowed maximum of 100,000 iterations of Expectation- Maximization (EM). Additionally, we compare KinPFN to a Kernel Density Estimator (KDE) that optimized for its bandwidth hyperparameter (Appendix D. as discussed in Section 5.1 The results for MAE and KS are shown in Table and respectively. We observe that KinPFN outper- forms all other methods from context size of 25 FPTs onwards. Results on larger context sizes are shown in Table 8 9 and [0 demonstrating constant improvement of the predictions with growing context sizes. Finally, we also compare KinPFN with GMM ensembles with modalities k (2, 3, 4, 4) } and k (2, 3, 3). weighted according to their marginal likelihoods, across differ- ent context sizes. The results are shown Table and for NLL and MAE, respectively.
25

---

Published as a conference paper at ICLR 2025
Table Evaluation KinPFN, KDE, and multiple GMM, and DP-GMM, models with different newly introduced testing set comprising 635 real-
initial modality assumptions k {2, 3, 4, 5) world first passage time distributions in terms of mean absolute error (lower is better) with context first passage time cutoffs N { (10, 25, 50, 75, 100).
Method
First Passage Times N
10
25
50
75
100
KinPFN
0.0843
0.0561
0.0393
0.0333
0.0296
GMM2
0.1003
0.0848
0.0790
0.0773
0.0756
GMM3
0.0988
0.0866
0.0815
0.0801
0.0778
GMM
0.0952
0.0860
0.0816
0.0799
0.0777
GMM,
0.0929
0.0842
0.0809
0.0797
0.0778
DP-GMM2
0.0866
0.0774
0.0761
0.0756
0.0745
DP-GMMg
0.0867
0.0774
0.0763
0.0762
0.0751
DP-GMMA
0.0865
0.0770
0.0763
0.0763
0.0751
DP-GMMs
0.0860
0.0768
0.0762
0.0760
0.0751
KDE
0.0813
0.0690
0.0653
0.0644
0.0630
Table Evaluation of KinPFN, KDE, and multiple GMM; DP-GMM; models with different initial modality assumptions & E [2, 3, 4, 5) newly introduced testing set comprising 635 real-world first passage time distributions in terms of Kolmogorov -miinnv (KS) statistic (lower is better) with context first passage time cutoffs N {10, 25, 50, 75, 100).
Method
First Passage Times N
10
25
50
75
100
KinPFN
0.1615
0.1098
0.0809
0.0700
0.0632
GMM2
0.2084
0.1705
0.1586
0.1541
0.1510
GMM3
0.2210
0.1794
0.1644
0.1586
0.1537
GMM,
0.2293
0.1829
0.1674
0.1604
0.1547
GMM;
0.2352
0.1836
0.1695
0.1625
0.1564
DP-GMM2
0.1695
0.1505
0.1496
0.1488
0.1471
DP-GMM3
0.1694
0.1506
0.1499
0.1495
0.1475
DP-GMMA
0.1691
0.1500
0.1500
0.1496
0.1475
DP-GMM;
0.1682
0.1494
0.1499
0.1491
0.1476
KDE
0.1590
0.1344
0.1278
0.1256
0.1231
26

---

Published as a conference paper at ICLR 2025
Table 8: Evaluation of KinPFN, KDE, and multiple GMM, and DP-GMM, models with different initial modality assumptions & (2,3, 4, 5) on newly introduced testing set comprising 635 real- world first passage time distributions terms of prior-data negative log-likele le-likellihodd loss (lower is better) with context first passage time cutoffs N E {250, 500, 750, 1000).
Method
First Passage Times N
250
500
750
1000
KinPFN
1.1756
1.1716
1.1703
1.1697
GMM2
1.1764
1.1715
1.1696
1.1690
GMM3
1.1621
1.1542
1.1519
1.1508
GMM4
1.1612
1.1506
1.1476
1.1458
GMM;
1.1642
1.1499
1.1458
1.1438
DP-GMM2
1,1853
1.1735
1.1700
1.1683
DP-GMM3
1.1793
1.1627
1,1565
1.1533
DP-GMM
1.1802
1.1631
1.1562
1.1526
DP-GMMs
1.1809
1.1637
1.1566
1.1529
KDE
1.1874
1.1841
1.1832
1.1828
Table 9: Evaluation KinPFN, KDE, and multiple GMM, and DP-GMM; models with different initial modality assumptions k 3,, 4, 5) on newly introduced testing set comprising 635 real- world first passage time distributions in terms of mean absolute error (lower is better) with context first passage time cutoffs N (250, 500, 750, 1000).
Method
First Passage Times N
250
500
750
1000
KinPFN
0.0205
0.0155
0.0137
0.0126
GMM2
0,0742
0.0730
0.0730
0.0728
GMM3
0.0764
0.0756
0.0756
0.0754
GMM4
0.0763
0.0754
0.0753
0.0751
GMMs
0.0762
0.0751
0.0751
0.0747
DP DP-GMM2
0.0746
0.0741
0,0739
0.0736
DP-GMM3
0.0760
0.0757
0,0759
0.0757
DP-GMM,
0.0759
0.0756
0,0758
0.0756
DP-GMMS
0.0759
0.0756
0.0758
0.0756
KDE
0.0624
0.0617
0.0617
0.0615
Table 10: Evaluation of KinPFN, KDE, and multiple GMM, and DP-GMM, models with different initial modality assumptions & {2, 5) on a newly introduced testing set comprising 635 real-world first passage time distributions in terms of Kolmogorov- -iriro (KS) statistic (lower is better) with context first passage time cutoffs N E (250, 500, 750, 1000).
Method
First Passage Times N
250
500
750
1000
KinPFN
0.0484
0.0389
0.0359
0.0336
GMM2
0.1482
0.1465
0,1463
0.1460
GMM3
0.1493
0.1466
0.1467
0. 0.1462
GMM4
0.1489
0.1462
0.1461
0.1454
GMM;
0.1499
0.1464
0.1462
0. 1454
DP-GMM2
0.1474
0.1472
0.1469
0.1465
DP-GMM3
0.1483
0.1473
0.1469
0.1463
DP-GMM,
0.1481
0.1471
0.1466
0.1461
DP-GMMs
0482
0.1470
0.1465
0.1460
KDE
0.1217
0.1205
0.1206
0.1204
27

---

Published as a conference paper at ICLR 2025
Table 11: Evaluation of KinPFN and two GMM-Ensembles with different modality assumptions = {2,3, 4, 4,5} and k = (2,3.4] weighted according to their marginal likelihoods, on newly introduced testing set comprising 635 real-world first passage time distributions in terms of prior-data negative log-likelihooo loss (lower is better) with context first passage time cutoffs N (10, 25, 50, 75, 100).
Method
First Passage Times N
10
25
50
75
100
KinPFN
1.374
1.244
1.205
1.192
1.186
GMM- 5MMM--Enemmlee2.455
6.417
1.917
1.389
1.261
1.218
4.270
1.639
1.312
1.228
1.202
Table 12: Evaluation of KinPFN and two GMM-Ensembles with different modality assumptions (2,3, 5,5) and k 3,,3, 3, 4}. weighted according to their marginal likelihoods, on newly introduced testing set comprising 635 real-world first passage time distributions in terms of mean absolute error (lower is better) with context first passage time cutoffs N E {10, 25, 50, 75, 100).
Method
First Passage Times N
25
50
75
100
KinPFN
0.084
0.056
0.039
0.0333
0.030
GMM- S.MM-EEnsemble...433
0.093
0.084
0.081
0.0800
0.078
GMM- 30MM-Enemblee...4
0.095
0.086
0.081
0.0800
0.078
28

---

Published as a conference paper at ICLR 2025
H.3 ADDITIONAL KinPFN APPROXIMATION EXAMPLES
In Section 5.1, we conducted approximations of first passage time distributions using KinPPN on 75-nucleotide RNA. The folding process for this RNA simulated with the Kinfold kinetic simulator (Flmm et al.] 2000), employing custom start and stop structures. We also approximated the folding time distribution for 56-nuclcotide RNA, for which we obtained ground truth data using the KFold simulator (Dykeman 2015). In all instances, the approximations were based on just 25 context first passage times. This appendix provides additional approximations for these RNAs, expanding the analysis by varying the number of context first passage times, specifically N {10, 25, 50, 75). Additionally, we performed approximations for 93-nucleotide RNA using the same simulation method as for the 75-nucleotide RNA (Figure II] and extended our analysis of the 56-nucleotide RNA to include results for a nlcnucleo RNA (Figure [2).. Additionally, Figure pressents representative approximations for two RNAs from newly introduced test set: a 97-nucleotide RNA and a 1 19-nuclooeide RNA, each evaluated with four different numbers of context first passage times, N {10, 25, 50, 75).
Gef''s Iuaee
Loff proon ael
Pos
Figure 11: Approximations of the cumulative distribution function (CDF) for the first passage time using KinPFN, with context times N {10, 25, 50, 75} (left to right), for an RNA sequence of 75 nucleotides (top) and 93 nucleotides (bottom). The folding process was simulated using custom initial and final structures rather than the open chain and minimum free energy conformation.
shai's Paar ad)
E'ei's Pose F21)
Aariarrvsn no1
Sprlecinan od
Tear
1.00
Digal Contecti
Dage:
Tigg:
08
Gradd hac/haaae
04
34
4.3
08
127
154
Figure 12: KinPEN first passage time CDF approiiaaitss with context times E (10, 25, 50, 75) (left to right) for 31 nucleotide long RNA (top) and 56 nucleotide long RNA (bottom). First passage times were obtained using the kinetic folding algorithm Kfold Dykemanynnnnna (0015).
29

---

Published as a conference paper at ICLR 2025
Do Peet
Codted Facuapo
Cantad Fid Fassay
111 sneer U03
Brant ats
Kuaiin 131
08
30
04
3.4
1999
1974
Figure 13: KinPEN first passage time CDF approximations with context times E {10, 25, 50, 75) (left to right) for 97 nucleotide long RNA (top) and 119 nucleotide long RNA (bottom) that part of the newly introduced test set.
30

---

Published as a conference paper at ICLR 2025
H.4 KinPFN RIEMANN DISTRIBUTION VISUALIZATION
To provide a more intuitive understanding of the actual KinPFN predictions, Figure illustrates two CDF approximations for different RNA molecules, each with N 25 context frst-passage
times, compared against the ground truth CDF based on 1,000 Kinfold (Flamm et al. 2000) times (left) alongside the probability density function (PDF) the corresponding approximated posterior predictive distribution (PPD), also known as the Riemann distribution (right) (Müller et al. 2022).
As discussed Section 4.2 the PFN predicts continuous distribution, which we discretized into finite number of buckets, forming the PDF bars. Each bar represents a bucket, and in our final KinPFN model, we used 1,000 buckets, a hyperparameter defined in Section
By examining the approximated PPD PDFs, can observe the multi-modal nature of the learned first- pass-passage time distributions. For instance, the distribution for 65 nucleotide long RNA in Fig- ure [4 (top) shows bi- bi-modality with two distinct peaks, while another first passage time distribution for a 86 nucleotide long RNA in Figure [4 (bottom) exhibits tri-modality with three peaks. Notably, the calculated CDFs for these multi-modal PPDs align closely with the ground truth CDFs from the 1. 1,000 real first-passage times, demonstrating the effectiveness our proposed prior, specifically, the family of multi-modal Gaussian distributions introduced Section [4. This prior, from which sampled synthetic first-passage time distributions to train KinPFN, enabled strong generalization RNA first fiss-passage time distributions.
1.0
Target
KinPFN PDF
Cumulative Population Probability
KinPFN
Context Passage
0.8
0.30
Riemann Density
0.6
0.4
0.15
0.05
109
10a
102
10a
10a
105
10"
107
10
10-2
10a
10*
10"
10:0
101s
1046
Time
Time (Riemann Distribution Buckets)
Target
KinPFN PPD PDF
Cumulative Population Probability
KinPFN
0.8-
Context First Passage Times
0.4
Riemann Density
0.6
0.3
0.4
0.2
0.2
0.1
0.0
10a
102
10a
10a
10a
10a
10
10a
0.0
10-5
10-2
10a
10*
102
1010
101s
10
Time
(Riemann Distribution Buckets)
Figure 14: KinPEN PPD CDF approximations (left) along with the corresponding multi-modal PPD PDFs (right) for two RNA molecules with lengths of 65 (top) and 86 (bottom) nucleotides.
31

---

Published as a conference paper at ICLR 2025
H.5 EUKARYOTIC TRANSFER AND RIBOSOMAL RNA
We present additional results for the first passage time distribution approximations using KinPFN for tRe and 5S rRNA from the eukaryote Saccharomyces cerevisiae, also known as brewer's yeast. Figure didipaay the cumulative distribution function (CDF) approximations of first passage times for tRNA (top) and 5S rRNA (bottom). with varying numbers of context first passage times, N {10, 25, 50, 75), as inputs to KinPFN. Additionally, Figure [ the CPU time required (in minutes) to compute 10, 25, 50, 75, and 1,000 first passage times for both Phe (blue) and 5$ rRNA (red) using the kinetic simulator Kinfold (Flamm et al. 20000.
6:911 sin
68:
641
Thme
Turget
Treat
Tepar
Teee: Datrnliia
Oratre Beroogg
50
37
10
Figure 15: KinPFN first passage time CDF approximations with context times N E {10, 25, 50, 75) (left to right) for Saccharomyces cerevisiae tRNApbe (top) and 5S rRNA (bottom).
tRNAphe
2500
5S-rRNA
Kinfold CPU Time (Minutes)
2000
1500
1000
500
0
10
25
50
75
1000
Number of Folding Simulations
Figure 16: Kinfold CPU time (in minutes) vs. number of folding simulations for Saccharomyces cerevisiae tRNAphe phe and 5S rRNA.
32

---

Published as a conference paper at ICLR 2025
H.6 APPROXIMATIONS FOR ADDITIONAL RNA TYPES
In this section, we show the results of the first passage time distribution approximations using KinPFN on two further RNA types: hsa-miR-7107-3p (RNAcentral-ID: URSM007S5FFB2_6600 from Homo sapiens (human) and the SAM riboswitch (S box leader) (RNAcentral-ID: S0000002FF92222243333 from Bacillus subtilis subsp. subtilis str. 168. Figure 17 displays the cumulative distribution function (CDF) approximations for the first passage times of these RNAs. The top row shows results for hsa-miR-7107-3pp 27-nucleotide mieroRNA, while the bottom row illustrates results for the SAM riboswitch, 92-nucleotide regulatory RNA. For both RNAs, the approximations are generated using varying numbers of context first passage times as inputs to KinPFN, with N E {10, 25, 50, 75} (from left to right). These results provide additional insights into the robustness and versatility of KinPFN across diverse RNA types.
Unntad lesteg time!
Corrent Netrect
Carput Pocrect Timns
32
11e6
Tagg Jone
Figure 17: KinPFN first passage time CDF approximations with context times N E {10, 25, 50, 75) (left to right) for a hsa-miR-710773p microRNA from Homo sapiens (human) (top) and SAM riboswitch from Bacillus subtilis subsp. subtilis 168 (bottom).
33

---

Published as a conference paper at ICLR 2025
H.7 APPLICATION: RNA FOLDING EFFICIENCY ANALYSIS
Here we show KinPFN approximations using additional context first passage times of N E (10, 25, 50} case study that focuses on comparing the folding efficiency of three 43-nucleotide RNA molecules (do, Φ1, Φ2), each predicted to fold into the same minimum free energy (MFE) structure (wistop
..............(((............... (((((((..)))))))....... As noted in Section 5.3 KinPFN accurately cap- tures the overall folding behavior of these RNAs using just ten context times. We further observe that increasing the number of context first passage times to 25 and 50 enhances the accuracy of these approximations, as shown in Figure 18
Con ARROCIIIINNN AELAANLL CONAA (A/00))
LORE
Figure 18: RNA folding efficiency analysis. The first plot from the left shows the ground truth CDFs F(t) for three sequences oo. Φ1 and Φ2 representing the fraction of molecules folded into the MFE conformation over time t. The second, third, and fourth plot displays the KinPFN approximations Ftt) with ten, 25, and 50 Kinfold times as context.
34

---

Published as a conference paper at ICLR 2025
H.8 KinPFN GENERALIZES TO GENE EXPRESSION DATA
In this section, we present supplementary approximations of mRNA gene expression data for interleukin-l-a (II-la), interleukin-l-3 (IL-13), and tumor necrosis factor-alpha (TNF-a). These additional results build upon the experiment described in Section 5.4, where KinPFN was applied to predict cumulative probability functions for gene expression in RAW 264.7 and BMDM cells using a subset of context data points. The approximations shown here utilize further numbers of context data points (10, 25, 50 and 75) and different seed, providing a more comprehensive understanding of the predictive capabilities of KinPFN across different biological datasets. The results in this sec- tion serve to reinforce the main findings and demonstrate robustness KinPFN in accurately replicating the outcomes of experiments such as those conducted by Bagnall et al. (020)) while using minimal input data.
290.
KD 2011
30.
20.
keuineek DD
15 20attt44r
Torget
Target LLB
Targer TNFo
KnPPN
KInPFN IL18
KiPPP TNFE
Contect IL3e
Contest IL3
Contest TNFa
Figure 19: Approximation of mRNA expression of IL-la, IL-13 and TNF-a RAW 264.7 and BMDM cells. We plot approximations using only 10, 25, 50 and 75 context data points per gene.
35


Reviews:
[{"review_id": "IVquwpuPG3", "reviewer": "Reviewer_TUhM", "summary": "The paper introduces a model for predicting RNA folding dynamics called KinPFN. Specifically, this work focuses on computing the time needed for an RNA to fold into a specific structure (i.e., the first passage time). KinPFN uses a prior-data fitted network (PFN) that is calibrated using a sample of synthetic first passage time to predict entire cumulative distribution or RNA structures. The methodology is benchmarked on random and real sequences, and the paper concludes with an illustration of an application to gene expression.\n\nOverall, the methodology is sound, and the results are convincing. The development of RNA folding dynamics prediction tools is timely and only a limited set of programs are currently available. Furthermore, machine learning approaches seem suited to this task, and, to my knowledge, this contribution is the first of its kind. The contributed approach is still a proof-of-concept, but it provides solid ground for further exploration of ML. The manuscript is clear and well written, yet it could benefit of clarifications suggested below.", "questions": "Is there any further conceptual novelty in this submission that I might be missing?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "Overall, it is a nice paper from a bioinformatics perspective. The machine learning component is a bit limited but I believe fits the broad definition of originality and significance through \"application to a new domain.\" The authors basically apply one existing framework (prior-data filtering networks or PFNs) to model RNA folding dynamics. The innovative aspect of this work is thus rather the application to RNA folding. The research is timely, and the gain in speed could lead to interesting RNA design applications. The authors mention it but do not discuss it much. However, even if it is still very early, there are interesting/promising applications at the end of the paper. It is a clean and solid piece of work from which could emerge highly useful applications.", "weaknesses": "•\tAs far as I understand it, the structural model is based on secondary structures and uses (for simulations) the nearest neighbor (NN) energy model. The application to secondary structure is only partially described in the paper. It is an important limitation, and I am concerned that readers may miss this information. I would suggest updating Fig. 2 (reproduced from Muller et al, 2022) to include more details on the sampling (e.g., using a secondary structure model and kinfold) and eventually on the benchmark too. This is just a suggestion as the author(s) may prefer to include this information at other places of the manuscript (e.g., expand the background section to describe the folding model).\n•\tA claim of the paper is to drastically accelerate folding simulation. I agree this is a nice feature, but I also wonder to what extent it is currently a major bottleneck (for biologists) or what new applications it will enable. I think the manuscript could benefit from further justifications/motivations. For instance, the usefulness of KinPFN for design applications sounds very promising, and I would have liked more discussion (or experiments?) related to this topic. Moreover, since the models currently use kinfold simulations, it could be limited by the accuracy of the NN energy model, which has limitations on longer RNAs. Maybe KinPFN offers new perspectives to deal with this challenge?\n•\tIt is not clear to me that PFNs perform much better than other options (e.g., KDEs) but at least they seem to be suited to the task. Table 1 compares KinPFN to DP-GMM and KDE models. The performance of KDEs seem very close to KingPFN, and I wonder if the author(s) could provide more context in the main body to discuss the significance of the results (e.g., from supp mat H.2)?\n•\tFig. 3a shows a distribution of the performance across RNAs of various lengths. It may be out of the scope of this manuscript, but I would be interested to understand better how other features such as the nucleotide composition, MFE value, number of base pairs or loops, etc. impact the results. Also, how does KinPFN perform on multi-stable RNAs?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cgXw4WzN46", "reviewer": "Reviewer_PRsD", "summary": "The authors propose a novel method to approximate RNA first passage times (FPTs) using Prior-data fitted networks (PFNs), which are transformers that return a posterior-predictive distribution subject to simulated draws from a prior dataset. In this case, the prior dastset is created from a synthetic prior. The authors put a great deal of effort into hand-crafting this prior (e.g. using biological prior knowledge) that will then be used to train their PFN. The result is a prior that is able to train a PFN that predicts FPT better than competing methods, such as KDE, GMM, and DP-GMMs. The paper's biological motivation is sound and the achievement would help researchers in this field. However, the paper has room for improvement to be published at ICLR instead of a computational biology journal. Why not make the prior adaptive based on the data? There is opportunity to solve this problem that wouldn't require hand-crafting a new prior, or, make it more general for different datasets that would make it an _excellent_ contribution.\n\nThe paper's prose is clear. I do think that some of the tables could be rearranged to give the reader more clear insights into how the method works, which I relay below.", "questions": "- Why not show the comparison between the sequence length and FPT in Table 1? I think that might provide more insight as to where KinPFN works better over other methods, unless sequence length isn't an important variable, which it seems to be since it's the subject of study in Figure 3.\n- Why not put the legend in Figure 5 in the appendix? I'm not going to be able to copy/paste that to check, anyways.\n- Why not use the KS test between CDFs as another comparison? This would help capture maximum discrepancy and add nuance to the experimental analysis.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Concrete biological problem of predicting FPTs that was well-motivated and explained.\n- Clarity in explanation of method used.\n- Clear advantage on benchmarks compared to other current solutions.\n- Dramatic decrease in folding time.\n- Demonstrated applicability to problems that also exhibit FPT characteristics.", "weaknesses": "- I would have liked to have seen Table 6 comparing all the different methods in the main paper. Plus, I would have liked to have seen comparison on MAE across methods, including in Table 1. \n- Reliance on mulitmodal Gaussians for the prior. \n- Reliance on only two metrics to measure accuracy of the metric.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8Zc7ZYv2z1", "reviewer": "Reviewer_DNAm", "summary": "This paper studies the RNA folding kinetics modeling problem, which is helpful for understanding RNA behavior and designing RNA. The authors propose to apply a deep learning method based on a prior-data fitted network to quickly estimate the distribution of RNA folding times. Experiments in synthetic datasets and real examples demonstrate the effectiveness of the proposed method.", "questions": "Please refer to the Weaknesses section for details.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1.\tThis paper is the first to use deep learning methods to study RNA folding kinetics modeling, an important problem in RNA biology.\n2.\tOn synthetic datasets, the proposed method demonstrates superior performance compared to traditional approaches such as kernel density estimation and Gaussian mixture models.\n3.\tThe proposed method has an advantage in running speed.\n4.\tThe paper provides comprehensive details on dataset construction and model training, ensuring high reproducibility.", "weaknesses": "1.\tThe current writing does not facilitate quick comprehension of the research problem for readers from diverse backgrounds. It would be beneficial if the author could include a figure illustrating specific data and formalization when introducing the problem. For instance, depicting the relationship between the RNA folding process and the corresponding change in folding fraction could enhance clarity.\n2.\tThe unique challenges RNA folding kinetics pose are not adequately summarized in the introduction. Additionally, the paper directly employs prior-data fitted networks to model the CDF without additional enhancements. Highlighting the improvements made to address the specific issues in this field would enhance the paper's contribution.\n3.\tIt would be clearer to explicitly state in the introduction or background section whether the paper focuses on RNA's tertiary or secondary structure, and how the folding ratio is calculated.\n4.\tCompared to the dynamic changes in RNA secondary or tertiary structures, the folding ratio provides very coarse-grained information about RNA folding dynamics, which seems still far from practical applications. The paper needs to further elucidate how this study can contribute to solving RNA biology problems.\n5.\tIn the experimental section, the results from Kinfold are used for validation, but the inherent error of Kinfold needs rigorous demonstration, which diminishes the persuasiveness of the results. Is it possible to use collected or published wet lab data for the evaluation of this problem?\n6.\ttRNA and rRNA are the most common and numerous types of RNA. It would be better to test a broader and more diverse range of RNA types.\n7.\tThere is a lack of research and discussion on deep learning methods suitable for the data in this problem. The paper only presents the prior-data fitted network for deep learning-based probability density estimation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OvQGY9hIKl", "reviewer": "Reviewer_FZT1", "summary": "Biomolecules adopt many conformation in situ and understanding the kinetics of the transitions between these conformations is useful for understanding their biophysical behavior. Expensive simulations can start at one conformation and measure how long it takes to transition to another conformation -- the passage times. To minimize the number of simulations needed to characterize the distribution of first passage times, the authors build a prior on the distribution of first passage times; this in principle allows them to do more efficient inference with fewer measurements.\n\nThe authors focus on RNA kinetics and expression data. They build their prior by fitting a language model on sequences of passage times. They demonstrate they fit data better than Dirichlet process and kernel density estimator baselines and have reasonable fits on real data.", "questions": "1. Why not train on real passage time data that you can simulate? Why synthetic?\n\n2. The prior described by the synthetic data is so simple it may be easy to just run MCMC. Why use a language model at all?\n\n3. Language models as priors can have some pathologies. How does the model behave as the amount of data becomes large? In figure 11 it looks like even with a lot of data the PFN doesn't converge to the true CDF.", "limitations": "", "rating": 3, "confidence": 4, "soundness": 1, "presentation": 3, "contribution": 1, "strengths": "The paper is sound, and has extensive experimental validation.", "weaknesses": "See questions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)