Signal to Sequence Attention-Based Multiple Instance Network for Segmentation Free Inference of RNA Modifications
Authors: Christopher Hendra
Alexandre H. Thiery
Jonathan Goeke
Venue: Submitted to ICLR 2023 (2023)
Keywords: Multiple Instance Learning
Deep Learning
RNA Modification
Computational Biology

Abstract: Direct RNA sequencing technology works by allowing long RNA molecules to pass through tiny pores, generating electrical current, called squiggle, that are interpreted as a series of RNA nucleotides through the use of Deep Learning algorithms. The platform has also facilitated computational detection of RNA modifications via machine learning and statistical approaches as they cause detectable shift in the current generated as the modified nucleotides pass through the pores. Nevertheless, since modifications only occur in a handful of positions along the molecules, existing techniques require segmentation of the long squiggle in order to filter off irrelevant signals and this step produces large computational and storage overhead. Inspired by the recent work in vector similarity search, we introduce a segmentation-free approach by utilizing scaled-dot product attention to perform implicit segmentation and feature extraction of raw signals that correspond to sites of interest. We further demonstrate the feasibility of our approach by achieving significant speedup while maintaining competitive performance in m6A detection against existing state-of-the-art methods.

Under review as a conference paper at ICLR 2023
SIGNAL TO SEQUENCE ATTENTION-BASED MULTIPLE INSTANCE NETWORK FOR SEGMENTATION FREE INFER- ENCE OF RNA MODIFICATIONS
Anonymous authors Paper under double-blind review
ABSTRACT
Direct RNA sequencing technology works by allowing long RNA molecules pass through tiny pores, generating electrical current, called squiggle, that are interpreted as a series of RNA nucleotides through the use of Deep Learning algorithms. The platform has also facilitated computational detection of RNA mod- ifications via machine learning and statistical approaches as they cause detectable shift in the current generated as the modified nucleotides pass through the pores. Nevertheless, since modifications only occur in a handful of positions along the molecules, existing techniques require segmentation of the long squiggle in order to filter off irrelevant signals and this step produces large computational and storage overhead. Inspired by the recent work in vector similarity search, we introduce a segmentation- free approach by utilizing scaled-dor dot product attention to perform implicit segmentation and feature extraction of raw signals that correspond to sites of interest. We further demonstrate the feasibility of our approach by achieving significant speedup while maintaining competitive performance in m6A detection against existing state-of- oftt---fthe-ar methods.
INTRODUCTION
RNA modifications have been discovered since the 1950s (Cohn Volkin [951 Kemp Allen Davis & Allen 1957) and have been found to play prominent role in a wide range of
1958
biological processes (Xu et al. 2017: Yankova et al., 2021 Nombela et al. 2021)]. Several methods exist to detect these modifications, most prominently N d'-methyladenosing (m6A) (Meyer et al.
2012
Dominissini et al. 2012 Chen et al. 2015 Ke et al. 2015 Molinie et al.) 2016: Linder et al.
2015
Koh et al. 2019; Dierks et al. 202 1)]. pscudouridine (1²) (Shhwartz et al. 2014a; Lovejoy
et al.
2014 Carlile et al. 2014 Liu et al. 2015), and N Ne-methylcytosing (m5C) (Squires et al. 2012
Hussain et al. 2013) Huang et al. 2019). These methods, while useful, require specific antibody or
chemical reagents as well as experimental expertise that is beyond the reach of most computational
labs.
The recent development of direct RNA sequencing technology by Oxford Nanopore (Gar alde et al. 2018) allows the direct sequencing of native RNA molecules. The technology works through the use of motor protein that controls the translocation of RNA molecules through the nanopores, generating an electrical current called squiggle that corresponds the identity of the molecules passing through the pores Figure Tp. The electrical current is deciphered into a sequence of four RNA nucleotides (G, A. C. U) through a process called basecalling and this involves training either Recurrent Neural Networks (RNN) or Convolutional Neural Networks (CNN) (Boža et al. 2017 Stoiber & Brown 2017 Teng et al. 2018 Zeng et af. 2020) using Connectionist Temporal Classification (CTC) approach (Graves et al., 2006).. The presence of a modified nucleotide often results in a shift in the electrical current which can be exploited for RNA modification detection. Nevertheless, modified nucleotides are rare and so only short portion of long RNA squiggle is
relevant for modification detection. As a result, segmentation algorithms (Loman et al. 2015 Stoiber et al.) are often used by existing detection methods during preprocessing in order to extract useful
signals matching to modified positions (Stoiber et al.) Leger et al. 2019 Lorenz et al.
2020 Deda
Pratanwanich al. 2021 Gao al. 2021
Begik et al. 2021
Parker al.] 202
Hendra al.

---

Under review as a conference paper at ICLR 2023
DNA/RNA
Nanspore
Segramtttion
Prepaassry
Modidictionn
Probability
6 AC TGG
Reference transeripts
(b)
(C)
[C)
(7)
Current
Mm
Farcaling
Shoalssy Vell
Modification
Probubility
Our Approppc
ACTEB
ACTOG
Figure 1: (a) RNA molecule being translocated through the Nanopore. Image is adapted from (Wan et al., (b) Electrical current, or nanopore squiggle, generated as the RNA nucleotides pass through the Nanopore. (c) The squiggle is deciphered a series of nucleotides through basecalling. RNA modification such as m6A modification can only occur in the presence of AC motif, so signals matching all other motifs might not be useful for detecting m6A modifications.ce The basecalled sequence is mapped to the reference transcripts, correcting some errors made during basecalling. (e) Segmentation step performed by most modification detection methods in order to map the squiggle corresponding to the candidate AC motif before further preprocessing and modification prediction. (f) Our method skips the segmentation step and outputs modification probability corresponding to the candidate positions directly
Stephenson et al. 2022 Sethi et al. 2022). However, modifications such as m6A for example, mostly occur within 18 out of the 1024 possible 5-mer motifs (Meyer et al. 2012, Dominissini et al., 2012 Schwartz et al., 2014b) while other modifications such as m5C or pseudouridine only oceur within segments containing the C or U nuclcotides. Since segmentation algorithms typically segment the entire transcriptome, the modification detection pipeline often requires a huge storage space to store the segmentation results and suffers from slow running time due to the many preprocessing steps required to extract relevant features from the potential modified positions.
2021
In this work we attempt to address these shortcomings by putting together several machine learning techniques that can help to streamline the RNA modification detection process. Firstly, we make use of the deep features learnt by the CTC basecaller, with the aim of integrating modification detection basecalling process in the future. Secondly, we implement an attention layer between sequence embeddings of the candidate modified positions and the deep CTC features to perform implicit segmentation and feature extraction of the target positions. Finally, address the issue with noisy modification labels, we implement an end-to-end Attention-based Multiple Instance Learning approach on top of the extracted attention features so as to perform robust classification of modified positions. We validate our approach by performing m6A detection task and demonstrate that approach is significantly faster than existing m6A detection methods while achieving comparable performance to the current tfate-of-theart algorithm. Our work contributes to the field of RNA modification by developing a more scalable solution to RNA modification detection and we hope to drive wider adoption of machine learning techniques problems in biology, especially in long read RNA sequencing.
2
METHOD
The direct RNA sequencing workflow involves basecalling of RNA squiggles, followed by alignment of the basecalled results to the transcriptome (Figure and for modification detection, another segmentation step is usually required by most detection algorithms. This step is often necessary as RNA squiggle is noisy and modifications only occur on handful of positions which suggests that most the signals are not useful for detecting RNA modifications. Nevertheless, segmentation algorithms produce a lot of unused segmented signal regions and most detection algorithms require
2

---

Under review as a conference paper at ICLR 2023
further preprocessing steps before predictions can be made, resulting in additional storage overhead and longer compute time. One way to speed up the detection process is to incorporate the modified nucleotides directly to basecalling process but this is significantly more difficult for RNA modification since we usually do not have access to modification labels on a single read level (Linder et al. 2015 Koh et al., (209)). In an attempt process to basecalling, we propose making use of the deep features from the penultimate layer of a RNA basecaller for modification detection. Intuitively, such features contains semantically meaningful representations that correspond to the underlying nucleotides sequence of given RNA molecule and can be generated during the basecalling process.
closer towards streamlining the modification detection
Formally, given a signal chunk a E R', and a reference transcript = (21,...,,N,, 2N_) E {G, A, C. C. associated with the signal E TR.L, our goal is to predict RNA modifications across all possible modified positions in S N₂ within the reference transcript
To do so, we first learn a basecaller that takes x as input and produces a prediction ((x) e {G, A, C, C,T)) for the associated underlying sequence. The predicted underlying sequence s(x:) can subsequently be aligned to the reference transcript z. Crucially, we propose to design the basecaller as the composition of two functions:
1. a feature extractor f TR. - TR..DD that takes as input signal chunk x E RR of length I. 2 and produces representation vector of fixed length T 2 1 and dimension D 2
2. a decoder function 9 RT. + R that takes as input the representation vector (xx) E RT.DD is leveraged (using dynamics programming) to produce a prediction 3(x) E (G, of the underlying nucleotide sequence.
The purpose of learning the basecaller is to identify whether a given signal chunk æ contains candidate modified positions. For example, m6A modifications only occur mostly within the DRACH (D=A, G, or U. R=A or G while H is A, C or U) (Meyer et al. 2012. Dominissini et al.) 2012). and so the basecalling step is crucial identify whether such motit's exist within a given signal chunk, On top of identifying the existence of modified motifs within given signal chunk æ, we are going to make use of the features fx) for modification detection.
Next, for each of the modified positions j within x. we learn a signal-sequence feature extractor h that outputs D-dimensional vector (ax,, E RD that corresponds the representation of position j within the transcript z with respect to the deep features fz). This is analogous to performing segmentation and extracting features from segments that correspond candidate modified positions as described in Figure Te..
Finally, in order to detect RNA modification at position j of transcript z, we collect the set Sz.j that includes all raw signal chunks x; E RR harbouring position of transcript identified through basecalling and mapping. We extract the representation vectors h( x. E RM for all signal chunks æ; E Sa.j and following earlier work Hendra et al. (202T). I) the RNA modification detection problem can then be formulated as Multiple Instance Learning (MIL.) problem (Maron & Lozano. Pérez 1998) that can be approached with modern statistical learning methods (Ilse et al. [2018) Lee ct al. 2019; Cheplygina et al. 2019). The following three subsections describe cach step in more
details.
2.1 BASECALLING
Let s(x) E {G, A, C, C,TYJstee Tyl(())) be the true nucleotide sequence associated with raw signal chunk æ. perfect basecaller will predict the nucleotide sequence s(x) given the raw squiggle chunk æ E R- Furthermore, the nucleotide sequence s(x) is continuous subset of the transcript in a sense that s(x) (Si,..., (|(()))) = (Z)(z).... (s(())) for some index 0 < i(x) < N₂ [s(x)).
Let f LR_ + TTD be a function parameterized by a neural network (such as CNN or RNN) that transforms the squiggle E RL_ to high dimensional representation vector f(x) E RT.D of length and dimension D. Under the CTC approach (Graves et al., 2006), we associate a probability distribution for each of the T high dimensional representation over A, +. where G, A. C, are the four nucleotides and is a gap character. The gap character allows the network to produce variable length output since the signals can represent variable-length nucleotide sequence
3

---

Under review as a conference paper at ICLR 2023
(Silvestre- Ryan & Holmes 2021). For this purpose, consider function g RRT R that
transform the representation vector f()) TRT.D into sequence of probability vectors. More
specifically and slightly abusing notations by identifying G,T, [1,,,,
21,2,344, 5). the vector
(x) = 9 f fx) ) R4 is such that
T (<>) [t, A] it, CJ + (())(,, G ( A) + (t)
for any index 1 < t v T. other words, at any position I t Σ the vector x [t)],,, R. represents a probability distribution over C,T,
The gap characters introduce a many-to-one mapping between the possible path including the gap character and valid nucleotide sequences. For example, the path GeeeCCTT is mapped to the valid nucleotide sequence GACCT. We denote this mapping with the operator B in the sense that, for example, A = GACCT. For given nucleotide sequence & E (A, C. C. consider the set of sequences {A, C, G, T, e) that maps in the sense that ( This allows one to define the probability p(s x)) E (0, 1) of the nucleotide sequence s given the raw
input E R_ by setting
II II 5,
p(s x))
Σ =B(3)=
(I)
The predicted nucleotide sequence associated to the raw signal chunk E RR is obtained by maximizing the score over all possible valid nucleotide sequences (A, (A,C,
((x) = argmax p(s x) ss is valid nucleotide sequence }
(2)
All these operation can be efficiently approximated with dynamic programming techniques. Since equation] T describes a likelihood function, the model can be trained by maximum likelihood estima- tion (Graves al.) (0066). We train basecaller following this methodology using the open source Bonito model
with sequence ground truth obtained from Nanopoli sh eventall ign (Loman et al. 2015).
2.2 EXTRACTION OF SIGNAL-SEQUENCE REPRESENTATION
DNN models are able to learn effective representations that can capture essential aspects of the data domain. Networks trained for object detection, for example, are often trained on image classification tasks for which larger datasets are available Girshick al. (2014). As a result, features from the can easily be adapted to different
deeper layer of the networks tend to more informative tasks Mikolov al. (2013); Donahue et al. y014); Devlin et al. (2018). Similarly. while RNA modifications are rare, sequence labels for basecalling training are abundant CCen et al. (202T) and should therefore provide rich features for RNA modification detection. Here view the basecaller training as a form of pretraining task for modification detection where the deep features from the second last layer of the basecaller network can be used to represent potentially modified nucleotide segments.
Sequence Prediction GGACTGG
transcripts
Signal Module
Decoder
Direst RNA-Sog reads
Printtmii tyy Navv
Sequence
Scaled Dos-Product Attention
Multiple Attention Layer
Medification Probaixiiity
Modile
ACTEG
Figure Overview of the model. The signal module comprises CNN layers followed by LSTM layers while the sequence module R comprises 3 bidirectional LSTM layers followed by linear layer. The scaled dot product attention combines the output the two modules to produce signal-sequence representation and the CTC decoder outputs sequence prediction from the signal representation. The MIL attention layer takes the signal-sequence representation to output modification probability
4

---

Under review as a conference paper at ICLR 2023
want to extract the M -ddimensionnl signal-sequence representation vector hf(z),, 2.) E RM Our goal here is to predict RNA modification at position j of transcript and so computing is analogous to segmenting the D-dimensional sequential features of (x) and summarizing its features across a high-dimensional segment that corresponds to position
Given the deep features f(x)
RT..
To begin, we calculate the embedding of x at position j by passing K = 10 flanking nu- cleotides around the j-th position using bidirectional LSTM. More specifically, given position j within the reference transcript, consider the sequence [=]j = (zj-K; Rj- K41...... 2j+K) E (A, C,G,G G, TJ,2+22 of K-neighbouring nucleotides. The representation of the position j is ob- tained by passing through the sequence [z]j.K through a bidirectional LSTM denoted as R (A,C, G, + R&M to obtain representation vector [[([=],,)) R.
Afterwards, we compute the representation h(x, =, EM of the j-th position within the reference transcript z and whose signal is present within the raw signal chunk æ E IR with a standard scaled dot-product attention mechanism (Vaswani et al. (017))
h(x, 2) Attention [R((()),,) [():)Wk), ((()) R.
for query [((=]5,k) E RM, key f(x) E RT.M and value f()) IR..M. The projection matrices WK E RD, and Wy R.M are learnable parameters of the attention mechanism. For query Q E M key K E RTM and value V E RT',M, the attention mechanism (Nuu et al. 2021 is defined as
T Σ 1=1
VQ VM
Attention(Qu, V),
Ve,
where
= softmax
E
RT
(4)
for any coordinate 1 S m S M. Here, we reason that the signal representation f(x) E RTD.D where the feature extractor f R- + RT.D has been obtained when training the basecaller, can be informative since it is trained to maximize the probability of observing the underlying true nucleotide sequence s(x) E (A, (A,C,CC C, T) Intuitively, the attention concentrates the representation of the signal (ar) on a sub-sequence that is the most similar to the positional representation IR (R(()), of the j-th position with the reference transcript 2.
2.2. DETECTING M9A MODIFICATIONS
In order detect m6A modification, we update the parameters of the LSTM-based feature extractor IR while keeping the feature extractor f trained during the basecalling fixed. As empirically demonstrated in Hendra al. (202T). m6A detection can be formulated as a Multiple Instance Learning (MIL) problem (Maron & Lozano-Pedd Pérez 1998). Recall that Sz,j is the set of signal chunks æ, E R that contains some part assoctated to the position j with the reference transcript
Each raw signal chunk a; E S2.j can be associated with a binary label Yi,j {0. 1} indicating whether the part of the signal æ; associated the position j contains m6A modification. However, we do not have access to this label Bi.j because of the lack of resolution in the labelling process: instead, we only have access to a single label y E {0, 1} representing the modification status at position j of the collective signal chunks. We propose extract the collection of signal-sequence M dimmennsional representation vectors [h 2)] m....., and pool them following the Attention- based Deep MIL framework of Isse et al. (0018). Set
H(j,))= = Σ az M(I,, E RM
(5)
E
where the probability vector (u,e),, is defined as a = softmax (u tanh (Vh(z, x.))) S.ES..)
RR S2,1
(6)
The matrices U E RI.H and V E H.H are learnable parameters, The coefficient az; E (0,
measures the relative contribution of the raw signal chunk x; in the collective representation H(j, z). The representation vector H(j, x) IR M can then be used as feature vector within standard logistic regression classifier. The complete model is trained end-to-end by minimizing the cross-entropy loss with stochastic gradient descent.
5

---

Under review as a conference paper at ICLR 2023
Dataset HCT1 16
Mean Acc
Median Acc
loU
91.2%
93.6%
88.1%
HEK293T
90.1%
93.5%
86.0
Table I: Basecalling Accuracy and Intersection over Union on HCT116 and HEК293T datasets
EXPERIMENTS
3.1 DATASETS
HCT116 direct RNA Sequencing Data: The HCTI cell line direct RNA sequencing data was provided by the SG-NEX project (Chen al. 202T). The dataset was spit on the gene level into train, validation, and test sets. We use the training set to train networks for basecalling and m6A detection and use the validation set for model selection. The dataset also comes with m6A labels generated using the m6ACe-seg protocol (Koh al., 2019). We follow the training procedure of Hendra et al. (2021) where we restrict our training data to sites harbouring DRACH motifs.
The HEK293T cell line direct RNA sequencing data is provided by (Pratanwanich et al., (02I). The m6A labels for this dataset was generated by m6ACE-seq (Koh al.J (2099) and mi CLTP (Linder al. 2015). We use this dataset to validate both the basecalling performance as well as m6A classification performance. Similar to the HCTI 16 dataset, we restrict our prediction the DRACH sites(D=A, G, or R=A G while H is A, C or U).
HEК293T direct RNA Sequencing Data:
3.2 BASECALLING AND MAPPING ACCURACY
We first evaluate the accuracy of our basecaller and whether it identify positions within raw signal chunks correctly by aligning the predicted sequence to its underlying reference label using the Smith- Waterman algorithm (Smith et al. 1981). We measure the mapped accuracy as:
Number of Matched Bases
Accuracy Number of Reference Bases
In order to extract accurate feature representation, need to identify whether the position wish to model exists within a given read chunk. To do this, we align each predicted sequence Sæ;; to its reference transcript z; and obtain set of predicted positions L, spanned by read The alignment step serves to correct small error in the prediction and so we do not necessarily need a very high basecalling accuracy to correctly identify the positions spanned by a given signal chunk. As such, we also measure the Intersection over Union (IoU) of the predicted positions L, against the ground-truth transcript positions L; represented in read i. This is given by:
loU 2, n La
L, U
On the HCT1 16 cell line, we manage achieve a mean accuracy 91.2% and median accuracy of 93.6% while on the HEK293T cell line, it achieves mean accuracy of 90.10 1% and median accuracy of 93.5%. Additionally, the model achieves an average loU 88. 1% on the HCTI 16 cell line 86.0% on the HEK293T cell line. This indicates that the model can recognize the underlying sequence of each signal chunk and its mapped alignment coincides strongly with the ground truth label. Another way to improve the mapping quality will be to consider the outputs from two adjacent overlapping signal chunks, strategy implemented by the original Bonito basecaller (Silvestre- Ryan & Holmes) 2021), which we leave for future work.
3.3 COMPARISON OF M6A DETECTION AGAINST EXISTING METHODS
Here we demonstrate the effectiveness of our approach by training our model to perform m6A detection based on the extracted signal-sequence representation. We name approach rawraw and

---

Under review as a conference paper at ICLR 2023
vala 40%
Figure 3: Run time comparison of the models HCT116 cell line
DRACH k-mers
DRACH k-mers
4 DRACH k-mers
1.0
1.0
1.0
True Positive Rate
0.8
True Positive Rate
0.8
Positive Rate
0.8
m6anet
0.4
0.4
0.4
EpiNano (0.78)
Epiñano (0.79)
m6araw (0.93)
m6araw (0.92)
EpiNano (0.78)
0.2
m&anet (0.92)
0.2
m6anet (0.91)
EpiNano (0.76)
Tomba (0.76)
Tembo (0,77)
0.0
EpiNano 5 (0.74)
0.0
nanam6A (0.79)
0.0
MINES (0.79)
0.0
0.2 0.4 0.6 0.8 1.0
0.0
0.2 0.4 0.6 0.8 1.0
0.0
0.2 0.4 0.6 0.8 1.0
False Positive Rate
False Positive Rate
False Positive Rate
m6araw (0.39)
m6araw (0. 62)
m6araw (D.50)
1.0
méanet
1.0
m6anet (0.57
1.0
m6anet (0.54)
Tombo
Tombo (0.30)
Tombo (0.28)
0.8
EpiNano (0.13)
EpiNano (0.21)
0.8
neloomA (0.36)
MINES (0.34)
EpiÑano (0.18)
Precision
0.6
EpiÑano (0.23)
Predisoon
5066
Precision
0.6
EpiNano (0.17)
0.4
0.4
0.4
0.2
0.2-
0.2
0.0
0.0
0.0
0.0
0.2
0.4 Recall
0.6
0.8
1.0
0.0
0.2
0.4
0.5 0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Recall
Recall
Figure 4: ROC Curves and PR Curves of m6Araw and other existing approaches HCT116 16 cell line
compare the runtime as well as ROC-AUC and PR-AUC several partition of the HCTI 16 against
several existing methods (Stoiber et al. Lorenz et al. 2020 Gao et al. 2021 Liu et al. 2021 Hendra et al. 2021) for m6A detection m6A modifications on all DRACH motifs, we compare m6Araw separately with these methods on the subset of the HCT116 16 test set and the HEK293T dataset that contain motifs required by each approach.
detailed in Hendra al. (021)) Since not all methods detect
Firstly, we note that A6Araw has the fastest running time out of all other methods on the HCTI with dataset (3) with just one GPU, roughly times faster than m6Anet, the current state-of-the-art method for m6A detection. On the other hand, the EpiNano EpiNano EpiNano and EpiNano models could not be run without crashing our machine with 126 GB of RAM. As such, we have to modify the code for the EpiNAno models discarding all non- DRACH motif motifs during preprocessing step so as to run it successfully. We do not have the run time comparison for the HEK293T cell line since some of the methods require segmentation algorithms that do not work with the HEK293T file format and additional run time for file conversion is required.
the HCTI 16 dataset, m6Araw perform comparably (ROC-AUC: 0.930, PR-AUC: 0.385, ROC- AUC: 0.927, PR-AUC:0.609 ROC-AUC:0.99 917, PR-AUC:0.499 against m6Anet (ROC-AUC: 0.926. PR-AUC::.4551 ROC-AUC: 0.916, PR-AUC:0.565, ROC-AUC:0.900 908, PR-AUC:0.5433 while outper- forming all other methods (Figure Table 2). We observe similar results in the HEK293T cell line where our model outperform existing methods (ROC-AUC: 0.818, PR-AUC: 0.319. ROC-AUC: 0.812, PR-AUC:0.333 3333, ROC-AUC:0.796 7.6, PR-AUC:0.389) and perform comparably against m6Anet (ROC-
7

---

Under review as a conference paper at ICLR 2023
5-mer Motifs
Model
ROC- AUC
PR-AUC (HCT116)
ROC-
PR-AUC (HEK293T)
AUC
(HCTI16)
(HEK293T)
8818 motifs
m6Araw (ours)
0.930
0.385
0.818
0.319
m6Anet
0.926
0,451
0.838
0.366
Tombo
0.707
0.121
0.507
0.0857
EpiNano
0.776
0.206
0.710
0.240
EpiNano
0.788
0.133
0.725
0.182
EpiNano 3
0.781
0.176
0.722
0.213
EpiNano
0.764
0.235
0.704
0.227
EpiNano 5
0.736
0.167
0.670
0.170
4 motifs
m6Araw (ours)
0.927
0.620
0.812
0.333
m6Anet
0.916
0.565
0.837
0.373
Tombo
0.759
0.296
0.504
0.099
nanom6a
0.787
0.364
0.719
0.203
444 motifs
m6Araw (ours)
0.917
0.504
0.796
0.389
m6Anet 0.908
0.543
0.825
0.440
Tombo 0.767
0.280
0.515
0.166
MINES 0.792
0.340
0.708
0.326
Table Performance comparison of m6Araw against existing m6A detection methods
AUC: 0.838, PR-AUC: 0.366, ROC-AUC: 0.837, PR-AUC:0.373, ROC-AUC.0.82) PR-AUC:0.440) (Figure 5) Table 2) These results suggest that our approach can produce informative signal-sequence representation to detect m6A modifications and is competitive against existing segmentation-bant approaches.
3.4 ROBUSTNESS OF M6A PREDICTION DESPITE LABEL NOISE
The modifications labels used to train and validate m m6Araw predictions on both HCT116 and HEК293T datasets often fail to capture genuine modified sites. Different protocols detect m6A
All DRACH k-mers
12 DRACH k-mers
4 DRACH k-mers
1.0
1.0
True Positive Rate
Positive Rate
0.8
0.6
0.6
True Positive
0.6
méanet (0.84)
m6araw (0.82)
Tombo (0.51)
0.4
0.4
EpiNano (0.71)
EpiNano (0.72)
m6araw (0.81)
msaraw (0.80)
0.2
EpiNano (0.72)
méanet (0.84)
0.2
m6anet (0.82)
EpiNano (0.70)
Tombo (0 50)
Tombo (0.51)
0.0
EpiÑano
0.0
nanom6A (0.72)
0.0
MINES (0.71)
0.0
0.2 0.4 0.6 0.8
0.4 0.6 0.8 1.0
0.0 0.2 0.6 0.8 1.0
False Positive Rate
False Positive Rate
False Positive Rate
m6araw (0.32)
msaraw (0.33)
m6araw (0.39)
1.0
méanet (0.37)
1.0
m6anet (0.37)
1.0
mmanet (0.44)
Tambo (0.09)
Tombo (0.10)
Tombo (G.17)
0.8
EpIÑano 1 (0.24)
EgiÑano 2 (0.18)
0.8
nanom6A (0.20)
0.8
MINES (0.33)
Precision
0.6
EpiNano 4 (0.23)
EpiNano 3 (0.21)
50.6
Precision
60.6
Precision
0.6
EpiNano 5 (0.17)
0.4
0.4
0.4
0.2
0.2
0.2
0.0
0.0
0.0
0.2
0.6 Recall
0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Recall
Recall
Figure 5: ROC Curves and PR Curves of m6Araw and other existing approaches on HEК293T cell
line
8

---

Under review as a conference paper at ICLR 2023
Density (preditted m5A sites)
HCT116
Density (predicted m6A sites)
HEК293T
mnAraw
máArom (+ KD sensitive)
1.0
Beckgrourd
beckground
0.8
0.7
0.6
0.6
0.5
0.4-
0.4
0.4
0.2
0.2-
0.3
0.2-
0.0
0.0
1000
2000
3000
4000 5000
S'UTR
CDS
3'UTR
S'UTR
CDS
3'UTR
Number of Top Positions
Figure 6: Metagene plots of m6Araw prediction on test sets of HCT116 and HEK293T and precision of the top predicted sites for m6Araw before and after METTL3-KO
modifications for example have reported different modified sites (Grozhik & Jaffrey 2018 Koh al. 2019). This limitation will affect both training and validation of any models trained detect m6λ modifications as the modification labels will be noisy and likely contain large number of false positives and negatives. Several work has therefore incorporated additional validation criteria based on the known biology of m6A modification. This involves checking for significant deviation in the signal intensity of wild type samples against samples in which METTL-3, a known m6A writer, gene has been knocked out (Lorenz et al. 2020) Hendra et al. 2021). Additionally, one can also check for the distribution of m6A predicted sites along each transcript since m6A modification is known to be enriched around the 3'RR (Linder et al. 2015 Ke et al. 2017)]. Here reason that our approach displays a robustness to label noise by demonstrating that m m6Araw can capture sites not previously labelled by miCLIP (Linder et al. (015)) m6ACe-seq (Koh et al. 2019). We see that m6Araw predicted sites display a strong enrichment towards the 3'UTR in both HCT1 16 and HEK293T cell lines(Figure 6). Furthermore, incorporating METTL3 sensitive sites modified sites result in roughly 20% increase in the model precision for the top predicted sites, suggesting that our previous precision is underestimated.
4 DISCUSSION
In recent years, we have seen an increasing number of computational tools being developed to detect RNA modifications from direct RNA sequencing data. These tools have facilitated growing number of studies into RNA modifications but at the same time require lot of compute resources. Our study explores the possibility of streamlining such processes by avoiding extra segmentation and preprocessing steps, and instead, detect RNA modifications directly from raw signals. We demonstrate that our approach can produce informative signal- sequence representations to detect m6A modifications, achieving competitive performance against state-of-the- method in m6A detection. Furthermore, we have also demonstrated the feasibility of integrating modification detection to basecalling by performing modification detection using features generated by the second last layer of our trained basecaller. In the future we hope combine RNA basecalling along with detection of other RNA modifications. Lastly, we hope that this work can lay the foundation for further study into representation learning in the context of detecting RNA modifications directly from raw signals.
REFERENCES
Oguzhan Begik, Morghan C Lucas, Leszek P Pryszcz, Jose Miguel Ramirez, Rebeca Medina, Ivan Milenkovie, Sonia Cruciani, Huanle Liu, Helaine Graziele Santos Vieira, Aldema Sas-Chen John Mattick, Schraga Schwartz, and Eva Maria Novoa. Quantitative profiling of pseudouridylation dynamics native RNAs with nanopore sequencing. Nat. Biotechmol,, 1988) 1291, October 2021.
Vladimír Boža, Broña Brejová, and Tomáš Vinař. Deepnano: deep recurrent neural networks for base calling in minion nanopore reads. PloS one, 126677017775, 2017.

---

Under review as a conference paper at ICLR 2023
Thomas M. Carlile, Maria F. Rojas-Duran Boris Zinshteyn, Hakyung Shin, Kristen M. Bartoli, and Wendy V. Gilbert. Pseudouridine profiling reveals regulated mmn pseudouridylation in yeast and human cells. Nature, (455(7555):133111 146, 2014. doi: 1..000//ature118800
Kai Chen, Zhike Lu, Xiao Wang, Ye Fu, Guan Zheng Luo, Nian Liu, Dali Han. Dan Dominissini, Qing Dai, Tao Pan, and Chuan He. High-resolution n(6) -methyladenosinn (m(6) a) map using aaano-crossiininn-assittt m(6) sequencing. Angew. Chem. Int. Ed Engl., 54(5):1587-1590 January 2015.
Ying Chen, Nadia M Davidson, Yuk Kei Wan, Harshil Patel, Fei Yao, Hwee Meng Low, Christopher Hendra, Laura Watten, Andre Sim. Chelsea Sawyer, Viktoriia lakovleva, Puay Leng Lee, Lixia Xin, Hui En Vanessa Ng, Jia Min Loo, Xuewen Ong. Hui Qi Amanda Ng. Jiaxu Wang. Wei Qian Casslynn Koh, Suk Yeah Polly Poon, Dominik Stanojevic, Hoang-Dat Tran, Kok Hao Edwin Lim, Shen Yon Toh, Philip Andrew Ewels, Huck- Hui Ng. N Gopalakrishna Iyer, Alexandre Thiery, Wee Joo Chng, Leilei Chen, Ramanuj DasGupta, Mile Sikic, Yun-Shen Chan, Boon Ooi Patrick Tan, Yue Wan, Wai Leong Tam, Qiang Yu, Chiea Chuan Khor, Torsten Wüstefeld, Ploy N Pratanwanich, Michael I Love, Wee Siong Sho Goh, Sarah B Ng, Alicia Oshlack, Jonathan Göke, and SG-NEx consortium. A systematic benchmark of nanopore long read RNA sequencing for transcript level analysis in human cell lines. April 2021.
Veronika Cheplygina, Marleen de Bruijne, and Josien PW Pluim. superrire:::::: survey of semi-supervised, multi-instance, and transfer learning in medical image analysis. Medical image analysis, 54:280-296, 2019.
Waldo E Cohn and Elliot Volkin. Saaeossdde--PPhoshatte from ribonucleic acid, 1951.
Frank F. Davis and Frank Worthington Allen. Ribonucleic acids from yeast which contain fifth nucleotide. Journal of Biological Chemistry, 227(2)907--915, 1957. doi: 10.1016/:00211-9558(1 70770-9.
Jacob Devlin, Ming-Woi Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint 1rXiv:1810.04805, 2018.
David Dierks, Miguel Angel Garcia-Campos, Anna Uzonyi, Modi Safra, Sarit Edelheit, Alice Rossi, Theodora Sideri, Radhika A Varier, Alexander Brandis, Yonatan Stelzer, Folkert van Werven, Ruth Scherz-Shouval, and Schraga Schwartz. Multiplexed profiling facilitates robust m6a quantification at site, gene and sample resolution. Nat. Methods, September 2021.
Dan Dominissini, Sharon Moshitch-Moshkoviee Schraga Schwartz, Mali Salmon-Divon. Lior Ungar, Sivan Osenberg, Karen Cesarkas, Jasmine Jacob-Hirsch Ninette Amariglio, Martin Kupiec, Rotem Sorek, and Gideon Rechavi. Topology of the human and mouse m6a RNA methylomes revealed by m6a-seq. Nature, 485(7397):201-25) April 2012.
Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng, and Trevor Darrell. Decaf: deep convolutional activation feature for generic visual recognition. In International conference machine learning, pp. 647-655, PMLR, 2014.
Yubang Gao, Xuqing Liu, Bizhi Wu, Huihui Wang, Feihu Xi, Markus V Kohnen, Anireddy S N Reddy, and Lianfeng Gu. Quantitative profiling of n-methyladenosine at single-base resolution in deem-iiffrrentiating xylem of populus trichocarpa using nanopore direct RNA sequencing. Genome Biol., 22(1):22, January 2021.
Daniel R Garalde, Elizabeth A Snell, Daniel Jachimowicz, Botond Sipos, Joseph H Lloyd, Mark Bruce, Nadia Pantic, Tigist Admassu, Phillip James, Anthony Warland, Michael Jordan, Jonah Ciccone, Sabrina Serra, Jemma Keenan, Samuel Martin, Luke McNeill, E Jayne Wallace, Lakmal Jayasinghe, Chris Wright, Javier Blasco, Stephen Young. Denise Brocklebank, Sissel Juul, James Clarke, Andrew Heron, and Daniel Turner. Highly parallel direct RNA sequencing on an array of nanopores. Nat. Methods, 15(3):201-206 March 2018.
Ross Girshick, Jeff Donahue, Trevor Darrell and Jitendra Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 580-587, 2014.
10

---

Under review as a conference paper at ICLR 2023
Alex Graves, Santiago Fernández, Faustino Gomez, and Jürgen Schmidhuber. Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In Proceedings of the 23rd international conference Machine learning, pp. 369-376, 2006.
Anya V Grozhik and Samie R Jaffrey. Distinguishing RNA modifications from noise in epitranserip- tome maps. Nat. Chem. Biol., 14(3):215-225, February 2018.
Christopher Hendra, Ploy N Pratanwanich, Yuk Kei Wan, WS Sho Goh, Alexandre Thiery, and Jonathan Goeke. Detection of m6a from direct ma sequencing using multiple instance learning framework. bioRxiv, 2021.
Tao Huang, Wanying Chen, Jianheng Liu, Nannan Gu, and Rui Zhang. Genome-wide identification of mrna 5-methylcytosine in mammals. Nature Structural Molecular Biology, 26(5):380-388 2019.
Shobbir Hussain, Jelena Aleksic, Sandra Blanco, Sabine Dietmann, and Michaela Frye. Charac- terizing 5-methyleytosine the mammalian epitranscriptome. Genome biology, 14(11):1-10, 2013.
Maximilian Ilse, Jakub M Tomezak, and Max Welling, Attention-based deep multiple instance learning. February 2018.
Shengdong Ke, Endalkachew A Alemu, Claudia Mertens, Emily Conn Gantman, John J Fak, Aldo Mele, Bhagwattie Haripal, Ilana Zucker-Scharff, Michael Moore, Christopher Y Park, Cathrine Broberg Vågbø, Anna Kusšnierczyk, Ame Klungland, James E Darnell, Jr, and Robert B Darnell. majority of m6a residues are in the last exons, allowing the potential for 3 UTR regulation. Genes Dev., 29(19) October 2015.
Shengdong Ke, Amy Pandya-Jones, Yuhki Saito, John J Fak, Cathrine Broberg Vågbø, Shay Geula, Jacob H Hanna, Douglas L Black, James E Darnell, and Robert Darnell. m6a mRNA mod- ifications are deposited in nascent pre-mRNA and are not required for splicing but do specify cytoplasmic turnover, 2017.
John W Kemp and Frank Worthington Allen. Ribonueleic acids from pancreas which contain new components, 1958.
Casslynn W Q Koh, Yeek Teck Goh, and W S Sho Goh. Atlas of quantitative single-base-resolution n6-methyl-adennnn methylomes, 2019.
Juho Lee, Yoonho Lee, Jungtack Kim, Adam Kosiorek, Seungjin Choi, and Yee Whye Tch. Set trans- former: A framework for attention- attention-based permutation -nniatiiiiinnniianntttt neural networks. In International Conference on Machine Learning, pp. 3744-3753. PMLR, 2019.
Adrien Leger, Paulo Amaral, Luca Pandolfini, Charlotte Capitanchik, Federica Capraro, Isaia Barbieri, Valentina Migliori, Nicholas M Luscombe, Anton J Enright, Konstantinos Tzelepis, Jernej Ule, Tomas Fitzgerald, Ewan Birney, Tommaso Leonardi, and Tony Kouzarides. RNA modifications detection by comparative nanopore direct RNA sequencing. November 2019.
Bastian Linder, Anya V Grozhik, Anthony O Olarerin- Olarerin-George, Cem Meydan, Christopher Mason, and Samie R Jaffrey. panelc--muclooiide-rsslttie mapping of m6a and m6am throughout the transcriptome. Nat. Methods, 12(8):667-72, August 2015.
Huanle Liu, Oguzhan Begik, and Eva Maria Novoa. EpiNano: Detection of ma RNA modifications using oxford nanopore direct RNA sequencing. Methods Mol. Biol., 0298:31-52. 2021.
Nian Liu, Qing Dai, Guanqun Zheng, Chuan He, Mare Parisien, and Tao Pan. N6-methyladenosine dependent RNA structural switches regulate RNA-protein interactions, 2015.
Nicholas J Loman, Joshua Quick, and Jared T Simpson. A complete bacterial genome assembled de novo using only nanopore sequencing data. Nat. Methods, 12(8):733-735, August 2015.
Daniel A Lorenz, Shashank Sathe, Jaclyn M Einstein, and Gene W Yeo. Direct RNA sequeneing enables ma detection in endogenous transcript isoforms at base-specific resolution. RNA, 26(1): 19-28, January 2020.
11

---

Under review as a conference paper at ICLR 2023
Alexander F Lovejoy, Daniel P Riordan, and Patrick O Brown. Transcriptome-wit wide mapping of pseudouridines: pseudouridine synthases modify specific mrnas in s. cerevisiae. PloS one, 910): c110799, 2014.
Oded Maron and Tomás Lozamo-Péres A framework for Multiple-Instanne Instance learning. In M I Jordan, M J Kearns, and S A Solla (eds.), Advances in Neural Information Processing Systems 10, pp. 570-576. MIT Press, 1998.
Kate D Meyer, Yogesh Saletore, Paul Zumbo, Olivier Elemento, Christopher E Mason, and Samie R Jaffrey. Comprehensive analysis of mRNA methylation reveals enrichment in 3 UTRs and near stop codons, 2012.
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado. and Jeff Dean. Distributed representations of words and phrases and their compositionality. Advances in neural information processing systems, 26, 2013.
Benoit Molinie, Jinkai Wang, Kok Scong Lim, Roman Hillebrand, Zhi- Chi-Xiang Lu, Nicholas Van Wit- tenberghe, Benjamin D Howard, Kaveh Daneshvar, Alan C Mullen, Peter Dedon, Yi Xing, and Cosmas C Giallourakis. m6A-LAIC-seg reveals the census and complexity of the m6a epitran- scriptome, 2016.
Zhaoyang Niu, Guoqiang Zhong, and Hui Yu. A review the attention mechanism of deep learning. Neurocomputing, 452:48661 62, 2021.
Paz Nombela, Borja Miguel-López, and Sandra Blanco. The role of móa, m5c and e ma modifications in cancer: Novel therapeutic opportunities. Molecular Cancer, 20(1):1-30, 2021.
Matthew T Parker, Geoffrey J Barton, and Gordon G Simpson. Yanocomp: robust prediction of m6a modifications in individual nanopore direct ma reads. bioRxiv, 2021.
Ploy N Pratanwanich, Fei Yao, Ying Chen, Casslynn W Q Koh, Yuk Kei Wan, Christopher Hen- dra, Polly Poon, Yeek Teck Goh, Phoebe L Yap. Jing Yuan Chooi, Wee Joo Chng. Sarah B Ng. Alexandre Thiery, W S Sho Goh, and Jonathan Göke. Identification of differential RNA modifications from nanopore direct RNA sequencing with xpore. Nat. Biotechnol,, July 2021.
Schraga Schwartz, Douglas A Bersstenn, Maxwell R Mumbach, Marko Jovanovic, Rebecca H Herbst, Brian X Ricardo, Jesse M Engreitz, Mitchell Guttman, Rahul Satija, Eric S Lander, et al. Transcriptome- -iide wide mapping reveals widespread dynamic-regulated pseudouridylation of nema and mrna. Cell, 159(1): 159(1))148-16) 162, 2014a.
Schraga Schwartz, Maxwell R Mumbach, Marko Jovanovic, Tim Wang, Karolina Maciag, G Guy Bushkin, Philipp Mertins, Dmitry Ter-Ovanesyan, Naomi Habib, Davide Cacchiarelli, et al. Perturbation of m6a writers reveals two distinct classes of mrna methylation internal and 5 sites. Cell reports, 8(1):284-266, 2014b.
AJ Sethi, M Guarnacci, A Ravindran, Srivastava, J Xu, K Woodward, W Hamilton, Gao, LM Starrs, R Hayashi, et al. Identification of m6a and m5c ra modifications at single-molecule resolution from nanopore sequencing. 2022.
Jordi Silvestre-Ryan and Ian Holmes. Pair consensus decoding improves accuracy of neural network basecallers for nanopore sequencing. Genome biology, 22(1):1-6, 2021.
Temple F Smith, Michael S Waterman, et al. Identification common molecular subsequences.
Journal of molecular biology, 147(1):195-197, 1981.
Jeffrey E Squires, Hardip R Patel, Marco Nousch, Tennille Sibbritt, David T Humphreys, Brian Parker, Catherine M Suter, and Thomas Preiss. Widespread occurrence of 5-methylcytosine in human coding and non-coding rna Nucleic acids research, 40(1 (0011)5333-5033 2012.
William Stephenson, Roham Razaghi, Steven Busan, Kevin M Weeks, Winston Timp, and Peter Smibert. Direct detection of ma modifications and structure using single- single-molecule nanopore sequencing. Cell genomics, 2020:100097, 2022.
12

---

Under review as a conference paper at ICLR 2023
Mareus Stoiber and James Brown. Basecrawller: streaming nanopore basecalling directly from raw signal. BioRxiv, pp. 133058, 2017.
Marcus Stoiber, Joshua Quick, Rob Egan, Ji Eun Lee, Susan Celniker, Robert K. Neely, Nicholas Loman, Len A Pennacchio, and James Brown. De novo identification of DNA modifications enabled by Genome-Guided nanopore signal processing.
Haotian Teng, Minh Duc Cao, Michael B Hall, Tania Duarte, Sheng Wang, and Lachlan JM Coin. Chiron: translating nanopore raw signal directly into nucleotide sequence using deep learning. GigaScience, 7(5):iiy37,, 2018.
Hiroki Ueda. nanodoc: RNA modification detection using nanopore raw reads with deep One-Class classification.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural information processing systems, 30, 2017.
Yuk Kei Wan, Christopher Hendra, Ploy N Pratanwanich, and Jonathan Göke. Beyond sequencing: machine learning algorithms extract biology hidden nanopore signal data. Trends Genet., 38(3): March 2022.
Kai Xu, Ying Yang, Gui-Hai Feng, Bao-Fa Sun, Jun-Qing Chen, Yu-Fei Li, Yu-Sheng Chen, Xin-Xin -in Zhang, Chen-Xin Wang, Yuan Jiang, Chao Liu, Ze- Yu Zhang, Xiu-Jie Wang, Qi Zhou, Yun-Gui Yang, and Wei Li. Mettl3-mediated m6a regulates spermatogonial differentiation and meiosis initiation, 2017.
Eliza Yankova, Wesley Blackaby, Mark Albertella, Justyna Rak, Etienne De Braekeleer, Georgia Tsagkogeorga, Ewa S Pilka, Demetrios Aspris, Dan Leggate, Alan Hendrick, Natalie A Webster, Byron Andrews, Richard Fosbeary, Patrick Guest, Nerea Irigoyen, Maria Eleftheriou, Malgorzata Gozdecka, Joao M Dias, Andrew J Bannister, Binje Vick, Irmela Jeremias, George S Vassil- iou, Oliver Rausch, Konstantinos Tzelepis, and Tony Kouzarides. METTL3 as a strategy against myeloid leukaemia. Nature, April 2021.
inhibition of
Jingwen Zeng, Hongmin Cai, Hong Peng, Haiyan Wang, Yue Zhang, and Tatsuya Akutsu. Causalcall: nanopore basecalling using a temporal convolutional network. Frontiers in Genetics, pp. 1332, 2020.
A APPENDIX
You may include other additional sections here.
13


Reviews:
[{"review_id": "vwWW3abLNSQ", "reviewer": "Reviewer_FDNP", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper works on the problem of RNA sequencing. This problem in the nutshell is to go from the electric signal x to the sequence of the nucleotides z. In particular, the paper considers the sequences that contain some particular RNA modifications. In order to tackle the modification, one needs to adjust the nucleotide sequence by making some changes.\n\nThe paper proposes to use several methods: the CTC loss for the alignment of the input to the output; the attention to perform the modification; and the multiple instance learning to train a model for instance modification.", "summary_of_the_review": "In general I liked the proposed application in this work, but I have to reject it for this conference because the novelty for the machine learning community is low. The secondary reason for rejection is that some crucial parts of the paper are hard to follow.", "strength_and_weaknesses": "# Strengths\n\nThe main strength of this paper seems to be the experimental evaluation. While I'm not an expert in the area, I cannot thoroughly review this part. Nevertheless, it seems that the paper conducts a great number experiments with prior and the proposed models on two datasets.\n\nAnother strength is how the paper describes the biology part. While my knowledge here is very low, I was able to follow and understand the problem setup.\n\n# Weaknesses\n\nThe main weakness is the low novelty of the paper from the point of view of the machine learning community. The methods used in the paper are well known and the main novelty seems to be the application to the new task.\n\nThen, it was hard to follow parts of the Section 2. The description of the ML methods used in the paper is very clunky and contains many small mistakes aggravating the issue.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "f_hrhcmbk8", "reviewer": "Reviewer_rbVq", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work proposed a scheme of segmentation-free inference of RNA modifications.\n\nThe scheme includes:\n1. Basecalling of RNA squiggles: transcribe the current signal into RNA nucleotides (G, A, C, U) using the method of connectionist temporal classification (CTC). The representation of the basecalling, say f(x), will be used for the subsequent basecalling model.\n2. Embedding of z at position j: passing K=10 flanking nucleotides around j-th position using bidirectional LSTM.\n3. Compute the final representation: using scale-dot attention between the embedding obtained in stage 2 and the basecalling representation f(x).\n4. Multiple Instance Learning (MIL): use attentive pooling to aggregate multiple signal chunk representations.\n5. RNA modifications detection model training.\n\nCompared to the conventional methods, this work does not require the segmentation process which accelerate the inference.", "summary_of_the_review": "Since the paper emphasizes the advantage of inference speech for the proposed method because of the segmentation-free process, the CPU time results should be added in Figures. This should be a very important factor either as a fair comparison, or for readers' interest.\n\nOverall, this paper uses many machine-learning techniques in other areas for RNA modifications. The segmentation-free scheme is novel. \n\nThis paper is above the acceptance threshold", "strength_and_weaknesses": "Strength:\n\n1. The paper seems to be the first to use segmentation-free methods for RNA modifications.\n2. Well written with enough experimental comparisons.\n\nWeakness:\n1. The authors should mention this is the first one to use segmentation-free methods for RNA modifications somewhere in paper.\n2. Typos:\n    (1). The formula before eq.1, there are two As, should be T for the second A.\n    (2). Section 3.2, s hat x_i, should it be s hat instead?\n3. In Section 3.3, could you please add which table you are describing for each paragraph? It would be easier for readers.\n4. For the results in Figure 3, I am wondering if you can add the results of testing m6Araw with CPU. It is fair to compare other methods using the same configurations. If the methods are used in practice, CPU time would be a more important factor from my perspective.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "Nblc1uYrij", "reviewer": "Reviewer_MYMy", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Authors applied deep learning algorithms to propose a segmentation free algorithm for RNA modification position detection. This study suggested a new tool for analyzing RNA modification position detection with efficient procedures from raw signals.", "summary_of_the_review": "Authors propose to apply deep learning algorithms (as well as MIL framework) for inference RNA modifications which is segment free from raw signal. The study provided a new efficient way for (RNA modification inference) if there are no or few study in this aspect. The idea seemed work well and could provide a new tool for inference of RNA modification.", "strength_and_weaknesses": "Strength: While I am not an expert in RNA analysis and detection, the methods used in this study are widely and popularly applied in image and speech processing for recognition and detection task (such as deep feature extraction, CTC sequence mapping and alignment, attention, as well as the multiple instance learning framework), it is a good idea to apply these techniques for the new task. If there is no such kind of research in this specific field (for inference of RND modifications), I believe that it will be a good start to explore these kind of techniques for this new task, and possibly bring a large convenience for advancing the study in this new field.\nWeakness: Several information are missed for clearly capture the idea and application: 1. A detailed network architecture framework may be drawn to show the signal flow including feature extraction and position detection (CNN and LSTM, layers, number of neurons, and possible effects of different selection of their size).\n2. Training data size, usually in image and speech, the training data size are large, e.g., millions of pictures for images, and thousands of hours of speech. I am not sure for RNA data, how large size of the training sequences. And usually, data of RND modification is rare, how authors deal with this kind of lack of data to balance the training data.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "8gQSkBOW4Z", "reviewer": "Reviewer_NJ1g", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this work the authors present m6araw, a method for detecting m6A modifications directly from raw ONT long reads. As the authors nicely explain, current methods require a two step process: One for translating the raw signal from ONT into RNA sequence, another one for detecting the m6A modifications in those sequence segments. Instead, m6araw performs both tasks simultaneously. One component of the model is trained to do sequence detection while the second part uses the hidden representation from the first part but combines it with additional representation to call the modification sites. The authors evaluate their method on two datasets showing that they perform similar to state of the art methods while enjoying a significant boost in mem/time performance.\n", "summary_of_the_review": "Overall the paper offers a method which uses components from other domains, tailored together in a sensible way to address an important question (detection of RNA modifications) with the main advantage being significant improvement in time/memory compared to existing methods.", "strength_and_weaknesses": "Overall, we found the paper to be much more clear and better written than a previous version we reviewed for NeuIPS. It was much easier to understand how the model is organized, the motivation behind the modeling approach, and the relation to previous work. The authors also added information about comparative time/memory which is crucial to make a claim about significance.\n\nAdmittedly, the proposed method for segmentation free detection of m6A uses sequence to sequence modeling components that by themselves are not novel and commonly applied in other domains (e.g. CTC loss, LSTM, and gap character). Thus, the method itself is not as new as the tailoring of these components to the problem at hand which, as far as we can tell, has not been done. That said, the combination of those components in a sensible way to produce competitive results is not a trivial feat and should be acknowledged, especially with ICLR being focused on learning representations.\n\nGiven the above our overall excitement about the paper is not high yet positive. We believe the specific issues listed below are addressable.\n\n1.\nPlease define the dimensions of the spaces you are dealing with and the relation between them: L, T, D, H. M. What are used sizes of those and why?\n2.\nBasecalling: Authors report results for this task but not other methods’ performance. We understand it’s not the main topic but it would be good to at least have a sense how well the method does on this task.\n3.\nThis question was asked about the previous version as well but was left unanswered: How does the model perform and generalize to unseen datasets that were not included in training? How does the generalization compare with other existing methods? Mateos et al., 2022 follow Yao et al 2021 and make a difference between classification of read signals from kmers context used for training (but read signals not used) aka “sensor generalization” and ability to classify signals from k-mers contexts not seen during training i.e. k-mer generalization. They also include assessment of effects in cases where multiple (and different) modifications may be present.\n4.\nWhat are the motifs in Table 2 left column? How are these params decided upon?\n5.\nWhat is “Background” in Fig 6? How are these results supporting your claims exactly?\n\nMinor comments:\nThe term “deep features” is used repeatedly but not defined.\nThe paper is peppered with small errors, some of which are listed below. The authors should make a concerted effort to scan and remove those.\n\nP2: The squiggle is deciphered a series of \nP3: …. Is leveraged….. \nP4: The output the two modules\nP6: The dataset was spit on (definitely my favorite)\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject