ReNovo: Retrieval-Based \emph{De Novo} Mass Spectrometry Peptide Sequencing
Authors: Shaorong Chen
Jun Xia
Jingbo Zhou
Lecheng Zhang
Zhangyang Gao
Bozhen Hu
Cheng Tan
Wenjie Du
Stan Z. Li
Venue: ICLR 2025 Poster (2025)
Keywords: Peptide Sequencing
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: We introduce ReNovo, a novel Retrieval-based De Novo peptide sequencing methodology that achieves improved performance.

Abstract: Proteomics is the large-scale study of proteins. Tandem mass spectrometry, as the only high-throughput technique for protein sequence identification, plays a pivotal role in proteomics research. One of the long-standing challenges in this field is peptide identification, which entails determining the specific peptide (sequence of amino acids) that corresponds to each observed mass spectrum. The conventional approach involves database searching, wherein the observed mass spectrum is scored against a pre-constructed peptide database. However, the reliance on pre-existing databases limits applicability in scenarios where the peptide is absent from existing databases. Such circumstances necessitate \emph{de novo} peptide sequencing, which derives peptide sequence solely from input mass spectrum, independent of any peptide database. Despite ongoing advancements in \emph{de novo} peptide sequencing, its performance still has considerable room for improvement, which limits its application in large-scale experiments. In this study, we introduce a novel \textbf{Re}trieval-based \emph{De \textbf{Novo}} peptide sequencing methodology, termed \textbf{ReNovo}, which draws inspiration from database search methods. Specifically, by constructing a datastore from training data, ReNovo can retrieve information from the datastore during the inference stage to conduct retrieval-based inference, thereby achieving improved performance. This innovative approach enables ReNovo to effectively combine the strengths of both methods: utilizing the assistance of the datastore while also being capable of predicting novel peptides that are not present in pre-existing databases. A series of experiments have confirmed that ReNovo outperforms state-of-the-art models across multiple widely-used datasets, incurring only minor storage and time consumption, representing a significant advancement in proteomics. Supplementary materials include the code.

Published as a conference paper at ICLR 2025
RENoVO: RETRIEVAL-BASED De Novo MASS SPEC- TROMETRY PEPTIDE SEQUENCING
Shaorong Chen': Jun Xia?i, Jingbo Zhou²*, Lecheng Zhang?, Zhangyang Gao", Bozhen Hu?, Cheng Tan', Wenjie Du', Stan Z. Li 'Zhejiang University, SShooo of Engineering, Westlake University, University of Science and Technology of China
Hangzhou, China
(chenshaorong, xiajun, stan. zq, 11 1i)ewesttakee edu.
ABSTRACT
Proteomics is the large-scale study of proteins. Tandem mass spectrometry, the only segh-thooughuu technique for protein sequence identification, plays a pivotal role in proteomics research. One of the long-standing challenges in this field is peptide identification, which entails determining the specific peptide (se- quence of amino acids) that corresponds to each observed mass spectrum. The conventional approach involves database searching, wherein the observed mass spectrum is scored against pre-constructed peptide database. However, the re- liance on pre-existing databases limits applicability in scenarios where the peptide is absent from existing databases. Such cireumstances necessitate de novo peptide sequencing, which derives peptide sequence solely from input mass spectrum, in- dependent of any peptide database. Despite ongoing udvancements in de novo peptide sequencing, its performance still has considerable room for improvement, which limits its application in large-scale experiments. In this study, we intro- duce novel Retrieval-based De Novo peptide sequencing methodology, termed ReNovo, which draws inspiration from database search methods. Specifically, by constructing datastore from training data, ReNovo retrieve information from the datastore during the inference stage to conduct retrieval- based inference, thereby achieving improved performance. This inmovative approach enables ReN- ovo to effectively combine the strengths of both methods: utilizing the assistance of the datastore while also being capable of predicting novel peptides that are not present in pre-existing databases. A series of experiments have confirmed that ReNovo outperforms state-of-the-art models across multiple widely-used datasets, incurring only minor storage and time consumption, representing a significant ad- vancement in proteomics. Supplementary materials include the code.
INTRODUCTION
In proteomics, field dedicated to studying proteins within living organisms, mass spectrometry serves crucial tool protein identification. The core of this process is resolving the peptide iden- tification challenge involves deciphering the amino acid sequence of peptide from observed mass spectrum (MS) data, illustrated in Figure [ A standard peptide identification workflow can be delineated as follows: Initially, proteins are broken down into their constituent peptides through en- zymatic digestion. The resulting peptides are then separated via liquid chromatography, producing primary scan (MS1) spectra. These spectra reveal the mass-to caca-r (m/z) ratios of the intact pep- tides (also known the precursors). Following this, the peptides undergo fragmentation, yielding secondary scan (MS2) spectra. peptide identification involves inferring the peptide sequence from the observed mass spectrum and precursor, thus facilitating comprehensive proteomic analyses.
As depicted in Figure [] there are currently two mainstream methods employed in peptide identifi- cation: database search methods and de nove peptide sequencing methods.
Equal contribution.
Corresponding author.

---

Published as a conference paper at ICLR 2025
MS2
Data
Peptide Identification Method
Peptide Sequence
Precursor
m/z=...
(1) Databuse Search Methed
(2) De Novo Peptide Sequencing
(3) ReNovo (Ours)
High identification Limited to database
Low identification
High identification Unliiitted to database
V Unlimited to databose
Paptidee datubase
extpet
cetpot
Peptide database
Figure 1: The semantic diagram of database search, novo peptide sequencing, and ReNovo.
Database search methods involve comparing observed mass spectrum against a pre-established peptide database of peptide-spectrum matches (PSMs). Despite its widespread adoption, this ap- proach has significant limitation: constrained identification scope. The database search method inherently restricts the analysis to only those peptides present within the supplied database. In sce- narios where the peptide is not present in any pre-existing database prior knowledge of the peptide database is limited, constructing comprehensive peptide database becomes infeasible.
De novo peptide sequencing represents alternative method that does not rely existing database. This method involves the direct interpretation mass spectrum to infer peptide and is invaluable in scenarios where databases unavailable. Such situations include but not limited to, the sequencing of antibodies (Beslic et al. (003),, the identification of novel antigens (Karun- ratanakul al. 2019), and the sequencing of metaproteomes lacking established databases (Hettich et al. 2013). However, substantial improvements are necessary to fully leverage its potential.
In this work, we have endeavored to synergize the strengths the two aforementioned peptide iden- tification methods by integrating concepts from database search methods into de novo peptide se- quencing task. This innovative approach, which term ReNovo (Retrieval-baat sequencing). is de novo peptide sequencing method that is integrated with a novel retrieval- framework. After the model training stage, ReNovo utilizes the training data build a datastore. During the inference stage, this datastore is queried and the retrieved information will be used for the final prediction. This innovative approach enables ReNovo to effectively combine the strengths of both the database search and de datastore during the inference stage to enhance the performance, while also being able to predict novel peptide sequences that are not present in any pre-existing peptide databases. The connections and differences among these three methods can be seen in Figure
de Novo peptide
peptide sequencing methods: utilizing the assistance of the
Our contributions can be summarized as follows:
1. Superior Performance. We have developed ReNovo, an innovative de novo peptide quencing method that demonstrates superior performance surpassing all the current state- of-the-art models across multiple widely-used datasets and evaluation metrics.
2. Novel Methodology. ReNovo utilizes an innovative "Model Training Stage Datastore Building Stage Retrieval Based Inference Stage" pipeline. To the best of our knowledge, ReNovo is the first-of-its-kind retrieval-based framework for de novo peptide sequencing task. This novel paradigm opens new avenues for proteomics researchers.
3. Low Overhead. Compared to other de novo peptide sequencing methods, ReNovo's 's unique datastore building stage and retrieval-based inference stage only incur minor time and storage consumption, yet result in substantial performance improvement.
2

---

Published as a conference paper at ICLR 2025
2 RELATED WORK
2.1 DATABASE SEARCH METHOD
Database search methods (Eng al. 1994 Perkins et al. 1999 Cox & Mann 2008 involve compar- ing the mass spectrum against escabltabli database. Notably, error-tolerant and open database search methods (Han al. 2004 Renard al. 2012 Kong et al. 2017) enable the identification of peptides with unexpected modifications or mutations. However, the effectiveness of these methods relies on a comprehensive extemal peptide database that encompasses all potential peptides, which limits their applicability in significant scenarios where comprehensive databases are unavailable.
2.2 De Novo PEPTIDE SEQUENCING METHOD
This method involves the direct interpretation of mass spectrum to infer peptide sequence, indepen- dent of any peptide database. Initially, researchers employed graph-based methods (Dančik et al.
1999 Taylor & Johnson 1997). dynamic programming algorithms (Ma et al. 2003) and Hidden Markov Models (Fischer et al.) 2005) to tackle the de novo peptide sequencing task. With the prosperity of deep learning, DeepNovo (Tran et al. 2017) is the first method applying deep neural networks predict the peptide sequence. Recently, Casanovo (Yilmaz et al.) 2022) first employs transformer [Vaswani] 2017) to the task of de novo peptide sequencing. Since then, a series of transformer-basee models (Eloff et al., [2023] Yang et al. 2024 Xia al. 2024) have been proposed to address specific challenges in de novo peptide sequencing. However, the performance of previous models is still far from what required for large-scale applications in proteomics. Additionally, compared to previous models that only use the training set for model training. ReNovo significantly enhances performance by leveraging a datastore constructed from the training data.
2.3 RETRIEVAL-AUGMEER
GENERATION
A series of studies have widely applied retrieval mechanisms machine translation. Notably, the KNN-LM (Khandelwal et al. 2019) and its subsequent adaptations (Khandelwal et al. (2200)(Zhem))
et al. 2021); Martins al. 2022) augment the model with a token-level symbolic database. Re- cently, retrieval-
setrieval-augmentt augmented generation (RAG) has gained success across various tasks including al. 2022):(Ra)))))
al., 2023), question answering (Lewis et al., 2(Aa) et al. 2023) and so on. ReNovo model draws inspiration from them. However, ReNovo novel compared to recent advances in RAG because the datastore retrieved by ReNovo originates solely from training data, rather than from any external or domain-speciff sources as in RAG. This choice aims to simulate well-world de novo applications without reliance on external databases.
language modeling (Borgeaud
METHODOLOGY
The ReNovo model undergoes three sequential stages: (1) Model Training Stage: During this stage, the ReNovo model trained a supervised manner using the training set. (2) Datastore Building Stage: Once the ReNovo model is trained, it generates context feature target amino acid pairs using training data, which are then stored in the datastore.
(3) Retrieval-Based Inference Stage: During inference, when performing next amino acid prediction, the ReNovo model will retrieve the datastore and incorporate the retrieved context feature target amino acid pairs to make the final prediction. We will introduce the task formulation in Section 3.1 followed by detailed stage-by- stage introduction of the three stages in Section 3.2 Section 3.3 and Section 3.4 respectively.
3.1 TASK FORMULATION
Formally, we define the input MS2 data as: s [ (m,, 1 where each peak (m,, I,) is 2-tuple comprising the mass-to-charge ratio (m/z) m, E R and intensity value 1, IR M denotes the number of peaks, which may vary across different MS2 data. We define the precursor as p (Mprec: (pree), where Myree R represents the m/z value of the precursor and Gprec E 21.2..... 10} indicates the charge state of the precursor ion. The peptide is denoted as a sequence of amino acids y { = (y), ....... V, where y; E AA represents the identity of the i-th amino acid and A.A. represents the set of all considered amino acid types. N corresponds
3

---

Published as a conference paper at ICLR 2025
to the peptide length, which may vary among different peptides. We represent the training set as (S, P.)), where S. P and y represent the sets of MS2 data s E S, precursor p E P. and ground truth peptide sequences y Y in the training set, respectively.
The de nove peptide sequencing models are designed to predict the peptide y given MS2 data s and precursor p:
P(y s, p)
where 0 is the parameter. If the model performs autoregressive generation, then the prediction at time step & can be expressed as p(y 91st-1,8, S, pi 0), and the above equation can be further expanded as:
P(y s.
II p(u $1
3.2 MODEL TRAINING STAGE
As shown in Figure the trainable components of ReNovo consist of a MS2 Encoder and a Peptide Decoder, all of which are based on the transformer architecture (Vaswani 2017). order encode LM
MS2 data s = {s; L. n=l into feature vectors { we follow previous methods (Yilmaz et al,, 2022 Xia al. 2024), treating each peak s; (m; I;) "word" in a MS2 "sentence" s. The peak embedding E; is obtained by separately encoding its m/z value m, and intensity value I; into E and E/ then combining them through summation, Formally,
EN
sin
mc
sin
m74
sin
174
COS
di3
CO8
COS
(3)
N
N
N
N1
N1
N1.
E WI:
(4)
F F
(5)
where d denotes the feature dimension, W are user-defined scalars that can be set to any value. Specifically, we set d = 512, N1 mass and
R represents trainable linear layer, N1 and N₂
where mmax = 10, 000 and minn = 0.001 in our work. These features provide a detailed representation of m/z value and help the model attend to m/z differences between peaks.
N2 Miss 2x
After encoding each peak s, of the input MS2 data s = { = {s AM] to obtain the features {E, JMM (E..... ÆM}. ). these feature sequences are then fed into the MS2 Encoder. The Peptide Decoder generates the sequence y autoregressively, producing each amino acid ye through p(y. 91=-1.8,, p) step by step based on the previously predicted amino acids y1a-11- The method for encoding precursors p (Mpree, Cpree) is consistent with the method used for en- coding peaks. During the training stage, ReNovo employs teacher-fore forciig strategy Williams & Zipser (1989), where yı.r-1 corresponds to the ground truth peptide y E Y. The loss function used for training ReNovo is cross-entropy loss:
(())
Σ
log p(ū 91.-1,8, p;
where d represents all the trainable parameters the ReNovo model, dt represents the ground truth target amino acid in y E y.
3.3 DATASTORE BUILDING STAGE
In this section, we will first define the "context feature" and the "target amino acid". Specifically, the ReNovo's prediction at time step ( can be expressed as Equation 7 We refer to (( p; 0*) output by the Peptide Decoder of ReNovo as the context feature, high-ddmensional feature vector. yt is the target amino acid corresponding to the context feature f f p; 8*).
p(v 01--1, 8. p; 0²) p(ve s;;
4

---

Published as a conference paper at ICLR 2025
K Nearest Neighbor
Amino
Distance
Context Feature
Acid
Datastore
10.5
[E]
Probability
MS2 Encoder
Peptide
Probability
Decoder
Distribution
[T]
[A]
[АTENTISSIIG
Context Features
Target Amino Acids Pairs
m/z,charge
[PIEMP]]
Previously Predicted
Predicted Amino Acid
MS2 Data
Precursor
Sequence
Figure 2: The framework of the ReNovo model. The ReNovo model takes MS2 data, the precursor and the previously predicted sequence as inputs, and produces context feature as output. This context feature is then used to query the datastore to yield the final predicted amino acid.
After the model training stage, ReNovo utilizes the pre-trained model P(y 0*) along with the training dataset (S,P,3 P.)) to build the datastore D, where 0* denotes the parameters of the pre- trained ReNovo model. Formally, the datastore 'D can be represented as:
D = (I, (1 Vye y
(8)
In other words, the key-value pairs (AC, V)) stored in the datastore D be interpreted as context feature- target amino acid pairs (f(y)==1, S, p; (r), The context feature / p; 0²) is feature vector generated by the pre-trained ReNovo model, which encapsulates information from the input MS2 spectra s, precursor and the previously predicted peptide sequence V1:1-1- The target amino acid Yi represents the ground truth prediction that the ReNovo model should output under the corresponding context represented by f( S, p; 0*). By constructing such context feature target amino acid pairs datastore 'D ReNovo essentially "memorizes" which target amino acid Y/ to output under various contextual conditions f S, p: f"). Consequently, during the retrieval-based inference stage which will be discussed in the next section, ReNovo can leverage the knowledge stored in the datastore D to make more accurate predictions.
To distinguish the ReNovo model from database search methods, we refer to the database used by the ReNovo model as the "datastore". The datastore of the ReNovo is constructed using the data that is used to train the ReNovo model, thus making inherently de novo peptide sequencing model. This paradigm ensures fair performance comparison with other de peptide sequencing methods. In contrast, database used database search methods are built from external data sources.
3.4 RETRIEVAL-BASED INFERENCE STAGE
During the inference stage, the model can be represented by schematic diagram in Figure 2 ReN- ovo augments the pre- pre-trained model with a retrieval mechanism that enables direct access to a datas- tore D (K. V) during the inference stage. We refer to this inference approach Retrieval- Inference. During the
inference stage, given MS2 spectrum and precursor p in the test dataset, the model generates the sequence y autoregressively, producing each amino acid step
5

---

Published as a conference paper at ICLR 2025
by step. At time step E, the ReNovo model first produces the context feature f(11.t11, s. p; 0*). This context feature is then utilized to query the datastore to find the K nearest neighbors N ((),,o)), (1, K)), where K is integer hyperparameter and (ky, U2) represent the context feature target amino acid pairs retrieved with the Fuclidean distances.
The retrieved context feature target amino acid pairs N {(ky, ")), E (1. K)} are subse- quently transformed into a probability distribution over the amino acids vocabulary AA by:
-u= exp (-d(k), [ddik,,/ynaa 1.,
PENN(Be S..=-)) = Σ
E AA (9)
T
(Kg.D))EN
PLNN (ye
S,
Pmu (yle 11 EByEAA EAA PLNN (yj 11 0*)" E A.A.
(10)
where d(ky. f p; )) denotes the Fuclidean distance between the query context feature (V1--1,, S, p: (*) and the retrieved context feature ky, and T denotes the temperature that is used to control the smoothness of the probabilities. The resulting Psa can be interpreted as the probability generated by ReNovo with the datastore retrieval, which is used to predict amino acid yt-
4 EXPERIMENTS
We have chosen to use several advanced and representative models (Deepnovo Tran et al. 2017) Pointnovo (Qiao et al. 202T), Casanovo (Yilma et al., 2022), Instanovo (Eloff et al.) 2023). He- lixnovo (Ynng et al., 2024). Adanovo (Xia et al. 2024)) as baselines for comparison with ReNovo. This setup ensures thorough and fair assessment of our ReNovo's performance against range of current advanced models. To maintain consistency and facilitate direct comparisons, we have standardized the experimental settings across all models unless indicated otherwise.
4.1 DATASET AND EVALUATION METRICS
For evaluation, we utilize three representative datasets: Seven-species Dataset Tran et al. 2017)
Nine-species Dataset Tran et al. 2017), and HC-PT Dataset(Eloff et al., 2023 These datasets were selected due to their varying sizes and characteristics in terms of resolution and origins. A detailed description of the dataset and analysis of dataset overlap is provided in Appendix A
The evaluation metrics we considered include: (1) Peptide-level precision. This metric acts as the primary measure of model's practical effectiveness. It is defined the proportion of peptides correctly predicted by the model. (2) Peptide-level AUC. It is determined by calculating the area under the peptide-level precision-recall curve, providing a comprehensive evaluation. (3) Amino acid-level precision and recall. These metrics offer more granular assessment of model perfor- mance at the individual amino acid level. They are defined as the precision and recall of predictions across all amino acids in the test set. Further details on can be found in Appendix B
4.2 EXPERIMENT SETUP
To simulate real-world scenarios requiring novel peptide identification, we utilized a leave-one-out approach, akin prior de novo peptide sequencing studies. For example, all the models were trained on data from six species and subsequently tested on the remaining species in Seven-species Dataset. The same applies to the other datasets. In the experimental section, we do not compare performance with database search methods. On one hand, ReNovo is inherently a de novo method, thus comparisons with similar de novo models more appropriate. On the other hand, we employ a leave-one-out methodology, ensuring that peptides appearing the testing set are completely distinct from those in the training set, scenario where database search methods not applicable.
4.3
MAIN RESULTS
The main results are shown in Table [ We compared ReNovo with representative baseline models across three datasets and four evaluation metries, previously mentioned.

---

Published as a conference paper at ICLR 2025
Table 1: Empirical comparison of de novo peptide sequencing models. The best and the second best are highlighted with bold and underline, respectively.
Peptide-level performance
Amino acid- level performance
Method
Seven species
Nine species
Seven
Seven-species species
Nine-species
Prec.
AUC
Prec.
AUC
Prec.
AUC
Prec.
Recall
Prec.
Recall
Prec.
Recall
DeepNovo
0.204
0.136
0.428
0.376
0.313
0.255
0.492
0.433
0.696
0.638
0.531
0.534
PoiniNovo
0.022
0.007
0.480
0.436
0.419
0.573
0.196
0.169
0.740
0.671
0.623
0.622
CasaNovo
0.119
0.211
0.322
0.697
0.442
HelixNovo
0.234
0.173
0.517
0.453
0.356
0.318
0.481
0.472
0.765
0.758
0.588
0.582
AdaNovo
0.174
0.135
0.505
0.469
0.212
0.178
0.379
0.385
0.698
0.709
0,442
0.451
ReNovo
0.278
0.228
0.568
0.528
0.467
0.436
0.512
0.514
0.770
0.769
0.651
0.648
ReNovo Outperforms All Current State-Of-The-Art Models Across All Datasets
As shown in Table 1 ReNovo surpasses all the baseline models in both peptide-level evaluation metrics and amino levellevel evaluation metrics across all three datasets.
In terms of the most critical metric, peptide-level precision, ReNovo outperforms the best base- line models (HelixNovo and PointNovo) by 18.8%, 9.86%, and 11.46% across the three datasets, respectively, achieving an average improvement of 13.31% %. This highlights that ReNovo's improve- ments are both substantial and consistent. Additionally, ReNovo demonstrates remarkable results in peptide-level AUC, surpassing the top baseline models (HelixNovo, AdaNovo and PointNovo) by 31.79%, 12.58%, and 16.89% across the there datasets, respectively, with average increase of 20.42%. Peptide-level AUC offers a comprehensive evaluation of model performance, further reinforcing that ReNovo's performance enhancements are both significant and well-rounded.
At the amino acid-level, ReNovo also exhibits notable improvements. Across the three datasets, ReNovo improves amino acid precision over the best baseline models by 4.07%, 0.65%, and 5.49%, respectively. Similarly, its amino acid recall increases by 8.9%, 1 1.45%, and 4.18%, respectively. An average improvement 12% was observed across all amino acid-level metrics.
ReNoyo Achieves More Significant Improvement in Peptide-level Metrics.
It is noteworthy that despite ReNovo utilizing amino acid-level retrieval where the next amino acid Ye is predicted based on the context feature target amino acid pairs retrieved from the datastore D, the gains at the peptide level (13.31 % precision and 20.42% in AUC) are significantly higher than those at the amino acid level (4. (2%). This indicates that the information retrieved from the datastore pertains to entire peptide sequence rather than being confined to individual amino acid-level information, enabling ReNovo to achieve superior performance at the peptide level performance, which is a more crucial factor when evaluating a de novo peptide sequencing model.
4.4 ABLATION STUDY AND HYPERPARAMETER TUNNING
Ablation Study: The Retrieval Mechanism of ReNovo Contributes Substantially.
We conducted ablation experiments to evaluate three different configurations: (1) ReNovo w/o Re- trieval: using the logits from the Peptide Decoder without relying on datastore retrieval, (2) ReN- ovo: using only the prediction results retrieved from the datastore, (3) ReNovo with Residual: combining both (1) and (2) via a residual connection. We analyzed the impact of these config- urations on ReNovo's 's performance, with the results shown in Figure 3 We observed significant performance decline the ReNovo model without datastore retrieval (ReNovo w/o retrieval), which demonstrates that the context feature target amino acid pairs retrieved from the datastore contribute substantially to ReNovo's ' performance improvements and indicates that the retrieval-base frame- work of ReNovo is highly effective. Although pure retrieval approach (ReNovo) is effective, further improve performance by combining the probability distribution obtained from using the log- its from the Peptide Decoder with the probability distribution in Equation 10 based retrieval via residual connection. This configuration (ReNovo with Residual) yields a modest performance gain.
ReNovo's Robustness With Respect To the Number of Retrieved Pairs K
During the retrieval-basee based inference stage, the ReNovo model retrieves K context feature target amino acid pairs from the datastore D. A larger K means the ReNovo model retrieves more pairs.

---

Published as a conference paper at ICLR 2025
0.5
0.7
0.6
0.6
0.5
0.5
0.4
0.2
0.2
0.1
0.1
Prec.
AUC
Prec.
AA Recall
Prwc. Pep.
AUC
AA Prec.
AA Recall
Prec.
AUC
Prec.
AA Recall
(a) Seven-species Dataset
(b) Nine-species Dataset
(c) HC-PT Dataset
Figure 3: ReNovo's 's ablation experiments conducted on three datasets.
We conducted sensitivity analysis experiments on the Seven-species Dataset to investigate the impact of the parameter K on the ReNovo's performance, and the results are shown in Figure 4 We observed that when K is relatively small (K < 32), increasing K results in significant performance improvement because the ReNovo model can more effectively leverage the information available in the datastore. However, further increases in K provide minimal additional performance gains.
0.25
0.20
0.48
0.46
Peptide Precision
0.44
Amino Acid Precision
Peptide AUC
Amino Acid Recall
0 2
128 The Number of Retrieved Pairs K
8
16
32
16 32
64 128
The Number of Retrieved Pairs K
(a) Peptide-lewe
(b) Amino acid level
Figure 4: The impact of the number of pairs K on the performance of the ReNovo model.
ReNovo's Robustnes With Respect To the Temperature T
In Equation TO the temperature parameter T controls the smoothness of the output probabilities. We conducted sensitivity analysis on the Seven-species Dataset to investigate the impact of the parameter T on the ReNovo's 's performance. The results are shown in Figure We can observe that when 7' is set within reasonable range (2 7), its impact performance minimal.
0.26
0.51
0.24
0.50
0.49
0,20
Peptide Precision Peptide AUC
0.48
Amino Acid Precision Amino Acid Recall
0.18
0.5 0.7
1.0
2.0
3.0
5.0
7.0 10
0.5 0.7 1.0
2.0
3.0
5.0
7.0 10
Temperature
Temperature T
(a) Peppidee-evel
(b) Amino acid-leve
Figure 5: The impact of the temperature T' on the performance of the ReNovo model.
8

---

Published as a conference paper at ICLR 2025
4.5 CONSUMPTION OF TIME AND STORAGE
In this section, we examine the time and storage consumption of the ReNovo model.
The Time Consumption of the ReNovo Model Is Negligible.
When evaluating running times, we ensured consistent experimental setups across all models in- cluding: the of Nvidia A100 GPU (80GB), setting the batch size 32, and calculating the average time by dividing the total time by the number of steps. The results, shown in Table reveal minimal differences in training time between ReNovo and other models. However, ReNovo's infer- ence stage requires datastore retrieval, resulting in longer inference time that is 1.53 to 2 times that of HelixNovo, depending on the dataset. Nevertheless, considering the substantial improvements detailed in Table I this increase in inference time is negligible and a reasonable trade-off.
Table 2: Training and inference times of various models.
Training Time (s) 9-species
Inference Time (s)
Model
7-species
HC-PT
7-species
9-species
HC-PT
DeepNovo
0.31
0.38
0.30
0.04
0.07
0.02
PointNovo
0.34
0.31
0.28
0.25
0.24
0.22
CasaNovo
0.36
0.33
0.32
0.27
0.28
0.26
InstaNovo
0.11
0.10
0.14
0.03
0.03
0.04
AdaNovo
1.16
1.07
0.96
1.48
1.50
1.46
HelixNovo
0.56
0.35
0.41
0.30
0.28
0.17
ReNovo
0.18
0.11
0.12
0.46
0.55
0.34
Table 3: The time consumption of ReNovo at each stage.
Model Training Datastore Building Retrieval- Inference
Time(s) Percentage
84,703 88.98%
2,207
8,278
2.32%
8.70%
We also compared the time consumption of the ReNovo model across three stages. The results, presented in Table 3 show that the datastore building stage accounts for approximately 2.32% of the total time in the 'Model Training Datastore Building Retrieval Based Inference" pipeline. Additionally, the relatively more time-consuming retrieval-based inference stage constitutes only about 8.70% of the total time for the entire pipeline. Therefore, the time consumption associated with both datastore building and retrieval-baae inference negligible.
The Storage Consumption of the ReNovo Model Is Negligible.
The storage consumption of ReNovo's datastore shown in Table Depending the scales of datasets, the datastores built for the three datasets contain approximately 5.6 million, 8.4 million, and 3.2 million context -taget amino acid pairs, respectively. The corresponding storage space occupied is 11 GB, 16 GB, and 6 GB. Considering that the datastore is plug-and-play occupying only storage space rather than memory space, the consumption can be considered negligible.
Table The datastore statistic.
Seven-species Dataset
Nine-species Dataset
HC-PT Dataset
Pairs Number Storage (GB)
5,626.944
8, 8,456,240
3,232,616 6.42
11.16
16.77
4.6 CASE STUDY
Through the case study presented in Table we illustrated how ReNovo, with the assistance of the datastore, can make more accurate predictions during retrieval-based inference. Given input MS2

---

Published as a conference paper at ICLR 2025
data s, ReNovo autoregressively generated the peptide sequence V1:--1 "RVNLARIDNE" For the next amino acid prediction for Vt, if the ReNovo does not leverage datastore retrieval (K = 0), it predicts the amino acid 'E' based on the logits (p(E) p(D)) output by the ReNovo's Peptide Decoder. In contrast, when employing retrieval-based inference (K' 32), both amino acids 'E' and 'D' are present among the target amino acids retrieved by ReNovo. After applying the weighting calculation based on Equation | and Equation [0] the model assigns higher weight to amino acid 'D', resulting in the final prediction of 'D' rather than 'E'. This enabled the ReNovo, with the assis- tance of the datastore information, to correctly predict the entire peptide sequence. This case study illustrates that the knowledge does not need to be implicitly stored within the parameters of ReNovo. Instead, it can be explicitly acquired through plug-and-play established datastore. By leveraging an established datastore, the aceuracy of peptide predictions can be significantly enhanced.
Table 5: Case Study. Visualization can be found in Figure in Appendix
Input MS2
Previously Predicted
RVNLARIDNE
Sample 1
Sample 2
Sample
Retrieved MS2
Retrieved Amino Acid
D
E
D
Retrieved Distance
22.11
26.21
26.23
Predicted Amino Acid(K = 0)
pD) = 0.45, pE) = 0.50,
Predicted Amino Acid(K = 32)
P(D) = 0.48, (EE) 0.47,
Ground Truth Amino Acid
D
Predicted Peptide(K- 0)
RYNLARIDNEEVM
Predicted Peptide(K = 32)
RVNLARIDNEDDVM(155....
Ground Truth Peptide
RUNLARIDNIDVM(+15550))
5 CONCLUSION
In this study, we introduce ReNovo, first-of-its-kind retrieval-based de novo peptide sequencing model. ReNovo employs a innovative "Model Training Datastore Building Retrieval Based In- ference"' pipeline. By constructing datastore from the training data, the ReNovo model can utilize the datastore to conduct retrieval-based inference, thereby achieving improved performance while also being able to predict novel peptide that are not present in any pre-existing peptide databases. Experiments confirm that ReNovo outperforms the state-of-the-a models across three widely-use datasets. Additionally, the time and storage consumption associated by the ReNovo are negligible. We believe that retrieval-bace based de novo peptide sequencing will establish a new paradigm in this field, offering valuable insights and inspiration researchers in this domain. In the future, ReNovo expected to lead to practical applications in proteomics, including enhancing the analysis of im- munopeptidomics and metaproteomics, as well enabling deeper exploration of the dark proteome.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This work was supported by National Natural Science Foundation of China Project No.623B2086 and No. U21A20427, the Science & Technology Innovation 2030 Major Program Project No. No.20222D0150100, Project No. WU2022A009 from the Center of Synthetic Biology and Inte- grated Bioengineering of Westlake University, and Project No. WU20230016 from the Westlake University Industries the Future Research. Finally, we thank the Westlake University HPC Center for providing part of the computational resource.
REFERENCES
Akari Asai, Zeqiu Wu, Yizhong Wang. Avirup Sil, and Hannaneh Hajishirzi. Self-rag: Learning to retrieve, generate, and critique through self-reflection. arXiv preprint arXiv:2310. 1511, 2023.
Denis Beslic, Georg Tscheuschner, Bernhard Y Renard, Michael Weller, and Thilo Muth. Com- prehensive evaluation of peptide de novo sequencing tools for monoclonal antibody assembly. Briefings in Bioinformaticx, 24(1) (b)bbb524552, 2023.
Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Milli- can, George Bm Van Den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, et al. Improving language models by retrieving from trillions of tokens. International conference on machine learning, pp. 2206 2240. PMLR, 2022.
Jürgen Cox and Matthias Mann. Maxquant enables high peptide identification rates, individualized ppb-range mass accuracies and proteome-wice protein quantification. Nature biotechnology, 26 (12):1367-1372. 2008
Vlado Dančík, Theresa A Addona, Karl R Clauser, James E Vath, and Pavel A Pevzner. De novo peptide sequencing via tandem mass spectrometry. Journal of computational biology, 6(3-4): 327-342, 1999.
Kevin Eloff, Konstantinos Kalogeropoulos, Oliver Morell, Amandla Mahona, Jakob Berg Jespersen, Wesley Williams, Sam PB van Beljouw, Marein Skwark, Andreas Hougaard Laustsen, Stan IJ Brouns, et al. De novo peptide sequencing with instanovo: Accurate, database-free peptide iden- tification for large scale proteomics experiments. bioRxiv, pp. 2023-08, 2023.
Jimmy K Eng, Ashley L. McCormack, and John R Yates. An approach to correlate tandem mass spectral data of peptides with amino acid sequences in a protein database. Journal the american society for mass spectrometry, (11:)))77999) 1994.
Bernd Fischer, Volker Roth, Franz Roos, Jonas Grossmann, Sacha Baginsky, Peter Widmayer, Wil- helm Gruissem, and Joachim M Buhmann. Novohmm: hidden markov model for de novo peptide sequencing. Analytical chemistry, 77(22):7255-7273 2005.
Yonghua Han, Bin Ma, and Kaizhong Zhang. Spider: software for protein identification from se- quence tags with de novo sequencing error. In Proceedings. 2004 IEEE Computational Systems Bioinformatics Conference, 2004. CSB 2004., pp. 206-215. IEEE, 2004.
Robert L. Hettich, Chongle Pan, Karuna Chourey, and Richard Giannone. Metaproteomics: har- nessing the power of high performance mass spectrometry to identify the suite of proteins that control metabolic activities in microbial communities. Analytical chemistry, 8509):2003-4214, 2013.
Korrawe Karunratanakul, Hsin-Yao Tang, David W Speicher, Ekapol Chuangsuwanich, and Sira Sriswasdi. Uncovering thousands of new peptides with sanecnce-mask-seacch hybrid de novo peptide sequencing framework. Molecular & Cellular Proteomics, 18(12):247822) 2491, 2019.
Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. Generalization through memorization: Nearest neighbor language models. arXiv preprint 19Xiv:1911.0011 2019.
Urvashi Khandelwal, Angela Fan, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. Nearest neigh- bor machine translation. arXiv preprint arXiv:2010.007/1 2020.

---

Published as a conference paper at ICLR 2025
Andy T Kong. Felipe V Leprevost, Dmitry M Avtonomov, Dattatreya Mellacheruvu, and Alexey I Nesvizhskii. Msfragger: ultrafast and comprehensive peptide identification in mass spectrometry- based proteomics. Nature methods, (M65:513-520, 2017.
VI Levenshtein. Binary codes capable of correcting deletions, insertions, and reversals. Proceedings of the Soviel physics doklady, 1966.
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval- detrental-uugmentedd genera- tion for Enowledge-intensive nlp tasks. Advances in Neural Information Processing Systems, 33: 2020.
Bin Ma, Kaizhong Zhang, Christopher Hendrie, Chengzhi Liang, Ming Li, Amanda Doherty-Kirby, and Gilles Lajoie. Peaks: powerful software for peptide de novo sequencing by tandem mass spectrometry. Rapid communications in mass spectrometry, 17(20):2337-2342 2003.
Pedro Henrique Martins, Zita Marinho, and André FT Martins. Efficient machine translation domain adaptation. arXiv preprint 2220:22044.1668, 2022.
David N Perkins, Darryl JC Pappin, David M Creasy, and John Cottrell. Probability-based protein identification by searching sequence databases using mass spectrometry data. ELECTROPHORE. SIS: An International Journal, 20(1 0):3511-35666 1999.
Rui Qiao, Ngoc Hieu Tran, Lei Xin, Xin Chen, Ming Li, Baozhen Shan, and Ali Ghodsi. Com- putationally rnstuueettressoluiioo-neeeerr
de novo peptide sequencing for high-resolution devices. Nature Machine Intelligence, 3(5):420-425, 2021.
Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton- and language models. Transactions of the Association
Yoav Shoham. In-context for Computational Linguistics, 1 11:1316-111 1331, 2023.
Bernhard Y Renard, Buote Xu, Mare Kirchner, Franziska Zickmann, Dominic Winter, Simone Ko- rten, Norbert W Brattig, Amit Tzur, Fred A Hamprecht, and Hanno Steen. Overcoming species boundaries in peptide identification with bayesian information criterion-driven error-tolerant pep- tide search (biceps). Molecular & cellular proteomics, (177) (17):M111001116) 11-014167, 2012.
J Alex Taylor and Richard S Johnson. Sequence database searches via de novo peptide sequencing by tandem mass spectrometry. Rapid communications in mass spectrometry, 1067::0077-1755 1997.
Ngoc Hieu Tran, Xianglilan Zhang, Lei Xin, Baozhen Shan, and Ming Li. De novo peptide sequenc- ing by deep learning. Proceedings of the National Academy of Sciences, 114(31):8247-8255, 2017.
Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(1)), 2008.
Vaswani. Attention all you need. Advances Neural Information Processing Systems, 2017.
Katty X Wan, Iln Vidavsky, and Michael Gross. Comparing similar spectra: from similarity index to spectral contrast angle. Journal of the American Society for Mass Spectrometry, 13(1): 85-88, 2002.
Ronald J Williams and David Zipser. A learning algorithm for continually running fully recurrent neural networks. Neural Computation, 1(2):270-280, 1989.
Jun Xia, Shaorong Chen, Jingbo Zhou, Tianze Lin, Wenjie Du, Sizhe Liu, and Stan Li. Adanovo: Adaptive's emph (De Novo} peptide sequencing with conditional mutual information. arXiv preprint arXiv:2403.07011 2024.
Tingpeng Yang. Tianze Ling, Boyan Sun, Zhendong Liang, Fan Xu, Xiansong Huang, Linhai Xie, Yonghong He, Leyuan Li, Fuchu He, et al. Introducing r-helixnovo for practical large-scale de novo peptide sequencing. Briefings in Bioinformaties, 25(2)bbaae021, 2024.
12

---

Published as a conference paper at ICLR 2025
Melih Yilmaz, William Fondrie, Wout Bittremieux, Sewoong Oh, and William S Noble. De novo mass spectrometry peptide sequencing with a transformer model. In International Conference on Machine Learning, pp. 25514-25522. PMLR, 2022.
Xin Zheng, Zhirui Zhang, Junliang Guo, Shujian Huang, Boxing Chen, Weihua Luo, and Jiajun Chen. Adaptive nearest neighbor machine translation. arXiv preprint arXiv:2105.13022 2021.
A DATASET
A.1 DATASET STATISTICS AND DESCRIPTION
The detailed statistics of the datasets are shown in Table 6
Table 6: The datasets statistics.
Dataset
Avg. peaks
Avg. peptide length
train/valid/test set size
Seven-species Dataset
466.05
15.79
317,009 / 17,740 / 17.049
Nine-species Dataset
134.91
15.01
499,402 / 28,572 27, 142
HC-PT Dataset
184.21
12.53
213,284 / 25,718/ 26,536
Seven-species Dataset (Tran al. 2017) This dataset comprises low-resolution mass spectra and corresponding peptide labels derived from seven distinct species. To maintain consistency with the primary experiments, our evaluation focused on the yeast species.
Nine-species Dataset (Tan et al. 2017) This dataset, which has been extensively utilized in previ- ous research, contains bigh-resolution mass spectra and peptide labels from nine different species. It is worth noting that this dataset incorporates three post-translational modifications (PTMs): oxi- dation of methionine, deamidation of asparagine and deamidation of glutamine.
HC-PT Dataset As described in the InstaNovo (Eloff et al. 2023), the HC-PT dataset consists of synthetic tryptic peptides representing all canonical human proteins and their isoforms. It also includes peptides generated by alternative proteases and HLA peptides. The distinguishing feature of this dataset is its high-resolution spectra for human-orighn peptides.
A.2 ANALYSIS OF DATASET OVERLAP
To prevent data leakage and demonstrate that the performance improvement of the ReNovo model arises from generalization rather than overfitting to the training dataset through retrieval, we ana- lyzed the overlap of spectra and peptides between the training and test sets across three datasets.
For spectra overlap, we referred to the method in Wan et al. (2002): Specifically, all spectra were discretized into binned spectra. We then calculated the average similarity between each binned spectrum in the test set and all those in the training set. Finally, we averaged these similarity values across all spectra in the testing set to obtain the overall dataset similarity (ranging from [-1, I]). which measures the overlap between the training and test spectra. The results are shown in Table
Seven-species Dataset
Nine-species Dataset
HC-PT Dataset
Similarity
0.172
0.101
0.119
Table 7: Similarity of binned spectra between test set and training set.
From Table 7 it can be observed that the spectra similarity for all datasets approximately 0.1. This indicates that the overlap between the spectra in the training set and the test set is quite low.
For peptide overlap, we employ a leave-one-out cross cross-validatee validation framework in which the peptides in the training set completely disjoint from those in the test set. In other words, during the pre- processing phase of the dataset, we ensured that there was no duplication between peptide sequences in the training and test sets, making the dataset suitable for evaluating de novo sequencing methods.
13

---

Published as a conference paper at ICLR 2025
To further explore the model's performance test sequences that lack similar sequences in the training set, we conducted experiments on the Nine-species Dataset, using AdaNovo (the best base- line for this dataset in Table [ for comparison. We began by calculating the Levenshtein distance (Levenshtein 1966) (a widely adopted method for measuring the similarity between two strings) for each test sequence against all sequences in the training set. We then identified the minimum Leven- shtein distance for each test sequence and filtered out test sequences with minimum Levenshtein distance <3 and <5, respectively, resulting Nine-species Test Dataset (>3) and Nine-species Test Dataset (>5). These datasets, having excluded peptide sequences similar to those in the training set. We maintained all other experimental settings unchanged, and the results are presented in Table
Model Name
Test Set
Peptide Precision
Peptide AUC
ReNovo
Nine-species Dataset (Original)
0.568
0.528
AdaNovo
Nine-species Dataset (Original)
0,505
0.469
ReNovo
Nine-species Test Dataset (>3)
0.465
0.423
AdaNovo
Nine-species Test Dataset (>3)
0.398
0.356
ReNovo
Nine-species Test Dataset (>5)
0.396
0.351
AdaNovo
Nine-species Test Dataset (>~5)
0.333
0.292
Table Performance Comparison on Filtered Test Datasets.
From Table 8 we observe that the performance of both ReNovo and AdaNovo declines on the filtered test datasets, which is expected as the test set contains fewer similar data to the training set. However, significantly on the same filtered test sets. This clearly indicates that the improved performance of ReNovo is due to generalization rather than overfitting to the training dataset.
also evident that ReNovo still outperforms the state-of-the-art baseline models
B EVALUATION METRICS
This section delineates the evaluation metrics employed in our experiment:
Peptide-level Precision This metric serves the principal indicator of model's practical utility, as the ultimate goal of de novo peptide sequencing is to assign complete and accurate peptide sequences each mass spectrum. A predicted peptide y is considered correct only if its entire amino acid sequence matches the ground truth y, that is, = y. Given a dataset of all spectra, if model correctly predicts N2 Maatch peptides, the peptide-level precision is calculated as N2 / /N
Area Under the peptide-level Precision-Recall Curve (AUC) In the following sections, we will refer this metric as the "AUC" for simplicity. The calculation process divided into several steps: Calculate confidence scores sc(v,) for individual amino acids y in the ReNovo model's output y. Define the overall confidence score y the mean its constituent amino acid confidence scores, that is sc(y) mean( sc(y:) ())))))))))
). y1 E Sort predictions in descending order based on confidence scores. Starting from the highest confidence prediction, accumulate the model's peptide recall and precision values. These cumulative values are then used as the horizontal axis and the vertical axis, respectively, for the points on the precision-recall curve. The AUC value obtained by calculating the area under the curve formed by this curve. The resulting AUC provides a robust measure of model performance, accounting for both precision and recall across different confidence levels.
Amino Acid-level Precision and Recall These metrics offer more granular assessment of model performance at the individual amino acid level. The process involves: Determining the number of matched amino acid predictions Naa Waatch based two criteria: a) Mass difference 0.1 Da from the corresponding ground truth amino acid. b) Either prefix or suffix with a mass difference < 0.5 Da from the corresponding sequence in the ground truth peptide. Calculating amino acid-level precision and recall as N Nae match/ /N /aa pred and N Naa where Naa and Naa is the total number of predicted amino acids and ground truth amino acids respectively. These metrics complement the peptide-level assessments.
14

---

Published as a conference paper at ICLR 2025
C RELIABILITY ANALYSIS
To evaluate the reliability of the ReNovo model, we included a recall-coverage curve shown in Figure which can provide valuable insights into its performance across different confidence levels. Specifically, amino acid-level confidence scores are computed by applying softmax layer to the raw output of the Peptide Decoder. We use mean score across all amino acids to derive a peptide- level confidence score. When plotting the recall-coverage curve, all predicted sequences are ranked based on their confidence scores. For amino acid-level curves, all amino acids within a given peptide assigned the same score.
1.5
3.2
3.3
6.5
6.5
0.4
6.1
6.
Coverage
Coverage
Caverage
(a) Seven-species Dataset
(b) Nine-species Dataset
(c) HC-PT Dataset
Figure 6: The recerl-coverage curves for ReNovo illustrate the relationship between recall and cov- erage. The horizontal axis indicates coverage, while the vertical axis shows peptide-level and amino
acid (AA)-level recall. The blue line corresponds to the peptide-level recall performance, whereas the yellow line represents the AA-level recall performance.
D VISUALIZATION OF CASE STUDY
To provide easily interpretable visual comparison of the similarities between the retrieved con- text features and those generated by ReNovo for the case study in Section 4.6 we applied t-SNE dimensionality reduction Van der Maaten Hinton (2008) project the context features onto two-dimensional plane. As highlighted in the case study described in Table 5 without retrieval, the model incorrectly predicts the amino acid 'E', However, with retrieval, two possible target amino acids: "D' and 'E', represented by blue and orange color in the Figure 7 respectively. Notably, the cluster corresponding to amino acid 'D' is larger and carries greater weight. By applying the weighting caleulations outlined in Equations 9 and 10 the model assigns a higher weight to amino acid 'D', resulting in a final prediction of instead of 'E', This demon- strates how ReNovo effectively utilizes datastore accurately predict the entire peptide sequence.
ReNovo model retrieves
200
-100
-200
-300
-200
-100
100
200
300
Figure 7: Visualization of Case Study, Only the 30 retrieved context features with the smallest Euclidean distance were retained and projected into 2D space. The red pentagram represents the context feature generated by ReNovo, while the blue and orange dots correspond to the retrieved context features associated with two target amino acids: 'D' and 'E', respectively. The intensity of the colors reflects the Euclidean distance, with darker shades indicating smaller distances.
15

---

Published as a conference paper at ICLR 2025
E
THE EFFECT OF DATASTORE SIZE ON RENOVO's PERFORMANCE
Datastore Pairs
Datastore Size
Peptide Precision
Peptide AUC
AA Precision
AA Recall
8,456.240
16.77
0.568
0.528 0.656
0.770 0.836
0.769
22.646,374
44.89 GB
0.696
0.834
Table 9: The performance of the ReNovo model across various datastore sizes.
To further investigate the effect of datastore size on ReNovo's performance, we conducted additional experiments using larger dataset (beyond the training set). Specifically, utilized the ReNovo model trained on the Nine-species Dataset, along with larger dataset from (Tran et al. 2017) that includes the same species, excluding data belonging the same species the test the datastore. The training set, test set, and other configurations were kept unchanged. The sizes of the two datastores and the corresponding ReNovo performance metrics detailed in Table 9
construct
We find it quite surprising that adding additional data to the datastore led to such a significant performance gain, while keeping the training dataset and model weights unchanged. This experi- ment highlights the effectiveness of the retrieval-based mechanism: (1) Knowledge does not need be implicitly stored within the model parameters; instead, it is explicitly stored in the datastore, which can be accessed during the inference phase in plug-and-play manner, thereby enhancing the model's s generative capabilities. (2) By generating sequences based on retrieved references rather than from scratch, the ReNovo model reduces the inherent complexity of producing coherent and contextually accurate sequences. We believe this insight could be highly enlightening for researchers in the field of peptide identification, specifically by demonstrating the potential of utilizing retrieval- based mechanisms and large-scale datastore to improve sequencing performance.
16


Reviews:
[{"review_id": "vqGn4OIXEF", "reviewer": "Reviewer_eX4t", "summary": "The paper introduces a novel approach for predicting peptide sequences from tandem mass spectra, achieving significant performance improvements with a simple retrieval-augmented generation method.", "questions": "Please see Weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "**Originality**. The paper proposes an original idea of connecting database search with *de novo* generation for predicting peptide sequences from mass spectra.\n\n**Quality**. The experiments are sound and the source code is provided.\n\n**Clarity**. The paper is well-written and easy to follow.\n\n**Significance**. The performance improvement upon prior work is substantial.", "weaknesses": "**Major Comments**\n\n- Missing related work. The paper does not discuss related work on retrieval-augmented generation (RAG) in the context of large language models (LLMs). Is the proposed method novel compared to recent advances in natural language processing?\n- Lack of comparison with database search methods. Although overcoming the limitations of database search is a primary motivation, the proposed method is not compared against database search approaches. At least a simple search baseline against the same training database should be implemented for comparison. Ideally, state-of-the-art database search methods should aslo be compared against.\n- Potential data splitting issues. The leave-one-out data split by species, while seemingly reasonable, might lead to data leakage. Specifically, the same peptides could appear across different species (potentially even with identical or highly similar MS² spectra). Please analyze the overlap of spectra and peptides between the training and test sets, for example, by assessing vector similarity between binned spectra and sequence identity to evaluate spectral and peptide similarity, respectively.\n\n**Minor Comments**\n\n- Figure 1: Clarify what is meant by “identification rate”—is it the accuracy of preptide prediction?\n- Table 1: Confidence intervals or standard deviations for the reported values are missing.\n- Line 160: What is the charge state of the precursor ion? Why is it importnant?\n- Line 163: “sets” -> “ordered sets of the same size” for mathematical soundness?\n- Equation 2: Typo: the comma at the end.\n- Line 196: Please clarify how the described features help to attend to differences.\n- Line 201: A citation for teacher-forcing is missing.\n- Equation 9: Adding one or two sentences explaining the intuition behind this equation could enhance clarity.\n- Section 4.1: References for the datasets are missing.\n- Section 4.4: If “ReNovo with Residual” is the best-performing method, please clarify why the residual component is not described in the methods section and not used in the final evaluation in Table 1.\n- Table 5: Please explain the meaning of “+15.99.”\n- Appendix A: References are missing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mZBAdafyaM", "reviewer": "Reviewer_xfGv", "summary": "The paper introduces a new method for predicting peptide sequences from MS/MS spectra, with a key innovation in combining two complementary paradigms in computational proteomics: database search and de novo sequence generation. Specifically, the paper introduces the concept of a datastore, which can be used in conjunction with an autoregressive transformer model. After model training, the datastore stores outputs from the trained model across the entire training dataset, which are then used during test-time inference via k-NN indexing. The method empirically outperforms several state-of-the-art approaches across three datasets, requiring only minor additional storage and time complexity. The authors include ablation studies on key hyperparameters. The code is available in the Supplementary materials.", "questions": "- How was the case study example in Table 5 selected? Is it possible to quantify the number of similar improvements across the entire test set?\n- What were the hyperparameters used for training, specifically the number of epochs and learning rate?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "Originality\n\nThe work’s originality mainly lies in the combination of two primary paradigms commonly applied to peptide MS/MS spectra annotation: database retrieval and de novo sequence generation.\n\nQuality\n\nThe paper is overal of a good quality, particularly in terms of dataset preparation, benchmarking the proposed method against prior approaches, and the analysis of results.\n\nClarity\n\nThe paper is well-structured, clearly defines the motivation for the proposed methods, and provides a thorough analysis of the results.\n\nSignificance\n\nThe work offers a new perspective on the problem of de novo peptide sequencing, which could have a strong impact on future research in this area.", "weaknesses": "- The main concern with the proposed approach is whether the state-of-the-art results are not a consequence of the model’s higher capacity rather than its design. To address this, it would be relevant to see, for example, results from the following experiment: evaluate the ReNovo without Retrieval model of larger size (e.g., by increasing the transformer’s number of layers and hidden dimensionality) and compare it to ReNovo, as illustrated, for instance, in Figure 3.\n\n- Another related concern is the train-test splitting strategy. The text states that “peptides appearing in the testing set are completely distinct from those in the training set.” It would be valuable to assess the method’s performance under a stricter splitting criterion, such as one based on a sequence similarity threshold rather than sequence identity. Such an experiment would demonstrate whether the improved performance is due to generalization rather than overfitting to the training dataset through the “datastore” approach. This could be also tested by measuring the model’s improvement on test sequences that lack similar sequences in the training set.\n\n- Certain essential parts of the text lack clarity.\n    - The term “context features” is central to the work but is not clearly defined. Does this refer to an embedding from the last transformer decoder layer? What does $f$ in line 213 signify?\n    - The “ReNovo with Residual” model is not fully defined. How exactly are “ReNovo without Retrieval” and “ReNovo” configured?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0uTYrRECiK", "reviewer": "Reviewer_Hu9o", "summary": "This paper Introduces a novel approach for peptide identification in proteomics, where conventional methods rely heavily on pre-constructed peptide databases, limiting their scope when dealing with novel peptides. ReNovo, leveraging a retrieval-based framework, bypasses this limitation by building a datastore from training data and retrieving relevant information during inference. This approach combines the strengths of both database search and de novo sequencing, enabling it to predict peptides absent from existing databases.", "questions": "The authors use training data to construct the datastore, but I believe this approach could benefit from further discussion and refinement. In mainstream RAG-based methods, the datastore is generally maintained as an independent entity, distinct from the training data for language models. This separation allows the datastore to act as an “additional” resource that complements a model’s learned knowledge. When the same dataset is used for both training the language model and constructing the datastore, much of the useful information may already be encoded in the model weights through the training phase, reducing the unique value of “retrieval.” Specifically, the language model’s objective—next-token prediction—already captures “context-target” relationships by predicting the next token based on the previous context. Token-level retrieval, in this context, could become redundant, as the model effectively performs this retrieval autonomously. I strongly recommend the authors separate training data from datastore data to better evaluate the impact of retrieval on de novo sequencing. It would also be useful to see how a larger datastore, too complex to incorporate into training, might act as an external resource that enhances model performance.\n\n\n\nDuring inference, could you clarify how the top K “next target” retrievals assist the model? Formulas 9-10 lack clarity in this respect. Simplifying the formulas and using clearer explanations would improve comprehension, as sophisticated formulas are only effective if they are accompanied by a clear rationale. Specifically, does the probability for each amino acid derive directly from the retrieved targets in the output layer, or is it calculated after embedding these K tokens into the input layers?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper presents strong motivation for combining database search with de novo sequencing, marking the first integration of the Retrieval-Augmented Generation (RAG) method into protein sequence prediction, which adds a novel dimension to the field. The authors effectively demonstrate the proposed method's success, showcasing the potential of integrating RAG into bioinformatics, particularly in areas with extensive databases.", "weaknesses": "1. The paper primarily employs a token-level RAG method from NLP but does not thoroughly discuss its origins or previous related work. Specifically, there is no comprehensive section on RAG methods, with most background instead focused on de novo methods. The approach closely resembles the KNN-LM paper, which originally proposed storing context and next tokens in a datastore; however, this foundational work is not referenced. \n\nAdditionally, the paper omits discussion on recent advancements in de novo methods, such as GraphNovo, PepNet, PrimeNovo, and InstaNovo, all of which are documented in Bittremieux et al.'s survey on de novo peptide sequencing methods, if author needs to refer to a good source.\n\n2. Missing Performance Comparisons: The model performance comparisons in the main table do not adequately reflect recent advancements in the field, with several relevant methods from the past year omitted. Including a broader range of models would provide a clearer view of the current state of the art.\n\n3. Clarification is needed on the training and testing datasets: whether a single model was trained on a single dataset and used for all number reporting or if separate models were trained for each dataset? Additionally, it would be helpful to confirm if the reported performance numbers for baseliens are replications or directly quoted from original papers. For instance, discrepancies are noted between some results here and those reported for CasaNovo paper i read from their published journal. \n\n4. For the case study, a direct alignment between the predicted and retrieved spectra would make the comparison clearer. The current presentation lacks an easily interpretable visual comparison of their similarities.\n\n5. If the first stage of training essentially reproduces the Casanovo model, the authors could consider using available open-source models for this step. Testing stronger baseline models could indicate whether enhanced context representations lead to better context features in the datastore. No need to re-train casanovo as first stage IMO. \n\n6. Including a recall-coverage curve could offer valuable insights into performance across varying confidence levels, which is especially useful for biologists relying on model certainty for predictions on unlabeled data. This would provide more depth than single averaged metrics, which may not fully capture the model's reliability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kjIJdggneR", "reviewer": "Reviewer_115j", "summary": "The authors present a method called Renovo for identifying peptide sequences from mass spectrometry data. The challenge of peptide identification is undeniably complex and significant, and this paper offers a solution that integrates prior sequence knowledge into de novo sequencing. This is achieved by revisiting learned co-occurrences of amino acids. The authors evaluate their method against widely used de novo techniques using standard community benchmarking datasets, demonstrating superior performance.", "questions": "•\tThe specific database information provided to Renovo during the benchmarks is unclear. What is the size of the database, and to what extent are related sequences excluded (are only identical sequences filtered out, or do filters also apply to those with PTMs, chemical modifications, or single amino acid substitutions)?\n\n•\tCan the authors compare their results to an error-tolerant/open database search method, such as MSFragger, which is currently considered the gold standard in the field? It may be beneficial to replicate a typical scenario, such as in a 7/9 species setup, where I could supply a complete database of closely related organisms. For instance, while humans and chickens diverged around 300 million years ago, it has been shown that error-tolerant approaches can effectively bridge that gap. Unless going for very obscure organisms, there are related species available and that should be the starting point.\n\n•\tIn practical proteomics applications, reliability is crucial, yet the benchmark setup appears limited in this context. Most mass spectra tend to be low quality, and redundancy in measurements is high. Standard practice (and often a requirement for publications in the field) necessitates that peptide identifications exceed a 1% false discovery rate (FDR) threshold. Currently, the authors only provide precision and recall metrics, which do not indicate how many peptides the search algorithm considers reliable. It is not essential for every spectrum to be identified. Can the authors also present the number of peptides identified above the 1% FDR threshold?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "•\tThe authors advance beyond existing de novo methods by allowing the integration of prior knowledge.\n\n•\tThey showcase better performance compared to established approaches.\n\n*       The manuscript is well written and easy to follow.", "weaknesses": "•\tMixture methods combining de novo and database searches have been well documented (notably Spider 10.1109/CSB.2004.1332434 and BICEPS 10.1074/mcp.M111.014167). This is not a novel contribution of this paper, as such approaches have been recognized for two decades and should be acknowledged as related work.\n\n•\tError-tolerant and open database searches are prevalent in the field but are not discussed here, despite their potential relevance (as noted in the following point).\n\n•\tThe authors claim that comparing against database searches is not valid, yet they incorporate prior database information into Renovo that is unavailable to other de novo methods, creating an unfair comparison.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)