Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencing
Authors: Zijie Qiu
Jiaqi Wei
Xiang Zhang
Sheng Xu
Kai Zou
Zhi Jin
ZhiQiang Gao
Nanqing Dong
Siqi Sun
Venue: ICML 2025 poster (2025)
Keywords: Peptide Sequencing
De novo
Reranking
Primary area: applications->everything_else
TL;DR: RankNovo is a novel deep reranking framework that improves de novo peptide sequencing by combining strengths of multiple models.

Abstract: De novo peptide sequencing is a critical task in proteomics. However, the performance of current deep learning-based methods is limited by the inherent complexity of mass spectrometry data and the heterogeneous distribution of noise signals, leading to data-specific biases. We present RankNovo, the first deep reranking framework that enhances de novo peptide sequencing by leveraging the complementary strengths of multiple sequencing models. RankNovo employs a list-wise reranking approach, modeling candidate peptides as multiple sequence alignments and utilizing axial attention to extract informative features across candidates. Additionally, we introduce two new metrics, PMD (**P**eptide **M**ass **D**eviation) and RMD (**R**esidual**M**ass **D**eviation), which offer delicate supervision by quantifying mass differences between peptides at both the sequence and residue levels. Extensive experiments demonstrate that RankNovo not only surpasses its base models used to generate training candidates for reranking pre-training, but also sets a new state-of-the-art benchmark. Moreover, RankNovo exhibits strong zero-shot generalization to unseen models—those whose generations were not exposed during training, highlighting its robustness and potential as a universal reranking framework for peptide sequencing. Our work presents a novel reranking strategy that fundamentally challenges existing single-model paradigms and advances the frontier of accurate de novo sequencing. Our source code is provided on GitHub.

Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencing
Zijie Qiu 12 Jiaqi Wei 32
Xiang Zhang 44 Sheng Xu Kai Zou 56 Zhi Jin Zhiqiang Gao
Nanqing Dong² Siqi Sun
Abstract
(
Hoss Spectronctry Workflow
De novo peptide sequencing a critical task in proteomics. However, the performance of current deep learning-based methods is limited by the in- herent complexity of mass spectrometry data and the heterogeneous distribution of noise signals, leading to -pta-spe biases. We present Ran- kNovo, the first deep reranking framework that enhances de novo peptide sequencing by lever- aging the complementary strengths of multiple sequencing models. RankNovo employs list- wise reranking approach, modeling candidate pep- tides as multiple sequence alignments and utiliz- ing axial attention to extract informative features across candidates. Additionally, we introduce two new metrics, PMD (Peptide Mass Deviation) and RMD (Residual Mass Deviation), which of- fer delicate supervision by quantifying mass dif- ferences between peptides at both the sequence and residue levels. Extensive experiments demon- strate that RankNovo not only surpasses its base models used to generate training candidates for reranking pre- pre-training, but also sets new state-of- the-art benchmark. Moreover, RankNovo exhibits strong zero-shot generalization unseen mod- els-those whose generations were not exposed during training. highlighting its robustness and potential as a universal reranking framework for peptide sequencing. Our work presents a novel reranking strategy that fundamentally challenges existing single-model paradigms and advances the frontier of accurate de novo sequencing. Our source code is provided on GitHub
Protesh Semple
Prstein Dipertion
Mami Spectranetty
AS/WS Spectrum
De Nova Asphic
Da MAA
pipridle
Nave Peppicce Sequncing
nove peştide trust?
con rerank and choose the best onel
k-lphlc
Figure I. (A) De Novo Peptide Sequencing Workflow Using Tan- dem Mass Spectrometry: Our objective to predict peptide se- quences from MS/MS spectra, as illustrated in the final two steps. (B) Motivation for RankNovo: Current de novo peptide sequenc- ing models exhibit data preference in their peptide predictions. RankNovo improves overall prediction accuracy by reranking the outputs of these models to identify the optimal sequence.
1. Introduction
Identifying proteins is a critical task in proteomics, with mass spectrometry- shotgun proteomics being widely regarded as the predominant technique for this purpose (Ae- bersold & Mann, 2003). As shown in Figure 1, this process begins with the enzymatic digestion of proteins into smaller peptide fragments, which are then analyzed using tandem mass spectrometry (MS/MS) to generate spectra (Nesvizh- skii al., 2003). These spectra are subsequently interpreted to infer peptide sequences, enabling precise identification and characterization of proteins. This foundational approach is pivotal for advancing research in proteomics (Aebersold & Mann, 2003).
Proteomics utilizes two primary methodologies for peptide sequence identification: database searching (Ma et al., 2003; Chen et al., 2020: Leprevost al., 2014; Shteynberg et al., 2011; Chi al., 2018) and de novo sequencing (Dančík et al., 1999; Chi et al., 2013). In database searching, ex- perimental spectra are matched against pre-existing entries in protein databases to identify the most likely sequences. Although effective for identifying known peptides, this ap- proach is inherently constrained by the completeness of the database, posing challenges when encountering novel
EEqual contribution FFudan University 'Shanghai Artificial Intelligence Laboratory 'Zhejiang University "University British Columbia 'NetMind.Al "ProtagoLabs Inc "Soochow University. Correspondence to: Siqi Sun <siqisun (aaaunn@uudann...n....... Nanqing Dong <dongnanqing @ pjlab.org.en>:
Proceedings of the nnnd International Conference on Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s). http:///gtthh.cm///EAAMMaaaaa.

---

Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencing
or uncharacterized sequences (Karunratanakul et al., 2019; Hettich et al., 2013). On the other hand, de novo sequenc- ing leverages the intrinsic patterns of tandem mass spectra to directly infer peptide sequences without requiring ref- erence database, enabling the discovery of novel peptides. Consequently, de novo sequencing has emerged as a critical technique for peptide identification, significantly advancing the scope of proteomic analysis (Ng et al., 2023).
ditional pairwise comparison frameworks commonly used in Natural Language Processing tasks (Ouyang et al., 2022; Jiang et al., 2023). To implement this reranking strategy, RankNovo formulates peptide candidates as a Multiple Se- quence Alignment (MSA) (Jumper et al., 2021; Rao et al., 2021; Abramson et al., 2024) and applies axial attention to extract sequential features. In particular, column- atten- tion plays a crucial role in enabling the flow of information and intricate comparisons between candidates (Huang et al., 2019; Ho et al., 2019; Wang et al., 2020a). Additionally, spectrum features are extracted using a Transformer encoder and integrated into the peptide track via cross-attention mechanism.
Over the past two decades, de novo sequencing has made substantial progress, evolving from graph graph-theoretic and dy- namic programming- based methods to more sophisticated approaches driven by deep learning (Ma et al., 2003; LeCun et al., 2015; Gao et al., 2023). DeepNovo (Tran et al., 2017) was the first to apply deep learning to novo sequencing, which inspired series of subsequent models(Zhou et al., 2017; Karunratanakul et al., 2019; Yang et al., 2019; Liu et al., 2023). More recently, Transformer architectures have been introduced to model de novo sequencing as a machine translation task (Yilmaz et al., 2022; Mao et al., 2023; Eloff et al., 2023a; Yang et al., 2024b; Xia et al., 2024a). Building upon this foundation, ContraNovo (Jin et al., 2024) further advanced the field by incorporating multimodal alignment strategies, achieving state-of-the-art performance.
Moreover, the key concentration on amino acid masses in de novo peptide sequencing (Jin et al., 2024) inspires us to propose two novel metries, PMD (Peptide Mass Deviation) and RMD (Residual Mass Deviation), as more nuanced replacement of typical reranking losses such as binary clas- sification loss. The two metrics quantitatively evaluate the mass difference between peptides at both the peptide and residue levels to provide more accurate supervision scores for RankNovo.
Experimental results show that RankNovo achieves state- of-the-art performance on de novo sequencing benchmarks, outperforming each of its component base models, includ- ing the current SOTA model, ContraNovo. We also con- ducted detailed analytical and ablation studies to verify the robustness of the model. Furthermore, we demonstrate that RankNovo, when trained on specific base models, can be effectively applied in zero-shot setting to peptide pre- dictions from unseen sequencing models, highlighting its strong transferability and its ability to capture deep knowl- edge for assessing peptide-spectrum matching performance.
Despite recent advancements in de novo peptide sequenc- ing, these methods still exhibit notable accuracy limitations compared to traditional database search approaches (Muth et al., 2018). The primary challenge stems from the inherent complexity of mass spectrometry data, which consists of a mixture of heterogeneous distributions. This complexity is driven by variations in experimental conditions, such as differences in instrumentation, protocols, and target protein species, each of which introduces distinct noise patterns into the acquired spectra (Zubarev & Mann, 2007; Chang et al., 2016). As shown in Fig. 1(B), no model is exempt from issues of generalization and preferential bias, as evi- denced by the presence of unique correct predictions from models that otherwise exhibit weaker overall performance. This observation motivates rethinking of de novo peptide sequencing as reranking task, where a trained mota-model selects the optimal prediction from a collection of outputs generated by multiple de novo models.
The contributions of this paper can be summarized as fol- lows: (1) We introduce the first deep learning-base based rerank- ing framework for peptide de novo sequencing, designed to bridge the gap between existing methods, thereby un- leashing their complementrry potentials. (2) We propose RankNovo, a list-wise reranking framework that models can- didates as multiple sequence alignments (MSA) and uses axial attention to extract informative features. (3) We further introduce two novel metrics, PMD and RMD, for accurate measurement of mass differences between peptides, pro- viding precise supervised signals for reranking models. (4) Extensive experiments demonstrate that RankNovo not only surpasses each of its individual ensemble components but also generalizes effectively to unseen models in a zero-shot setting, highlighting its robustness and adaptability.
In this paper, we introduce RankNovo, a novel deep rerank- ing framework designed address the preferential bias challenges inherent in peptide sequencing. In such com- plex task, peptide candidates generated for the same spectra often exhibit only minor mass differences. To effectively differentiate between these closely related candidates, Ran- kNovo employs a list-wise reranking approach, processing and reranking all candidates in a single forward pass. This strategy enables the model to incorporate information across candidates, facilitating more precise discrimination between similar sequences. This approach stands in contrast to tra-
2

---

Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencing
2. Related Work
realm of protein modeling, modern deep learning method- ologies frequently employ multiple sequence alignments (MSAs) (Feng & Doolittle, 1987) to harness the rich evolu- tionary and structural information embedded within proteins. For example, the MSA Transformer (Rao et al., 2021), a large-scale protein language model, utilizes axial attention to process extensive aligned MSA data efficiently. Like- wise, the prominent protein structure and interaction pre- diction models AlphaFold2 (Jumper et al., 2021) and AI- phalFold3 (Abramson et al., 2024) leverage axial attention to effectively model the input matrix in latent space, enabling broad spectrum of applications in protein modeling and design.
2.1. De Novo Peptide Sequeneing
De novo sequencing algorithms have witnessed the adaption from early dynamic programming with rule-based scoring functions (Ma et al., 2003; Chi et al., 2010; Ma, 2015) to deep-learning based end-to-end regression, mainly based on transformer architecture (Yilmaz et al., 2022; Mao et al., 2023; Eloff et al., 2023a; Yang et al., 2024b; Jin et al., 2024; Zhou et al., 2024; Eloff et al., 2023b; Xia et al., 2024a; Zhang et al., 2025; Yang et al., 2024a; Xia et al., 2024b).
Despite these advancements, current methods still face in- herent limitations due to the complexity of spectra. In this study, we aim to address these limitations by developing an effective reranking model to select the best matching candidate, thereby enhancing the overall capability of the de novo sequencing algorithm.
Extending these foundational works, our research introduces the first application of the axial attention mechanism to pep- tide modeling. By examining the similarities and differ- ences between candidates and spectra, our model effectively reranks these candidates. This innovative application ex- pands the utility of axial attention, illustrating its potential in peptide modeling.
2.2. Candidate Reranking
In the reranking task, methods are typically categorized into three types: point-wise, pair-wise, and list-wise (Zhuang et al., 2023). The point-wise method independently evalu- ates the relevance of a single query-candidate pair (Nogueira et al., 2019; 2020). The pair-wise method assesses the rel- ative relevance between two candidate pairs for a given query (Burges et al., 2005; Burges, 2010; Ouyang et al., 2022; Jiang et al., 2023). The list-wise method considers the relevance of all candidate pairs for each query collectively, utilizing all candidate features, which enhances performance potential (Han et al., 2020; Gao et al., 2021; Ren et al., 2021; Cao al., 2007; Xia et al., 2008).
3. Method
3.1. Problem Formulation
De novo peptide sequencing seeks to deduce the amino acid sequence from given mass spectrum. Formally, the input set g = {ō, m c.Tee } is composed of three elements: the spectrum 5, a collection of mass-to-charge ratios (m/z) and intensity signals; the precursor charge c an integer, and the precursor mass mmrre, floating-poii value. A spec- trum containing l peaks (signal pairs) can be represented as {AA, I) The objective is to identify a set of po- tential residues, defined as R [r, .r, providing the input G. The core concept of RankNovo is to integrate multiple relatively weak yet diverse de novo models and to train the model to select the optimal solution among their outputs. We referred to these models providing candidate predictions as base models.
The inherent complexity of natural language processing has traditionally constrained reranking algorithms to rely predominantly on weak supervision signals, particularly rel- ative preference indicators, irrespective of the methodologi- cal approach employed (point-wise, pair-wise, or list-wise). Our research presents a novel contribution through the de- velopment of precise metric frameworks (PMD and RMD) that facilitate effective list-wise reranking of candidate pep- tides. This metric-driven approach is implemented through an exiill-atentiinn-aasee peptide encoder architecture, which demonstrates superior capability in discriminating subtle variations among candidates, thereby enabling precise dif- ferentiation in the reranking process.
3.2. Spectrum and Peptide Embedding
Spectrum embedding We filter the peaks in the spectrum & with m/z range of [Pmin, /mma)]. Then, the intensities i of the remaining k peaks are square-root transformed and normalized as I Vi,
ΣE VT,
2.3. Axial Attention
Following previous works, we use fixed sinusoidal embed- ding function sm=/ to project m/z signal into d-dimensional. Since all μ falls between Momin and Mmazz we embed the ratio of μ and pmmn and use Hmaz as the scale basis of wave
Axial attention (Huang et al., 2019; Ho et al., 2019; Wang et al., 2020a;b; Choromanski et al., 2020) markedly re- duces computational complexity while maintaining the abil- ity to capture global context by applying the self-attention mechanism along specific axes of the input data. In the
Pminn
3

---

Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencing
Mathhdds
Matrices
Candidutes
Back Proptaae
Medal A
Poptilee
Multiple Candidate Peptide Construction
Output
Result: B.
Ranask
Peptide C
Ground Truth
Model D
Poprida b
RankNove
Sepetrum Topur
B
RankNovo Model Architecture
Peptide Mass Deviation
Residual Mass Deviation
Multile Peptider Encoder
DP Aoose
Query
Target
Multiiple
Torget V
6
peptides
Terget
W
Aast
Zara
Roguter
Fiuader
Frnk
Precursor
Spectrum Inpus
Spectrum Encoderr
Rankloovo
An overview of the RankNovo architecture. (A) Multiple base models generate peptide sequenee candidates from the spectrum input, which subsequently reranked by RankNovo. (B) The architecture of RankNovo incorporates multi multi-ppptide encoder, utilizing axial attention along both row and column dimensions and cross-attention to effectively integrate spectrum features. (C) The coarse-grate -rained PMD metric assesses peptide-level differences through dynamic programming baased based sequence aligument. (D) The fine- ine-grained RMD
Figure 2.
metric provides a more granular assessment by capturing residue-level mass deviations between the query and target peptides.
length:
[hass h2. hg.................. hg]
can be denoted as:
sm/= (2T wit.. ([((//mm ()),
if 2
h = T (res:) H (m, preliii) Houllx (m; (
Pamis
(1)
c2s((2T1 )/(( (esax
}3)),
if = 2k +
hass = B (cls) + moc (m)r + B chargg (c))c) } + 0a/2
Pain
(3)
Intensity signals are projected to d dimension with a lin- car layer because of its relatively lower accuracy and are summed with corresponding m/z vectors as the initial spec- trum embedding E", with shape [k, d]. Here, no additional positional embeddings are used, the peaks are inherently unordered in nature.
where + denotes the concatenation operation over the se- quence length dimension and Oa/2 denotes learnable 0- quality embedding. RankNovo processes all peptide can- didates in a single forward pass. Thus, each candidate's embedding padded to the longest and stacked into the initial MSA embedding S Additionally, a learnable po- sitional embedding is added each row of S ensuring the model is aware of the token order in a peptide. It is important to note that shuffling the order of rows does not affect the loss prediction due to the absence of column-wise positional embedding.
Peptide Candidate Embedding De novo sequencing is a mass-centric task. Therefore, the prefix and suffix masses of a residual are also embedded in addition to a learnable amino acid embedding, following ContraNovo. Given the model dimension d, the dimensions of the learnable em- bedding, prefix, and suffix, denoted as dres, dpreetis dsutiix are d and T respectively. Moreover, the precursor of spectrum are embedded into dpree (9) dimensions as the sum of fpee-ddimeniina. precursor mass vector and precursor charge vector (Jin et al., 2023). All masses are embedded using fixed sinusoidal positional embedding:
3.3. Accurate Assessment of Peptide Difference
set to 2:5 I
In the context of peptide sequencing tasks, accurate labeling of predictions is crucial for meta-models to effectively iden- tify optimal predictions. Conventional labeling methods for base model predictions, such as binary classification of cor- rectness or edit distance metries, inadequately capture the nuanced differences between predicted and actual peptide sequences, particularly with respect to amino acid masses, which are fundamental to the mass-centric sequencing pro- cess.
sin
2een
for i = 2
H (mu,
for i = 2k ++1 T (prefix, pree)
cos
10000
2am
(2)
While the learnable embedding functions for amino acids and precursor charges can be represented as ee and Beharge
To address these limitations, we introduce PMD and RMD, two novel metries designed for precise quantification of
Then the initial peptide embedding ho =
4

---

Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencing
mass differences between peptides. These metrics provide more informative and accurate supervision for RankNovo to learn from, thereby improving its discriminative capabili- ties. PMD and RMD are complementary training objectives during training.
of the intrinsic properties of mass spectrometry data. In mass spectrometry, peptide bonds between amino acids cleaved, generating b- and y-ions. The b-ions, originating from the N-terminus, offer a detailed structural fingerprint of the peptide.
Peptide-level Assessment (PMD) PMD employs a dy- namic programming approach analogous the Needleman- Wunsch algorithm (Needleman & Wunsch, 1970) for se- quence alignment, with specific focus on amino acid masses. Given set of all possible residues, including both amino acids and post- post-translational modifications (PTMs), defined as R = [r,, 12,,... ........... ) where n denotes the num- ber of distinct residue types, we introduce a corresponding mass look-up table M R -> IR+ Here, M(;;) represents the mass of residue r; for E 21, n}. We define the divergence score matrix P R=**, where
RMD is derived from the prefix masses of the query peptide Q and the target peptide K, denoted as Q m map: ,,,]] and [mk Miz...... mp.... where my = resentation is closely aligned with the b-ion mass spectrum. The RMD between these two sequences is represented as a vector V with elements, where each element is defined
and my, = =E M(re,) This rep-
V. Mg. M)) where x (1) = arg min Mg:
Mag
Here (1)) is a learned alignment function that seeks to mini- mize the mass difference between the Q and K. By training the model to predict RMD, we encourage to capture and distinguish subtle structural deviations between peptides. This residual-level task improves the model's 's ability to iden- tify fine-grained peptide differences, complementing the higher-level insights given by PMD.
- - M(ry)). if 4,j (1, (M((r)) -
0.
if==j
Pij =
(4)
The gap penalty g is formalized as the expected symmetric divergence between two distinct residues, given by:
g g-Bigj (r)) nnn
M(n.) - M(r;)
3.4. Backbone of RankNovo
371
The backbone of RankNovo needs to fulfill three tasks: (1) Extracting spectrum feature, Extracting peptide features within and among candidates, (3) Mixing spectrum feature and peptide feature to score and rerank peptide candidates.
(5)
where Eizj (P ry)] is the expectation of the symmetric divergence between the residues.
Given query peptide sequence Q = |F Foz: q... and a target peptide sequence K- = [iku Tk2...... .... where n and m represent the lengths of the predicted and correct pep- tides respectively, we initialize a matrix F E (m++1) The matrix F is populated using the following recurrence relation:
Spectrum feature extraction can be easily accomplished by a Transformer encoder. After embedding, the initial spectrum representation E is updated by Niner repetitive self-attention layer:
E = Acelr 1)), 1= = 1,2,..., Ninyer
(8)
ifi 1,j =1 iri A 1,J 1 iri 1,J A
On the other hand, a hybrid peptide track is designed to ad- dress tasks (2) and (3) jointly. The peptide track processes the embedded multiple sequence alignment (MSA) feature S |exxtdd where e represents the number of candidates, f is the sequence length, and d is the model dimension. The final spectrum feature E EMisser E R is broadcasted across candidates by repeating it to shape [c, k, d]. and then inte- grated into the peptide track. The feature update mechanism is defined as:
g(i 1): g
Fiy
1).
min
(F, of Pa- F. ts Fis g).
otherwise
E (1.2.....
+ JE (1.2...,++1))
(6)
The final output of PMD between the two peptides is com- puted as F Fu+Lm+1// Dividing by g normalizes the value to an order of magnitude around 100, facilitating model fitting. PMD achieves a score of zero only when the pre- dicted peptide exactly matches the correct peptide, making it a precise metric for peptide distance assessment in mass bpectrometry-baseed proteomics.
S Acrons (Arn
Nuser
where Anw. Aco. and Across denote the row-wise, column- wise, and cross-attention mechanisms, respectively. Here, axial attention is employed to extract peptide features and facilitate information flow between candidate peptides. The iterative application of row and column attention ensures a
Residual-level Assessment (RMD) In addition to the peptide-level metric PMD, which the meta-model uses to select the top prediction, we introduce more fine-grained peptide difference score, RMD. This metric takes advantage
5

---

Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencing
receptive field that spans the entire E x k token grid while maintaining reduced complexity of O(cl² + k2 in con- trast to the (²) complexity of standard multi-head self- attention mechanisms. Cross- Cross-attention is integrated to in- corporate spectrum features into the peptide track, allowing for enhanced alignment between peptides and spectra, and improving overall task performance.
every residue is matched. Here, residual matching means (1) differing by 0.1 Da in mass and (2) both the prefix and suffix differing within 0.5 Da. Also, since previous works evaluate model capabilities at residual-level as well, amino acid precision is also taken into consideration. Here, amino acid precision is defined as / Vai meaning the percentage of matched residuals among all residuals.
3.5. Training with Joint Loss
4.2. Main Results
The final MSA feature S Nuw is utilized to predict the PMD and RMD between each candidate peptide and the label peptide. For the peptide-level metric, PMD, the CLS to- ken of each candidate is extracted and passed through linear layer to predict PMD, formulated as: PMD Linear (bar) E R. Similarly, the d-dimensional representa- tion of each amino acid is projected through linear trans- formation to predict the residue- level RMD, expressed as: RMD- [Linear (h: Naser ) E R)i=-...... Both Penn and FAN are computed using RMSE loss, The optimization objective for training RankNovo is defined as:
Performance on 9-species-V V1 Benchmark Dataset. In our evaluation, RankNovo exhibits superior performance across all species on the pivotal benchmark, 9-species- V1, both at the peptide and amino acid levels (Table 1). Specifically, RankNovo achieves average peptide recall of 0.660, sur- passing its strongest base model, ByNovo, by 6.1 11,,, and outperforming the current state-of-the-arr ContraNovo, by 4.3%. At the amino acid level, RankNovo reaches a pre- cision of 0.829, outperforming Bovo by 2.6% and Con- traNovo by 4.1%. These results underscore RankNovo's ability to accurately sequence peptides and amino acids across diverse species.
= A + (1 - CRMD
(10)
Two key conclusions can be drawn from these results: first, RankNovo establishes new state-of-the-art in de novo peptide sequencing, surpassing the previous benchmark set by ContraNovo; and second, Rank consistently out- performs all its constituent base models, demonstrating its ability to effectively integrate diverse model outputs, leverage their respective strengths, and mitigate individual weaknesses, thereby reducing generalization error.
In this work, A is set 0.5 consistently.
4. Experiments
4.1. Experiment Setup
Datasets. Following the precedent set by recent studies (Yil- maz et al., 2023; Zhang et al., 2024), we employ three public peptide-spectrune match (PSMs) datasets: MassIVE- KB (Wang et al., 2018) for training, and 9-species-Vi V1 (Tran et al., 2017) and 9-species-VV V2 (Yilmaz et al., 2023) for eval- uation, enabling comparisons with state-of-the-art de novo peptide sequencing methods. Detailed dataset information is provided in the Appendix A.
Performance on 9-species- V2 Benchmark Dataset. The experimental results in Table clearly indicate that Ran- kNovo consistently outperforms both baseline and compar- ative models on the 9-species-V2 dataset, demonstrating superior performance in amino acid precision and peptide recall. Specifically, RankNovo achieves the highest aver- age amino acid precision of 0.906 across all species, with substantial improvements in species such as Bacillus, C. bacteria, and Honeybee. Furthermore, RankNovo attains an average peptide recall of 0.781, outperforming other mod- els across the majority of species, with particularly strong performance in Yeast, Rice bean, and Tomato. These results emphasize the adaptability and effectiveness of RankNovo across diverse set of species.
Implementation Details. RonkNovo incorporates six de novo sequencing models, each varying in methodol- ogy, as base models during training. These models in- clude Casanovo-V2, ContraNovo, ByNovo, R-Casanovo, R-ContraNovo, and R-ByNovo. Of these, Casanovo V2 and ContraNovo are directly adopted from the original works and represent both the current and previous state- of-the-art approaches. The latter four models, ByNovo, R- R-ContraNovo, and R-ByNovo, are developed and trained by us. The details of base models, training settings, and hyperparameters of RankNovo can be found in Appendix B.
4.3. Detailed Analyses
We report average performance the 9-species-V V1 bench- mark, with detailed per-species performance in Appendix E.
Metrics. Since the reranking task only concerns peptide- level selection, the widely accepted metric, peptide recall, is our most important metric. Peptide recall is defined as Npep /eeet/NPP /NEeP, here Neee, is the number of matched peptides and No is the number of total peptides. The identified peptide is regarded as matched to the label peptide only if
Analysis of Zero-shot Performance. We demonstrate the zero-shot capability of RankNovo by training it exclusively on predictions from the two powest-performing base models and progressively incorporating predictions from unseen
all
6

---

Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencing
Methods
Bacillus
C. bucteria
Honeybee
Human
M.mazei
Mouse
Rice bean
Tomato
Yeust
Average
9-species-Vi V1
PEAKS
0.719
0.586
0.633
0,639
0.673
0.600
0.644
0.728
0.748
0.663
Amino
DeepNovo
0.742
0.602
0.630
0,610
0.694
0.623
0.679
0.731
0.750
0.673
Acid
PointNovo
0.768
0.589
0.644
0.606
0.712
0.626
0.730
0.733
0.779
0.687
Casanovo
0.749
0.603
0.629
0.586
0.689
0.668
0.721
0.667
Precision
Casanovo V2f
0.806
0.685
0.727
0.690
0.774
0.768
0.769
0.799
0.762
0.753
ContraNovo
0.828
0.706
0.761
0.771
0.798
0.799
0.804
0.808
0.782
0.784
ByNovo
0.858
0.723
0.791
0.767
0.823
0.803
0.836
0.828
0.804
0.804
RankNovo
0.874
0.746
0.810
0.802
0.840
0.828
0.859
0.844
0.816
0.824
PEAKS
0.387
0.203
0.287
0.277
0.356
0.197
0.362
0.403
0.428
0.322
DeepNovo
0.449
0.253
0.330
0.293
0,422
0.286
0.4.66
0.454
0.462
0.376
Peptide
PointNovo
0.518
0.298
0.396
0.351
0.478
0.355
0.511
0.513
0.534
0,439
Recall
Casanovo
0.537
0.330
0.406
0,426
0,506
0,521
0.490
0.448
Casanovo V2l
0.646
0.460
0.527
0,492
0.592
0.493
0.628
0.637
0.629
0.567
ContraNovo
0.684
0.487
0.576
0.624
0.628
0.563
0.676
0.655
0.669
0.618
ByNovo
0.708
0.499
0.597
0.584
0.639
0.545
0.696
0.667
0.676
0.623
RankNovo
0.738
0.539
0.630
0.642
0.672
0.583
0.733
0.691
0.703
0.660
9-species-V2 V2
Amino
Casanovo V2l
0,888
0.791
0.823
0.872
0.877
0.813
0.891
0.891
0.915
0.862
Acid
ContraNovo
0.901
0.807
0,848
0.920
0.896
0.839
0.913
0.898
0.919
0.882
Precision
ByNovo**
0.920
0.823
0.876
0.917
0.914
0.841
0.932
0.912
0.934
0.897
RankNovo
0.926
0.838
0.885
0.929
0.920
0.860
0.938
0.918
0.938
0.906
Peptide
Casanovo V2
0.793
0.558
0.669
0.712
0.754
0.555
0.772
0.783
0.837
0.714
NontraNovol
0.815
0.575
0.711
0.820
0.780
0.616
0.799
0.794
0.854
0.752
Recall
ByNovo
0.833
0.582
0.731
0.789
0.799
U.596
0.814
0.807
0.871
0.75%
RankNovo 0.851
0.620
0.752
0,820
0.813
0.629
0.833
0.822
0.885
0.781
Table 1. Evaluation of RankNovo in comparison to baseline and base models on the 9-species-V1 and 9-species-Vi V2 test sets. Bolded entries indicate the best-performing models. The symbol "I" indicates that the model serves as both baseline and base model. "x" signifies that the base models were developed and trained by us. Here, ByNovo is the best base model. So the performance of three self-trained base models (R-Casa, R-Contra, R-By) only provided in Appendix D.I.
(A)
(0)
0.855
20.30 3%
20 4%
0.5
0.9
0.825
8.9%
0.6 6.8%
0.8
0.600
10
Roaal
0.7
25.3%
0.4
Gassnoee
Contrnoov
Casarino 1/2
R.Caemooo R-Cormalion
02
Ours
Cantralovo
Number of Inference Models
Byhlovo
R-BpNowo
Poptido Lorgth
M101
Figure 3. (A) Zero-shot performance of RankNovo when trained on two base models. (B) Unique-correctly selected percentage base models. (C) Influence of peptide length. (D) The performance comparison of amino acids with similar masses.
models into the candidate sets during inference (Figure 3 (A)). The detailed experimental setup is provided in Ap- pendix E.I. As the number of inference models increases, the average peptide recall improves, rising from 0.586 with 2 models to 0.649 with 6 models. These results highlight the robust zero-shot capability of RankNovo, demonstrating its efficiency in reranking predictions from models not used during training. This underscores the value of RankNovo for future applications in de novo peptide sequencing.
trum samples to retain those where (1) RankNovo's chosen peptide matched the labeled peptide and (2) RankNovo's choice was provided by only one base model. For these filtered samples, calculated the percentage of times each base model was chosen, using this as measure of contribu- tion. As illustrated in Figure 3 (B), Casanovo-V? V2 had the lowest contribution at 6%, while R-ByNovo had the highest at 30%. These results demonstrate that every model con- tributes to the final performance, as removing any of them would lead to failures on specific test samples.
Contribution of Each Base Model. Given the varying ca- pabilities of base models, it is crucial to ensure that each contributes meaningfully to RankNovo's performance. Oth- erwise, their predictions may introduce unnecessary noise during the reranking process. We analyzed peptide candi- dates' and RankNovo's selections using the Bacillus species data from the 9- 9-species- V1 benchmark. We filtered spec-
Analysis of Peptide Length. We assess the performance of RankNovo and baselines in recognizing peptides of varying lengths, with particular emphasis on their effectiveness for both shorter and longer peptides. As shown in Figure 3 (C)), our findings reveal that RankNovo exhibits significantly higher recall compared ContraNovo for shorter peptides,
7

---

Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencing
Ohjective
Avg. Pep. Recall
ID
PMD
RMD
Col-Attn.
Avg. Pep. Recall
Point-wise
0.647
6.70
1
0.650
Pair-wise
0.648
List-wise 0.646
2
v
0.652
PMD+RMD 0.660
3
v
0.653
0:05
4
0.660
090
Table 2. Average Peptide Recall on 9- species-VI I test set under the training ob- jective of different reranking losses.
Table 3. Ablation of training metrics combina- tion and column-wise attention modules.
Table 4. Ablation study of base model combinations.
suggesting enhanced proficiency in recognizing these se- quences. As analyze longer peptides, discemnble trend emerges: the recall for both models shows downward trajectory, indicative of a decline in recognition capability as peptide length increases. This reduction in performance can be attributed to the heightened complexity associated with longer peptide structures, which may impede model ac- curacy. Nevertheless, RankNovo consistently outperforms ContraNovo, although the margin of superiority narrows with increasing peptide length.
descriptions are provided in Appendix C. As shown in Ta- ble 2, the three frameworks exhibit comparable performance when reranking de novo sequencing results, achieving ap- proximately 0.647 peptide recall. However, this falls sig- nificantly short of the 0.660 recall achieved by RankNovo using our novel metrics, PMD and RMD. These results un- derscore the specialized efficacy of the RankNovo in peptide sequencing tasks.
Base Model Combinations Ablation. We conducted abla- tion studies to assess the impact of using fewer base models on overall performance. We created five subsets of the fi- nal base model set, each containing a different number of base models, and compared the performance of RankNovo when trained and tested with the outputs of these model sets. The selection criteria for these subsets are detailed in Appendix D.2. The results (Figure 4) demonstrate consis- tent increase in peptide recall as the number of base models increases. This observation supports the hypothesis that a greater diversity of choices leads to improved performance.
Analysis of Amino Acids with Similar Masses. In de novo peptide sequencing, a sequence is deemed accurately recon- structed only when each residue in the predicted peptide aligns with its corresponding residue in the reference se- quence. Prediction accuracy varies across different amino acids, particularly for those with similar masses, which are challenging to distinguish due to nearly overlapping spec- tral profles. For instance, oxidized methionine (M(O)) and phenylalanine (F) differ by 0.33 Da, while lysine (K) and glutamine (Q) differ by 0.46 Da. Figure 3 (D))compares RankNovo with two baseline models, Casanovo-Ve V2 and ContraNovo. Utilizing the 9-species- V1 dataset, recall was computed for each amino acid. Notably, RankNovo achieves an 8.0% improvement in recall for M(O) relative the baseline models. These results underscore RankNovo's en- hanced ability to differentiate between amino acids with closely related masses, effectively capturing subtle mass variations within peptide sequences.
Training Objective Ablation. Two novel metrics, PMD and RMD, provide the learning objective for RankNovo. The results of experiments 2, and 4 in Table 3 demonstrate that the absence of either metrie leads to decrease in peptide recall. The combination of both metrics is necessary to achieve optimal performance.
Backbone Model Ablation. The results of experiments 3 and 4 in Table reveal a decline performance without the column- wise attention module, as evidenced by the 0.653 peptide recall after its removal, compared to the original 0.660. This finding supports the hypothesis that incorpo- rating axial attention facilitates the integration of peptide features and contributes to optimal performance.
4.4. Reranking Framework Comparison and Ablation Study
We compare the performance of RankNovo with other reranking frameworks and conduct ablation studies on its key components. The evaluation is performed on the 9- species- V1 benchmark. Detailed results and analysis are presented in Appendix C and D.
5. Conclusion
In this paper, we introduced RankNovo, novel list-wise deep reranking framework designed to enhance the accu- racy of de novo peptide sequencing under the guidance of our mass deviation metrics, PMD and RMD. RankNovo achieves new state-of-the-art performance on established benchmarks and exhibits strong zero-shot generalization capabilities. The primary limitation of RankNovo lies in the
Reranking Framework Comparison. We compare Ran- kNovo with three types of reranking frameworks outlined in RankT5 (Zhuang et al., 2023): posnt-wise, pair-wise, and list-wise reranking. Following RankT5's methodology, these frameworks are implemented using identical backbone models, differing only in their training objectives. Detailed
8

---

Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencing
relatively lower inference speed due the proportional time cost in collecting peptide candidates (Appendix E.5). Future work could explore efficient candidate sampling methods, such as utilizing base models with partially shared weights to reduce computational overhead.
References
Abramson, J., Adler, J., Dunger, J., Evans, R., Green, T., Pritzel, A., Ronneberger, O., Willmore, L., Ballard, A. J., Bambrick, J., et al. Accurate structure prediction of biomolecular interactions with alphafold 3. Nature, pp. 1-3, 2024.
Despite the speed constraints, RankNovo represents the first deep reranking framework offer a flexible trade-off be- tween inference time and performance, introducing novel perspective for performance enhancement. We anticipate that, influenced by RonkNovo, future algorithms in this field will benefit from the synergistic approach of simultanc- ously improving single-model performance and developing advanced reranking strategies.
Aebersold, R. and Mann, M. Mass spectrometry- proteomics. Nature, 422(6928) 207, 2003, ISSN 1476-4687. URL https: /owii/oiiioo www.ooog//..../.oo nature01511.
Bittremieux, W., May, D. H.,, Bilmes, J., and Noble, W. A learned embedding for efficient joint analysis of millions mass spectra. Nature methods, 19(6):675-67) 2022.
Impact Statement
Breci, L. A., Tabb, D. L., Yates, J. R., and Wysocki, Cleavage n n-terminal to proline: analysis of database of peptide tandem mass spectra. Analytical chemistry, 75 (9):1963-19) 1971, 2003.
Our work advances machine learning applications in pro- teomics by developing novel deep learning framework for peptide de novo sequencing from tandem mass spectrometry data. By improving sequencing accuracy through seq-to-seq architecture with reranking optimization, this method could accelerate biomedical discoveries disease biomarker iden- tification and personalized medicine development.
Burges, C., Shaked, T., Renshaw, E., Lazier, A., Deeds, M., Hamilton, N., and Hullender, G. Learning to rank using gradient descent. In Proceedings ofe 22nd international conference on Machine learning, pp. 89-96, 2005.
We foresee two primary societal impacts: (1) Enabling more accessible protein analysis that could lower costs for clinical proteomics applications, and (2) Facilitating drug discovery through improved characterization of therapeutic peptides. Potential ethical considerations include ensuring equitable access to this technology across research communities and responsible handling of sensitive health data in clinical ap- plications.
Burges, C. From ranknet to lambdarank to lambdamart: An overview. Learning. 11(23-581):81, 2010.
Cao, Z., Qin, T., Liu, T.-Y., Tsai, M.-F., and Li, Learning rank: from pairwise approach listwise approach. In Proceedings of the 24th international conference Machine learning, pp. 129-136, 2007.
While there are no immediate malicious use cases inher- ent to the algorithm itself, we acknowledge that any pro- teomics advancement carries dual-use potential. To mitigate risks, we will open-source our model with usage guidelines and collaborate with domain experts establish best prac- tices for clinical translation. This work ultimately aims to strengthen the positive impact of machine learning in driving scientific discovery while maintaining responsible innovation principles.
Chang, C.,, Zhang. J., Xu, C., Zhao, Y., Ma, J.. Chen, T., Hc, F., Xie, H., and Zhu, Y. Quantitative and in-depth survey the isotopic abundance distribution errors in shotgun proteomics. Analytical chemistry, 88(13):6844-633 2016.
Chen, C., Hou, Tanner, J., and Cheng, J. Bioinformatics methods for mass spectrometry-basee proteomics data analysis. International journal of molecular sciences, 21 (8):2873, 2020.
2020..
Acknowledgement
Chi, H., Sun, R.-X., Yang, B., Song, C.-Q., Wang, L.-H., Liu, C., Fu, Y., Yuan, Z.-F., Wang, H.-P., He, S.-M., et al. pnovo: de novo peptide sequencing and identification using hed spectra. Journal of proteome research, 9(5): 2713-2724 2010.
This project was partially supported by Shanghai Artificial Intelligence Laboratory (S.S.). This work is partially sup- ported by Netmind.Al and ProtagoLabs Inc. This work is also partially supported by CURE (Hui-Chun Chin and Tsung-Dar Lee Chinese Undergraduate Research Endow- ment) (24924), and the National Undergraduate Training Program on Innovation and Entrepreneurship grant(24924).
Chi, H., Chen, H., He, K., Wu, L., Yang, B., Sun, R.-X., Liu, J.. Zeng, W.FF. Song, C.-Q., He, S.-., et al. pnovo+: de novo peptide sequencing using complementary hed and etd tandem mass spectra. Journal of proteome research, 12(2):615-66 2013.

---

Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencing
Chi, H., Liu, C., Yang, H., Zeng, W.FF, Wu, L.,, Zhou, W.-., Wang, R.-M., Niu, X.-N., Ding, Y.-H., Zhang, Y., al. Comprehensive identification of peptides in tandem mass spectra using an efficient open search engine. Nature biotechnology, 165511):00-1111, 2018.
Ho, J., Kalchbrenner, N., Weissenborn, D., and Salimans, T. Axial attention in multidimensionaal transformers. arXiv preprint arXiv: 191.1::89122218000 2019.
Huang, Z.,, Wang. X., Huang, L.,, Huang, C., Wei, Y., and Liu, W. Ccnet: Criss-cross attention for semantic segmen- tation. In Proceedings of the IEEF/CVF international conference computer vision, pp. 603-612, 2019.
Choromanski, K., Likhosherstov, V., Dohan, D., Song, X., Gane, A., Sarlos, T., Hawkins, P., Davis, J., Mohiuddin, A., Kaiser, L., et Rethinking attention with performers. arXiv preprint XXXx:20009.1499. 2020.
Jiang, D., Ren, X., and Lin, B. Y. LIm-blender: Ensembling large language models with pairwise ranking and genera- tive fusion. arXiv preprint arXiv:2306.02333 02561,,, 2023.
Danéík, V., Addona, T. A., Clauser, K. R., Vath, J. E.,, and Pevzner, P. A. De novo peptide sequencing via tandem mass spectrometry. Journal of computational biology, 6 (3-4):327-342, 1999.
Jin, Z., Xu, S., Zhang, X., Ling, T., Dong, N., Ouyang, W., Gao, Z.,, Chang, C.,, and Sun, S. Contranovo: contrastive learning approach to enhance de novo peptide sequencing. arxiv. arXiv preprint arXiv:2312. 1158,, 2023.
Eloff, K., Kalogeropoulos, K., Morell, O., Mabona, A., Jes- persen, J. B., Williams, W., Beljouw, S. P. v., Skwark, M., Laustsen, A. H., Brouns, S. J., et al. De novo peptide sequencing with instanovo: Accurate, database-free pep- tide identification for large scale proteomies experiments. bioRxiv, pp. 2023-08, 2023a.
Jin, Z., Xu, S., Zhang, X., Ling, T., Dong, N., Ouyang, W., Gao, Z., Chang, C., and Sun, S. Contranovo: A contrastive learning approach to enhance de novo peptide sequencing. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 144-152, 2024.
Eloff, K., Kalogeropoulos, K., Morell, O., Mabona, A., Jes- persen, J. B., Williams, W., van Beljouw, S. P., Skwark, M., Laustsen, A. H., Brouns, S. J., et al. De novo peptide sequencing with instanovo: Accurate, database-free pep- tide identification for large scale proteomics experiments. bioRxiv, pp. 2023-08, 2023b.
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. Highly accurate protein structure prediction with alphafold. Nature, 596(7873):583-588)) 2021.
Karunratanakul, K., Tang, H.-Y., Speicher, D. W., Chuang- suwanich, E., and Sriswasdi. S. Uncovering thousands of new peptides with dequence-mask-seanee hybrid de novo peptide sequencing framework. Molecular & Cellular Proteomics, 1 (8112):2478-244
Feng, D.-F and Doolittle, R. F. Progressive sequence align- ment as a prerequisitetto correct phylogenetic trees. Jour- nal of molecular evolution, 25:351-360, 1987.
Gao, J., Chen, J., Wei, J., Jiang, B., and Luo, A.L Deep multimodal networks for m-type star classification with paired spectrum and photometric image. Publications of the Astronomical Society of the Pacific, 135(1046): 044503, 2023.
LeCun, Y., Bengio, Y., and Hinton, G. Deep learning. nature, 521(7553):436 2015.
Leprevost, F. V., Valente, R. H., Lima, D. B., Perales, J., Mclani, R., Yates III, J. R., Barbosa, V. C., Junqueira, M., and Carvalho, P. C. Pepexplorer: a similarity- for analyzing de novo sequencing results. Molecular & Cellular Proteomics, 1309):240-22889 2014.
tool
Gao, L., Dai, Z., and Callan, J. Rethink training of bert rerankers in multi-stage retrieval pipeline. In Advances in Information Retrieval: 43rd European Conference on IR Research, ECIR 2021, Virtual Event, March 28-April 2021, Proceedings, Part II 43, pp. 280- 226. Springer, 2021.
Liu, K., Ye, Y., Li, S., and Tang, H. Accurate de novo pep- tide sequencing using fully convolutional neural networks. Nature Communications, 14(1):7974, 2023.
Han, S., Wang, X., Bendersky, M., and Najork, M. Learning-to ttrrinn with bert in tf-ranking, arXiv preprint arXiv:2004.088: 088766 2020.
Ma, B. Novor:
real-time peptide de novo sequencing soft- ware. Journal of the American Society for Mass Spec- trometry, 26(11):1888) 18)1 2015.
Hettich, R. I., Pan, C., Chourey, K., and Giannone, R. J. Metaprotcomics: harnessing the power of high perfor- mance mass spectrometry to identify the suite of proteins that control metabolic activities in microbial communities. Analytical chemistry, 85(9):4203 4214,, 2013.
Ma, B., Zhang, K., Hendrie, C., Liang, C., Li, M., Doherty- Kirby, A., and Lajoie, G. Peaks: powerful software for peptide de novo sequencing by tandem mass spectrome- try. Rapid communccations mass spectrometry, 17(20): 2337-2342, 2003.
10

---

Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencing
Mao, Z., Zhang, R., Xin, L.,, and Li, M. Mitigating the miisingg-ragmeentttin problem in de novo peptide se- quencing with two-stage graph-based deep learning model. Nature Machine Intelligence, 5(11):1250-1260, 2023.
and Nesvizhskii, A. I. iprophet: multi-level integrative analysis of shotgun proteomic data improves peptide and protein identification rates and enrrr estimates. Molecular & cellular proteomics, (1112))1111000000
2011.
Spivak, M., Weston, J., Bottou, L., Kall, L.., and Noble, W. S. Improvements to the percolator algorithm for peptide identification from shotgun proteomics data sets. Journal of proteome research, 8(7):3777-3745, 2009.
Mcllwain, S., Tamura, K., Kertesz-Farkas, A., Grant, C. E., Diament, B., Frewen, B., Howbert, J. J., Hoopmann, M. R., Kall, L., Eng. J. K., et al. Crux: rapid open source protein tandem mass spectrometry analysis. Journal of proteome research, 13(10):4488-444 4491, 2014.
Tabb, D. L.,, Smith, L. L.,, Breci, L. A.. Wysocki, V. H., Lin, D., and Yates, J. R. Statistical characterization of ion trap tandem mass spectra from doubly charged tryptic peptides. Analytical chemistry, 75(5):1155 11651163, 2003.
Muth, T., Hartkopf, F., Vaudel, M., and Renard, B. Y. A potential golden age to cases, and future avenues for de novo sequencing in pro- teomics. Proteomics, 18(18): 700150, 700150,, 2018.
tools, recent use
Tran, N. H., Zhang, X., Xin, L., Shan, B., and Li, M. De novo peptide sequencing by deep learning. Proceedings of the National Academy of Sciences, 114(31):224782522 2017.
Needleman, S. B. and Wunsch, C. D. A general method applicable to the search for similarities in the amino acid sequence of two proteins. Journal of molecular biology, 48(3):443-453, 1970.
Wang. H., Zhu, Y., Green, B., Adam, H., Yuille, A., and Chen, L.-C. Axial-deeplab: Stand-alone axial-attention for panoptic segmentation. In European conference on computer vision, pp. 108 126. Springer. 2020a.
Nesvizhskii, A. I., Keller, A., Kolker, E., and Aebersold, R. A statistical model for identifying proteins by tandem mass spectrometry. Analytical chemistry, (raiiist).771774466666 75(17):4646- 4658, 2003.
Wang. M., Wang. J., Carver, J., Pullman, B. S., Cha, S. W., and Bandeira, N. Assembling the community dis- coverable human proteome. Cell systems, 7(4):412-47) -21,, 2018.
Ng, C. A., Zhou, Y., and Yao, Z.PP. Algorithms for de-novo sequencing of peptides by tandem mass spec- trometry: A review. Analytica Chimica Acta, pp. 341330, 2023.
Wang. S., Li, B. Z., Khabsa, M., Fang, H., and Ma, H. Linformer: Self-attention with linear complexity. arXiv preprint arXiv:2006..o 04768, 2020b.
Nogueira, R., Yang, W., Cho, K., and Lin, J. Multi- stage document ranking with bert. arXiv preprint arXiv: 14424, 2019.
Wu, S., Luan, Z., Fu, Z., Wang, Q., and Guo, T. Biatnovo: A self-attention based bidirectional peptide sequencing method. bioRxiv, pp. 2023-05, 2023.
Nogueira, R., Jiang, Z., and Lin, Document ranking with pretrained arXiv:2003.067133 2020.
model. arXiv preprint
Xia, F,, Liu, T.-Y,, Wang, J., Zhang, W., and Li, H. List- wise approach to learning to rank: theory and algorithm. In Proceedings of the 25th international conference on Machine learning, pp. 1192-1 1 199, 2008
Ouyang. L., Wu, J.. Jiang, X., Almeida, D.,. Wainwright, C., Mishkin, P., Zhang. C., Agarwal, S., Slama, K., Ray, A., et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730-27744, 2022.
Xia, J., Chen, S., Zhou, J., Ling, T., Du, W., Liu, S., and Li, S. Z. Adanovo: Adaptive' emph (De Novo} peptide sequencing with conditional mutual information. arXiv preprint arXiv:2403.0701: 2024a.
Rao, M., Liu, J., Verkuil, R., Meier, J., Canny, J., Abbeel, P.. Sercu, T., and Rives, A. Msa transformer. In Interna- fional Conference on Machine Learning, pp. 8844-8856. PMLR. 2021.
Xia, J., Liu, S., Zhou, J., Chen, S., Xiang, H., Liu, Z.,, Liu, Y., and Li, S. Z. Bridging the gap between database search and de novo peptide sequencing with searchnovo. bioRxiv, pp. 2024-10, 2024b.
Ren, R., Qu, Y., Liu, J., Zhao, W. X., She, Q. Wu, H., Wang, H., and Wen, J.-R. Rocketqav2: A joint training method for dense passage retrieval and passage re-ranking. arXiv preprint arXiv:2110.07361 2021.
Yang, H. Chi, H., Zeng. W.-F., Zhou, W.J., and He, S.- M. pnovo 3: precise de novo peptide sequencing using a learning-to-ras to-k framework. Bioinformatics, 35(14): i183-i190, 2019.
Shteynberg, D., Deutsch, E. W. Lam, H., Eng, J. K., Sun, Z., Tasman, N., Mendoza, I.., Moritz, I.,, Aebersold, R.,

---

Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencing
Yang, T., Ling, T., Sun, B., Liang, Z., Xu, F., Huang, X., Xie, L.., He, Li, L.,, He, F., et al. Introducing x-helixnovo for practical large-scale de novo peptide sequencing. Brief- ings in Bioinformatics, 25(2)bbbae211, 2024a.
Zubarev, R. and Mann, M. On the proper of mass accu- racy in proteomics. Molecular & Cellular Proteomics, 6 (3):777-38,, 2007.
Yang, T., Ling, T., Sun, B., Liang, Z., Xu, F., Huang, X., Xie, L., He, Y., L.. He, F., al. Introducing r-helixnovo for practical large-scale de novo peptide sequencing. Brief- ings in Bioinformatics, 25(2):bba0221, 2024b.
Yilmaz, M., Fondrie, W., Bittremieux, W., Oh, S., and No- ble, W. novo mass spectrometry peptide sequencing with a transformer model. International Conference on Machine Learning, pp. 25514-25522. PMLR, 2022.
Yilmaz, M., Fondrie, W. E., Bittremieux, W., Melendez, C. F. Nelson, R., Ananth, V., Oh, S., and Noble, W. S. soquence-to-squeenee translation from mass spectra to peptides with a transformer model. BioRxiv, pp. 2023 01, 2023.
Zhang. X., Ling, T., Jin, Z., Xu, S., Gao, Z., Sun, B., Qiu, Z., Dong, N., Wang, G., Wang, G., et al. n-primenovo: An accurate and efficient non-autoreeressive deep learning model for de novo peptide sequencing. bioRxiv, pp. 2024- 05, 2024.
Zhang, X., Ling, T., Jin, Z., Xu, S., Gao, Z., Sun, B., Qiu, Z., Wei. J., Dong, N., Wang, G., et al. T-primenovo: an accu- rate and efficient non-autoregressive deep learning model for novo peptide sequencing. Nature Communications, 16(1):267, 2025.
Zhou, J., Chen, S., Xia, J., Liu, S., Ling, T., Du, W., Liu, Y., Yin, J., and Li, S. Z. Novobench: Benchmarking deep learning-based de novo peptide sequencing methods in protcomics. arXiv preprint arXiv:2406.11906 2024.
Zhou, X.-X,, Zeng, W.FF. Chi, H., Luo, C., Liu, C., Zhan, J., He, S.-M., and Zhang, Z. pdeep: predicting ms/ms spectra of peptides with deep learning. Analytical chemistry, 89
12690-02691226 2017.
Zhuang, H., Qin, Z.. Jagerman, R., Hui. K., Ma, J., Lu, J., Ni, J., Wang, X., and Bendersky, M. Rankt5: Fine-tuning t5 for text ranking with ranking losses. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 2308- 2313, 2023.
Zolg. D. P., Wilhelm, M., Schmidt, T., Médard, G., Zerweck, J.. Knaute, T., Wenschuh, H., Reimer, U.. Schnatbaum, K., and Kuster, B. Proteometools: Systematic charac- terization of 21 post-translational protein modifications by liquid chromatography tandem mass spectrometry (Ic- ms/ms) using synthetic peptides. Molecular & Cellular Proteomics, (79):185011661 2018
12

---

Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencing
A. Dataset Details
MassIVE-KB is a widely used training dataset employed in previous studies such as GLEAMS (Bittremieux et al., 2022) and CasaNovo. Its popularity stems from its substantial size, comprising 30 million PSMs, and its diverse distribution, characterized by sources from different instruments and rich variety of posttranslationa modifications. The 9-species-V dataset, introduced by DeepNovo, contains approximately 1.5 million PSMs with database search false discovery rate of 1 %. These PSMs are derived from nine distinct experiments conducted on the same instrument but analyzing peptides from various species, ensuring the dataset's 's diversity. The 9-species- V2 dataset, an updated version of 9-species-VI collected in CasaNovo-V2, contains 2.8 million PSMs. Building upon VI, V2 was refined using the Crux protein identification tool (MeIlwain et al., 2014) and filtered with a Percolator (Spivak et al., 2009) q- value < 0.01, enhancing its quality.
In addition to the PSMs datasets, we collected the best peptide predictions from six baseline models: Casanovo, ContraNovo, ByNovo, Re-Casanovo, Re-ContraNovo, and Re-ByNovo for each spectrum. Due the substantial size of the MassIVE-KB training dataset and the computational constraints of beam search, we employed greedy decoding for peptide collection in the training phase. Additionally, spectrums that are correctly predicted by all six base models are excluded, leaving 7 million spectrums for the training set. Conversely, for the evaluation datasets (9-species-V) V1 and 9-species-V22 we utilized beam search decoding with a beam size of 5. This approach aligns with previous works and enables optimal benchmark performance during evaluation.
B. Implementation Details
B.I. Hyperparameters
RankNovo is implemented with the following hyperparameters: 8 layers for both the spectrum encoder and peptide feature mixer, 8 attention heads, model dimension of 512, feed-forward dimension of 1024, and a dropout rate of 0.30.
For spectrum and peptide preprocessing, spectra are filtered according to the following criteria: minimum m/z ratio of 50.5 Da, maximum m/z ratio of 4500.0 Da, maximum peak number of 300, precursor m/z tolerance of 2.0 Da, and precursor mass tolerance of 50 ppm. Spectra with more than 300 peaks are truncated, retaining only the 300 peaks with the highest intensities. Spectra that do not satisfy the precursor m/z tolerance and precursor mass tolerance are removed. Additionally, peptides longer than 100 amino acids truncated. During evaluation, all base models generate peptides using beam search with size of 5.
RankNovo is trained using an Adam W optimizer with a learning rate of le-4 and weight decay of 8e-5, The model is trained with a batch size of 256 for 5 epochs, including 1-epoch warm-up period. A cosine learning rate scheduler is employed, and gradients are clipped to 1.5 using L2 norm. The training is conducted on 4 A100 40G GPUs.
B.2. Baselines
Our benchmark evaluation first compares RankNovo with its base model components to assess the effectiveness of the reranking framework. The components include Casanovo- V2, ContraNovo, ByNovo, R-Casanovo, R-ContraNovo, and R- ByNovo, which collectively represent both current and previous state-of-the-ar art models for de novo sequencing, particularly ContraNovo and Casanovo-V2. For consistency with prior work, we also evaluate four additional benchmark algorithms: DeepNovo, PointNovo, Casanovo- V1, and PEAKS. Notably, PEAKS employs dynamic programming based approach, while the remaining three are deep baaeed models.
B.3. Base Model Selection
The selection of base models decides the performance upper bound of RankNovo. The selection of base models should follow three criteria: (I) The training datasets of each base model should have no interest with the test dataset, which is common data leakage problem in ensemble learning. (2) Base models should be diverse in data preference.
In our research, we selected six models for de novo peptide sequencing as our base models: Casanovo (Yilmaz et al., 2022; 2023), ContraNovo (Jin et al., 2024), ByNovo, R-Casanovo, R-ContraNovo, and R- R-ByNovo as shown in Figure 4. All these models are Transformer-based. but each employs different methodologies. Casanovo and ContraNovo are based on previous work, and we directly use the official checkpoints for these models. The latter four models, ByNovo, R- Rovo,,raNovo, and R- R-ByNovo, are developed and trained by us:
13

---

Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencing
Casonovo
ContraNovo
ByNovo
Al Milhi
Spectrun Excoder
Spectrum Excoder
B/V lons Classificatio
Encoder
Contrastive
PEPTIDE
Peptide Eaceder
Cress Entropy
PSPTIDE
Gress Lous
PEPTIDE
Peptide Encoder
Gross Entropy Lass
R- R-Casanovo
R- RoCotrraNove
B-ByNovo
Spectrum Enceder
Spectrum Encoder
Encoder
classification
Controstive
Peptide
Cress Entropy Loss
EDITPE*
Peptide Encoder
Gross Entropy
Peptide Encoder
Cross Entrapy Less
Lass
Figure 4. The architecture of six base models for de novo peptide sequencing.
1. CasaNovo V2: This fundamental Transformer- de novo sequencing model treats de novo sequencing as sequence-to-sequence machine translation task, translating spectra into peptides.
2. ContraNovo: This model leverages contrastive learning to enhance feature extraction. It also more effectively utilizes amino acid masses, as well as prefix and suffix masses, during the encoding and decoding processes.
3. ByNovo: Developed in-house to increase model diversity, ByNovo incorporates the prediction of BY ions using the
output of the spectrum encoder as auxiliary task. Refer Appendix B.4 for ByNovo implementation details.
4. R-Casanovo: Inspired by recent studies (Eloff et al., 2023a; Wu et al., 2023), this model trains to decode peptide sequences in reverse. The one-way nature of augo-segressive decoding leads to different results when the sequence decoded in reverse. R-Casanovo the reverse-decoding version of Casanovo.
5. R-ContraNovo: The reverse decoding version of ContraNovo.
6. R-ByNovo: The reverse decoding version of ByNovo.
To avoid the data leakage problem, these six models are all trained on Massive-KB, the largest available de novo peptide sequencing dataset in public, and evaluated in zero-shot manner the Nine-Species and Nine-Species V2 datasets, following previous works. On the other hand, the difference in methodologies successfully leads to the variety in data preference of base models (Fig. 1). These settings meet the criteria for base model selection. The architectures of these six models in detail can be found in the Appendix.
B.4. ByNovo Details
ByNovo introduces an auxiliary task for identifying key ion peaks alongside the primary peptide sequencing task. Real- world spectral data often contain numerous noise PEAKS and ion PEAKS that are weakly related or unrelated to de novo sequencing, which can interfere with the model's performance (Breci et al., 2003; Tabb et al., 2003). To mitigate this issue, ByNovo incorporates ion peak identification as an auxiliary objective, modeling ion recognition as a token-level classification problem by introducing an ion annotation head within the encoder.
Specifically, ByNovo first labels the ion PEAKS in the spectrum, assigning an ion type other) to each peak. The task is formalized as maximizing the conditional probability of predicting the ion type 4, given the mass-to-charge ratio m, and charge 2;, as shown in the equation:
e (misies
P(4 m; 0) = EveL (m
(11)
14

---

Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencing
where f(-) is the classification model, and 0 denotes the model parameters. For the entire ion peak sequence in spectrum, ByNovo maximizes the joint probability, as defined in the equation:
PLL S)
II P(4 my, 2,,
(12)
In this classification task, ByNovo uses Focal Loss as the supervision loss function, defined in the equation:
Cion = (1 pt)" log(p:)
(13)
where P: is the predicted probability the correct class, and y > 0 is a focusing parameter. Focal Loss assigns smaller penalties to well-classified examples with high confidence, while increasing the loss for hard-to-classee -olassssyl samples. This encourages the model to focus on learning difficult examples, which beneficial for detecting easily overlooked b/y ion PEAKS in spectra.
By explicitly supervising ion peak classification token classification task, the model is guided to learn the critical features distinguishing b/y ions from other ions. In complex spectral scenarios, this supervision signal implicitly constrains and regularizes the peptide sequence prediction process, improving sequencing accuracy. This multi-task learning approach helps the model learn more discriminative feature representations, reduces the risk overfitting, and enhances generalization performance.
C. Reranking Framework Comparison
Objective
Col-Attn
Bacillus
C. bacteria
Honeybee
Humun
M.nazei
Mouse
Rice bean
Tomato
Yeast
Average
Point
0.869
0.741
0.803
0.796
0.835
0.824
0.841
0.811
0.819
Pair
0.87
0,741
0.803
0.799
0.836
0.826
0.854
0.841
0.811
0.82
List
0.865
0.737
0.799
0.786
0.829
0.825
0.832
0.844
0.817
0.815
Amino
PMD+RMD
X
0.871
0.745
0.809
0.801
0.839
0.828
0.854
0.844
0.812
0.822
Acid
Precision
Point
0,871
0.74
0.806
0.8
0.837
0.825
0.852
0.841
0.812
0.82
Pair
0.871
0.742
0.804
0.798
0.837
0.825
0.853
0.842
0.812
0.82
List
0.866
0.738
0.797
0.780
0.832
0.826
0.846
0.845
0.821
0.817
PMD+RMM
0.874
0.746
0,802
0.828
0.859
0.844
0.816
0.824
Point
X
0.727
0.528
0.614
0.628
0.660
0.575
0.721
0.680
0.690
0.647
Pair
X
0.728
0,529
0.614
0,631
0.661
0,573
0.722
0.681
0.692
0.648
List
0.725
0.540
0.622
0.613
0.663
0.590
0.694
0.670
0.705
0.646
Peptide
PMD+RMD
0.732
0.535
0.623
0.638
0.664
0.579
0.727
0.685
0.695
0.653
Recall
Point
0.731
0.529
0.619
0.634
0.663
0.577
0.722
0.681
0.693
Pair
0.729
0,531
0.616
0.634
0.662
0.575
0.721
0.684
0.696
List
0.728
0.536
0.613
0.605
0.663
0.588
0.710
0.694
0.706
0.649
PMD+RMD
0.738
0.539
0.63
0.642
0.672
0.583
0.733
0.691
0.703
0.660
Table 5. Performance comparison on 9-species-V V1 test set when the reranking framework varies. The symbol "f" indicates that the model the final RankNovo mentioned the main text.
Our RankNovo reranking framework features using the accurate peptide mass deviation metric PMD and RMD as similarity labels. Additionally, we use axial attention (particularly column-wise attention compared to ordinary language models) to boost peptide feature mixing. Here we compare this framework to some established reranking settings in order to prove that RankNovo captures the key modality feature of peptide and mass spectrums and is superior methodology than those used in NLP tasks the peptide sequencing task.
Our comparison involves two aspects: the reranking loss level and the backbone model level. We mainly compare RankNovo with the classic RankT5 (Zhuang et al., 2023) framework. RankT5 summarized the three common styles of reranking losses: point-wise, piir-wise, and list-wise loss. Suppose a given query qi has potentially relevant candidate documents da, d,2,- N n to rerank, a reranking framework uses backbone language model M extract the latent ži representing the relationship between qi and day. Then, žiy is projected (in RankT5, the 'projection' is accomplished by learning a new word) to predict the similarity score क... The process can be summarized as:
Úij = Projection (gi, d4y))
(14)
15

---

Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencing
For the peptide sequencing task, the relevance label Vij as a binary classification label (since our metric PMD and RMD not adopted). Then the point-wise loss function for each sample &i equals summation of binary cross entropy (BCE) losses between each query-document pair.
Point (Vi, 91) =
Σ Jluy=1
log(o
Σ Jl
(1))) = 1+ 1
(15)
Pairwise reranking loss focuses on enlarging the predicted similarity deviation between relevant query-document pairs and the irrelevant ones, which can be represented as:
C (yis U1)
Σ
bu>>,y 1og(1 (1 + c -Bs)
(16)
List-wise loss views reranking as a N-class classification. And the loss function can be represented as:
g
CLia (1is U1)
Σ viylog(
(17)
Ey euay
For the backbone model M, the encoder-decoder framework is inarguable. The only concern is whether the candidates dy should be able to 'see' each other. Some pair-wise or list-wise reranking work uses paired candidates' input and a post-ranking procedure. Here, we use column-wise attention modules to enable list-level perception fields because the existing methods for enabling communications between candidates are too diverse, making exhaustive comparison unrealistic.
Detailed results can be found in Table 5. Whether using column-wise attention not, the best model among point-wise, pair-wise, and list-wise frameworks falls behind at least 0.1 than RankNovo in terms of peptide recall, which achieves an average of 0,660 peptide recall across the nine species. Therefore, RankNovo is more suitable for the sequencing task than common NLP reranking frameworks. It's worth noticing that amino acid recall and peptide precision do not necessarily follow the same trend, especially when the peptide recalls between two models are close, because different models may solve tasks of varying lengths. However, the peptide sequencing task, the prime concern is whether spectrum can be identified. Therefore, our analysis focuses on peptide recall, as in Appendix D.
D. Ablation Study
D.I. Full benchmark result
In our implementation, RankNovo utilizes six base models: Casanovo-V2 V2, ContraNovo, ByNovo, R-Casanovo, R- ContraNovo, and R- R-ByNovo. To investigate the relative superiority of these models, we provide the detailed benchmark performance on 9-species-V1 (Table 6) and 9-species- (Table 7). These results are used as a criterion for model selection in the following sections.
D.2. Base Model Contribution Ablation
In this section, we would like to examine the necessity of each base model to achieve optimal performance, both during training and inference.
Abalation analysis on selected model subsets The six base models of RankNovo have dozens of subsets. Therefore, it's impossible to study every combination. Here, in order to study the influence of the number of base models and each model, we select five subsets. As indicated in Table 6, the performance of these six models on 9-species-V/ from poor to strong is: Casanovo-Ve V2, R-Casanovo, R-By R-Byoovo, R-ContraNovo, ContraNovo, and ByNovo. The five subsets are formed by sequentially removing the strongest model until reaching a minimum model number of 2. The details of these five combinations are listed in Table
16

---

Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencing
Methods
Bacillus
C. bucteria Honeybee Human
M.mazei
Mouse
Rice bean
Tomato
Yeast
Average
Amino Acid Precision
PEAKS
0.719
0.586
0.633
0.639
0.673
0.600
0,644
0.728
0.748
0.663
Baselines
DeepNovo
0.742
0.602
0.630
0.610
0.694
0.623
0.679
0.731
0.750
0.673
PointNovo
0.768
0.589
0.644
0.606
0.712
0.730
0.733
0.779
0.687
Casanovo
0.749
0.603
0.629
0.586
0.679
0.689
0.668
0.721
0.684
0.667
Casanovo V2
0.806
0.685
0.727
0.69
0.774
0.768
0.769
0.799
0.762
0.753
Base
ContraNovo'
0.828
0.706
0.761
0.771
0.798
0.799
0.804
0.808
0.782
0.784
Models
ByNovo"
0.858
0.723
0.791
0.767
0.823
0.803
0.836
0.828
0.804
0.804
R-Casanovo"
0,804
0.699
0.728
0.719
0.769
0.776
0.782
0.795
0.762
0.759
R- R-ContraNovo'
0.839
0.716
0.775
0.782
0.806
0.811
0.816
0.822
0.798
0.796
R-ByNovo'
0.855
0.724
0.794
0.762
0.821
0.81
0.835
0.831
0.762
0.799
Ours
RankNovo
0.874
0,746
0.81
0.802
0.84
0.828
0.859
0.844
0.816
0.824
Peptide Recall
PEAKS
0.387
0.203
0.287
0.277
0.356
0.197
0.362
0.403
0.428
0.322
Baselines
DeepNovo
0.449
0.253
0.330
0.293
0.422
0.286
0.436
0.454
0.462
0.376
PoiniNovo
0.518
0.298
0.396
0.351
0.478
0.511
0.513
0.534
0,439
Casanovo
0,537
0,330
0.406
0.341
0.478
0.426
0.506
0.521
0.490
0.448
Casanovo V2
0.646
0.46
0.527
0.492
0.592
0.493
0.628
0.637
0.629
0.567
Base
ContraNovo"
0.684
0.487
0.576
0.624
0.628
0.563
0.676
0.655
0.669
0.618
Models
ByNovo"
0.708
0,499
0.597
0.584
0.639
0.545
0.696
0.667
0.676
0.623
R-Casunovo"
0.628
0,467
0.515
0,511
0.57
0.505
0.611
0,611
0.601
0.558
R- R-CoonraNgrvo"
0.682
0,499
0.583
0.606
0.621
0.566
0.673
0.654
0.664
0.616
R-ByNovo"
0.703
0.493
0.59
0.554
0.637
0.543
0.685
0.659
0.629
0.610
Ours
RankNovo
0.738
0.539
0.63
0.642
0.672
0.583
0.733
0.691
0.703
0.660
Table Evaluation of RankNovo in comparison to baseline and base methods on the 9-species-V Vi test set. Bolded entries indicate the
best-performing models. The symbol "f" indicates that the model serves both baseline and base model. "x" signifies that the base models were developed and trained by us.
Methods
Bacillus
C. bacteria
Honeybee
Human
M.mazei
Mouse
Rice bean
Tomato
Yeast
Average
Casanovo V2t
0.888
0.791
0.823
0.872
0.877
0.813
0.891
0.891
0.915
0.862
Amino
ContraNovo
0,901
0,807
0,848
0.920
0.896
0.839
0.913
0.898
0.919
0.882
Acid
ByNovo"
0.92
0,823
0,876
0.917
0,914
0.841
0.932
0.912
0.934
0.897
Precision
R. "-Casmovv"
0.876
0.804
0.814
0.891
0.867
0.821
0.881
0.891
0.898
0.860
B.ContraNovo"
0.909
0.815
0.865
0.923
0.901
0.849
0.919
0.907
0.925
0.890
R-ByNovo'
0.919
0.822
0.879
0.912
0,912
0.843
0.932
0.913
0.936
0.897
Novo
0.926
0.838
0.885
0.929
0.920
0.860
0.938
0.918
0.938
0.906
Casanovo V2t
0.793
0.558
0.669
0.712
0.754
0.555
0.772
0.783
0.837
0.714
Peptide
ContraNovo
0.815
0.575
0.711
0.820
0.780
0.616
0.799
0.794
0.854
0.752
Recall
ByNovo
0.833
0.582
0.731
0.789
0.799
0.596
0.814
0.807
0.871
0.758
R Casanovo
0.759
0.558
0.643
0.732
0.723
0.558
0.721
0.768
0.799
0.696
R-ContraNova
0.821
0.581
0.719
0.815
0.779
0.620
0.804
0.798
0.861
0.755
R-ByNovor
0.781
0.589
0.815
0.803
0.873
0.756
RankNovo
0.851
0.620
0.752
0.820
0.813
0.629
0.836
0.822
0.885
0.781
Table Evaluation of RankNovo in comparison haseline and base methods on the 9-species- V2 Bolded entries indicate the
hest-performing models. The symbol "f" indicates that the model serves as both a baseline and a base model. "x" signifies that the base models were developed trained by us.
Train
N. Infer
Bacillus
bacteria
Honeybee
Human M.mazei Mouse
Rice bean
Tomato
Yeast
Average
2
2
0.832
0.721
0.752
0.735
0.795
0.785
0.804
0.806
0.796
0.781
Amino Acid
3
3
0.864
0.749
0.8
0.774
0.828
0.811
0.846
0.829
0.796
0.811
4
4
0.865
0.751
0,802
0.794
0.834
0.819
0.847
0.831
0.817
0.818
Precision
5
5
0.87
0.756
0.804
0,802
0.835
0.825
0.852
0.832
0.822
0.822
6'
6
0.874
0.746
0.81
0.802
0.84
0.828
0.859
0.844
0.816
0.824
2
0.667
0.482
0.548
0.533
0.598
0.519
0.649
0.641
0.636
0.586
Peptide
3
3
0.711
0,516
0.599
0.575
0.645
0.55
0,701
0.672
0.641
0.623
4
4
0.72
0,525
0.613
0.613
0.656
0.568
0.711
0.679
0.673
0.64
Recall
5
0.73
0.534
0.618
0.633
0.665
0,583
0.724
0.683
0.688
0.651
6
6
0.738
0.539
0.63
0.642
0.672
0.583
0.733
0.691
0.703
0.660
Table 9, Peptide recall evaluation of RankNovo on 9-species V1 test set when the training base model set and the inference base model set are the same and vary. The symbol "4" indicates that the model the final RankNovo mentioned the main text.
Firstly, we consider the impact of some base models being completely disregarded during training and inference. From
17

---

Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencing
Model Num.
Base Model Set
2
Casanovo- V2, Casannnvo
3
Casanovo- V2, R- R-Casanovo, R- R-ByNovo
4
Casanovo- V2, R-Casanovo, R- R-ByNovo, R-ContraNovo
5
Casanovo- V2, R-Casanovo, R- R-ByNovo, R-ContraNovo, ContraNovo
6
Casanovo- V2, R- R-Casanovo, R R-Byyovo, R- R-ContraNovo, ContraNovo, ByNovo
Table Description different combinations of base models
Table 9, we can see that can more base models that are used, the peptide recall on the 9-species- V1 dataset ascends, from the lowest 0.586 of two models to the highest 0.660 six models. On the other hand, even when all six models are used during inference, the absence of models during training affects the final performance. As in Table 10, the combination of two base models achieves the lowest peptide recall of 0.649, 1.6% worse than the combination of all models.
Train
N. Infer
Bacillus
C. bacteria
Honeybee
Human
M.mazei
Mouse
Rice bean
Tomato
Yeast
Average
2
6
0,870
0.754
0.806
0.803
0.837
0,821
0.856
0.833
0.826
0.823
Amino Acid
3
6
0,872
0.757
0,805
0.8
0.835
0,822
0.852
0.834
0.826
0.823
4
6
0.872
0.756
0,807
0,805
0.839
0.822
0.855
0.835
0.827
0.824
Precision
5
6
0.875
0.761
0.81
0,806
0.839
0.826
0.858
0.837
0.828
0.827
6'
0.874 6
0.746
0.81
0,802
0.84
0.828
0.859
0.844
0.816
0.824
2
6
0.727
0.525
0.613
0.627
0.663
0.577
0.723
0.684
0.686
0.647
Peptide
3
6
0.731
0.533
0.617
0.628
0.661
0.579
0.719
0.685
0.693
0.649
6
0.733
0.533
0.625
0.638
0.668
0.580
0.729
0.688
0.700
0.655
Recall
5
6
0.734
0.537
0.624
0.638
0.67
0.584
0.729
0.689
0.696
0.657
6'
6
0.738
0.539
0.63
0.642
0.672
0.583
0.733
0.691
0.703
0.660
Table 10. Peptide recall evaluation of RankNovo 9-spocies-V V1 test set when the training base model set varies and the inference basc model set is fixed. The symbol "I" indicates that the model is the final RankNovo mentioned in the main text.
Combining these two experiments, two important results are shown. Firstly, the impact of not using all models exists, both at training and at the inference stage. This means the integration of more diversity during training enriches the knowledge of RankNovo. Secondly, we can see that the number of models during inference is more important than that during training. Both training with two models, the peptide recall rises by 10.7% when the number of inference models increases from 2 to 6. This shows RankNovo's zero-shot generalization ability, which is more delicately shown in Section E.
Additional results on reversed model subsets The model subsets combination in Table 8 is further used in Sec 4.4. To verify that sequentially removing the best base model is not the key factor of our findings, we generate reverse combination of model subsets by sequentially removing the worst model. The generated model subsets are listed in Table 11.
The same experiments as above are conducted on this combination. As shown in Table 12 and Table 13. The overall trend remains consistent under this alternative setting. Knowledge acquired by certain base models can still be adapted to other base models in zero-shot manner during inference. Increasing the number of base models during training continues to yield better overall performance. However, expected, the performance trends are less pronounced compared to our original configuration, where the strongest models were sequentially removed. This additional analysis strengthens our understanding of the interplay between base model selection and RankNovo's overall performance.
Since the original settings yield more pronounced performance differences between configurations, it is adapted for further experiments and analysis.
18

---

Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencing
Model Num.
Base Model Set
2
ByNovo, ContraNovo
3
ByNovo, ContraNovo, R-ContraNovo
4
ByNovo, ContraNovo, R-ContraNovo, R-ByNovo
5
ByNovo, ContraNovo, R-ContraNovo, R- R-ByNovo, R-Casanovo
6
ByNovo. ContraNovo, R-ContraNovo, R R-ByNovo, R- R-Casanovo, Casanovo-Vy
Table Description different combinations base models. The combinations are generated by sequentially removing weakest model.
Train
Infer
Bacillus
bucteria
Honeybee
Human
M.mazei
Mouse
Rice bean
Tomato
Yeast
Average
2
2
0.855
0.721
0.787
0.781
0.819
0,809
0.832
0.825
0.799
0.803
Amino Acid
3
3
0.864
0.733
0.799
0.798
0.832
0.824
0.844
0.836
0.812
0.816
4
4
0.868
0.737
0.806
0.8
0.836
0.825
0.85
0.84
0.812
0.819
Precision
5
5
0.871
0.743
0.808
0.804
0.838
0.829
0.855
0.842
0.818
0.823
6
6
0.874
0.746
0.81
0,802
0.84
0.828
0.859
0.844
0.816
0.824
2
2
0.707
0.501
0.599
0.617
0.644
0.563
0.703
0.667
0.684
0.632
3
3
0.719
0.518
0.613
0.638
0.653
0.577
0.709
0.677
0.694
0.644
Peptide Recall
4
4
0.727
0,522
0.619
0.634
0.658
0.576
0.715
0.681
0.696
0.648
5
0.732
0.530
0.624
0.638
0.663
0.582
0.727
0.702
0.654
6'
6
0.738
0.539
0.63
0.642
0.672
0.583
0.733
0.691
0.703
0.660
Table 12. Peptide recall evaluation of RankNovo 9-species-V V1 test set when the training base model set and the inference base model set are same vary. The symbol "t" indicates that model is the final RankNovo mentioned the main text. The model subsets here are created by sequentially removing the weakest model, as introduced in Table 11.
Train
N. Infer
Bacillus
bacteria Honeybee
Human
M.mazei
Mouse
Rice bean
Tomato
Yeast
Average
2
6
0.871
0.745
0,808
0.801
0.839
0.828
0.856
0.843
0.814
0.823
Amino Acid
3
6
0.871
0.745
0.809
0.802
0.839
0.829
0.856
0.844
0.815
0.823
6
0.873
0.745
0.810
0.802
0.841
0.829
0.857
0.845
0.817
0.824
Precision
5
6
0.872
0.743
0.81
0.802
0.839
0,829
0.858
0.843
0.817
0.824
6'
6
0.874
0.746
0.81
0.802
0.84
0.828
0.859
0.844
0.816
0.824
6
0.727
0.526
0.615
0.627
0.665
0.581
0.726
0.685
0.687
0.649
3
6
0.732
0.529
0.623
0.632
0.665
0.583
0.727
0.685
0.698
0.653
Peptide
Recall
4
6
0.735
0.535
0.625
0.639
0.669
0.583
0.728
0.688
0.700
0.656
5
6
0.738
0.533
0.628
0.639
0.673
0.584
0.735
0.690
0.701
0.658
6
6
0.738
0.539
0.63
0.642
0.672
0.583
0.733
0.691
0.703
0.660
Table 13. Peptide recall evaluation of RankNovo on 9-species-VI V1 test set when the training base model set varies and the inference base model set is fixed. The symbol "f" indicates that the model the final RankNovo mentioned in the main text. The model subsets here are created by sequentially removing the weakest model, as introduced in Table 11.
D.3. Training Objective Ablation
Objective
Bacillus
C. bacteria
Honeybee
Human M.mazei
Mouse
Rice bean
Tomato
Yeast
Average
Amino
RMD
0.869
0.755
0.807
0.802
0.838
0.822
0.853
0.834
0.827
0.821
Acid
PMD
0.871
0.742
0.810
0.806
0.836
0,823
0.856
0.835
0.821
0.822
Precision
PMD RMD!
0.874
0.746
0.81
0.802
0.84
0.828
0.859
0.844
0.816
0.824
RMD
0.731
0.529
0.618
0.632
0.664
0.576
0.723
0.684
0.691
0.65
Peptide
PMD
0.731
0.534
0.623
0.637
0.664
0.577
0.726
0.685
0.694
0.652
Recall
PMD RMDÍ
0.738
0.539
0.63
0.642
0.672
0.583
0.733
0.691
0.703
0.660
Table 14, Evaluation of performance on 9-species- V1 test set when training under different objectives. The symbol "4" indicates that the model the final RankNovo mentioned the main text.
19

---

Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencing
In this work, we introduces novel metries, PMD and RMD, the learning objective of reranking models. Here conduct the ablation study of the effect of the combined use of these two metries. As shown in Table 14, using RMD alone achieves the lowest peptide recall of 0.650, while only using PMD alone is better, with a peptide recall of 0.657. The best peptide recall of 0.660 is achieved when both PMD and RMD are used.
D.4. Model Architecture Ablation
Col-Attn Bacillus
C. bacteria Honeybee
Human
M.mazei
Mouse
Rice bean
Tomato
Yeast
Average
AA Precision
0.871
0.745
0.809
0.801
0.839
0.828
0.854
0.844
0.812
0.822
Vt
0.874
0.746
0.81
0.802
0.84
0.828
0.859
0.844
0.816
0.824
Peptide
x 0.732
0.535
0.623
0.638
0.664
0.579
0.727
0.685
0.695
0.653
Recall
0.738
0.539
0.63
0.642
0.672
0.583
0.733
0.691
0.703
0.660
Table 15, Performance comparison RankNovo on 9-species- V1 test set between using column- wise attention the peptide feature mixer not. The symbol "f" indicates that the model is the final RankNovo mentioned in the main text.
The effect of whether using column-wise attention has already been mentioned Section In this section, we emphasize its effect when the training objective is chosen be the combination of PMD and RMD. From Table 15 when using column-wise attention modules, the average peptide recall across the nine species rises from 0.653 to 0.660. This shows column-wise attention's contribution to the optimal performance of RankNovo.
that
E. Additional Results
E.1. Analysis Zero-shot Performance
N. Train
N. Infer
Bacillus
C. bacteria
Honeybee
Hunan
M.mazei
Mouse
Rice bean
Tomato
Yeast
Average
2
0.721
0.752
0.735
0.795
0.804
0.806
0.796
0.781
Amino Acid
2
3
0.86
0.732
0.799
0.773
0.826
0.813
0.845
0.835
0.778
0.807
2
4
0.861
0.733
0.798
0.786
0.828
0.82
0.845
0.837
0.804
0.812
Precision
2
5
0.864
0.737
0.801
0.797
0.832
0.825
0.849
0.839
0.807
0.817
6
0.757
0.809
0.806
0.840
0.859
0.836
0.829
0.826
2
2
0.667
0).822
0.548
0.533
0.598
0.519
0.649
0.641
0.636
0.586
3
0.707
0.51
0.596
0.579
0.642
0.551
0.705
0.671
0.64
0.622
Peptide Recall
2
4
0.716
0.518
0.605
0.604
0.653
0.567
0,709
0.677
0.662
0.635
2
5
0.722
0.525
0.611
0.628
0.661
0.579
0.72
0.681
0.68
0.645
6
0.729
0.527
0.615
0.629
0.665
0.579
0.725
0.686
0.688
0.649
Table Zero-shot performance fixed training base model set two models on unseen models. The numbers are calculated on the 9 9-species-V1 dataset.
We demonstrate the zero-shot capability of RankNovo by training it exclusively predictions from the two lowest- performing base models and progressively incorporating predictions from unseen models into the candidate sets for each spectrum during inference. As shown in Table 16, as the number of inference models increases, the average peptide recall improves, rising from 0.586 with models to 0.649 with 6 models.
20

---

Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencing
E.2. Analysis of Amino Acid Identification with Similar Masses
The experimental results across the nine species, as illustrated in Figure 5, exhibit consistent improvement recall for key amino acids (M(O), Q. F, K) when leveraging RankNovo over the baseline methods, Casanovo V2 and ContraNovo. RankNovo consistently outperforms the baselines across all species, particularly in M(O) and F. The recall improvements most pronounced in species like yeast, ricebean, and honeybee, where RankNovo demonstrates significant performance gains. These results emphasize the strong generalization capabilities of RankNovo across diverse species and its effectiveness in addressing the ambiguities introduced by amino acids with similar masses. The consistent superiority of RankNovo underscores its potential to advance peptide sequencing, especially within complex biological datasets.
bacillus
human
mouse
0.9
0.9
0.9
0.8
0.7
0.7
0.7
Recal
Rocall
0.8
0.6
0.6
0.5
0.5
0.4
Casanovo
Ours
0.4
Casanova
Ours
0.4
Casanovo
Ours
ContraNovo
ContraNovo
ContraNovo
M(O)
K
M(O)
K
M[O]
dambacteria
yeast
mmazel
0.9
0.9
0.9
0.7
0.7
0.7
Recall
Rocall
0.6
0.6
0.6
0.5
0.5
0.4
Casianovo V2
Ours
0.4
Casanova V2
Ours
0.4
Casanovo
Ours
ContraNovo
ContraNovo
ContraNovo
M(O)
Q
K
M(O)
K
M(O)
K
ricebean
tomato
honeybee
0.9
0.9
0.9
0.8
0.8
0.8
0.7
0.7
Recall
Recall
Rocall
0.6
0.6
0.6
0.4
Casenovo V2
Ours
0.4
Casanovo V2
Ours
0.4
Casanovo V2
Ours
ContraNovo
ContraNova
ContraNova
M(O)
K
M(O)
F
M(O)
Q
K
Figure 5. The performance comparison of amino acids with similar masses. The numbers are calculated on the 9-species- V1 datasct.
21

---

Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencing
E.3. Analysis of Peptide Length
The results in Figure 6 demonstrate that our model consistently surpasses the baseline methods, Casanovo V2 and ContraNovo, across wide variety of species. Specifically, for shorter peptides (lengths 7 to 17), our model achieves significantly higher recall across all species, underscoring its enhanced capacity to capture key sequence patterns in simpler peptide structures. As peptide length increases, performance across all models declines progressively, indicating that longer peptides introduce additional structural complexity that impairs recognition accuracy. Nonetheless, our model maintains a competitive advantage, consistently outperforming the baselines for most species. However, the performance gap diminishes as peptide length increases, likely due to the heightened challenges associated with recognizing longer sequences. These results highlight the effectiveness of our model in processing peptides of varying lengths, as well as its strong generalization capability across diverse species.
bacillus
human
mouse
1.0
1.0
1.0
0.8
0.8
0.8
Peplide Paplide Recall
Poptide Recall
Peptide Recall
0.8
0.6
0.6
0.4
Casanovo V2
Casanovo V2
Casanovo V2 ContraNovo
ContraNovo
ContraNovo Ours
0.2
Ours
0.2
0.2
Ours
7
12
17
22
27
12
17
22
27
12
17
22
27
Poptide Length
Poptide Length
Peptide Length mmazei
clambacteria
yeast
1.0
1.0
1.0
0.8
0.8
0.8
Peplide Racall
Poptide Recall
Peptide Recall
0.4
0.4
0.4
Casanovo V2
Casanovo
Casanovo ContraNovo Ours
ContraNovo
ContraNovo
0.2
Ours
0.2
Ours
0.2
12
22
27
Peptide Length ricebean
Poptide Length tomato
Poptide Length honeybee
1.0
1.0
0.8
Popiida Recall
Peptide Recall
Peptide Recail
0.6
0.6
0.6
0.4
0.4
0.4
Casanovo V2
Casanovo V2
Casanovo V2 ContraNovo
ContraNovo Ours
ContraNovo
0.2
0.2
Ours
0.2
Ours
27
Peptide Longth
Peptide Length
Peptide Longth
Figure 6. Influence of peptide length on 9-species- V1 datasct.
22

---

Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencing
E.4. Contribution of Each Base Model
By analyzing the contributions of individual base models across nine species, we uncover distinct patterns of efficacy, as depicted in Figure 7. Each base model exhibits varying degrees of influence on RankNovo's peptide selection, underscoring their complementary strengths. Notably, R- R-ByNovo consistently demonstrates the highest contribution in most species, reaching 41.7% in yeast, while Casanovo- V2 contributes less significantly, particularly in species like tomato and mouse. This variation suggests that different models capture species-specific features with varying effectiveness. The consistent, albeit variable, contributions each base model highlight the critical importance model diversity; removing any single model would likely degrade performance for certain species, These findings illustrate the robustness of the ensemble approach, where integrating multiple models compensates for the limitations of individual ones, enabling RankNovo generalize effectively across broad range of species and peptide structures.
bacilus
humen
mouse
22.3%
35.0%
24.2%
14.1%
8.9%
5.6%
6.15
1.6%
4.0%
7.1%
0%
15.2%
17.8%
90.3%
31.9%
30.3%
yeoo!
16: 1%
15.7%
18.1%
29.2%
20.4%
11.7%
0.0%
9.3%
15.3%
5.4%
25 9%
28.5%
ricebean
tometo
honeybee
19.15
12.0%
13.5%
27.2%
9%
4.5%
7.5%
12.0%
10.5%
22.2%
19.6%
24.8%
27.1%
V2
ByNovo R-Casanovo
ContraNovo
R-ByNovo
Figure Unique-correctly selected percentage base models. The numbers are calculated on the 9-species-V dataset.
23

---

Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencing
E.5. More information about training time and inference cost
Model
Parameters (M) Training Time (Day)
Infer. Cost (s/spectra)
Infer. Speed (spectras/s)
Casa. & R-Casa.
47.3
3
0.127
7.87
Contra. & R-Contra.
68.6
4
0.173
5.78
By. & R-By.
49.7
0.169
5.92
RankNovo
50.5
Table 17. Summary of model parameters, training time, and inference speed of six base models and RankNovo.
N. Infer
Candidates Collection (s/spectra)
Reranking (s/spectra) Total Cost (s/spectra)
Infer. Speed (spectra/s)
0,254
0.004
0.258
3.88
3
0,423
0,006
0.429
2.33
4
0,596
0,008
0,604
1.66
0.769
0.010
0.779
1.28
6
0.938
0.011
0.949
1.05
Table 18. RankNovo's inference speed when using different numbers of base models. The combination base models refers Table
As shown in Table 17, RankNovo comprises 50.5M parameters and requires 4 days of training utilizing four 40GB A100 GPUs, which is comparable the base models. Since RankNovo a reranking framework, its inference speed is anticipated be slower than single-model approaches. However, the reranking process itself is not the primary time constraint. The majority of RankNovo's inference time is consumed in gathering peptide candidates from base models (Table 17), these require sequential autoregression and beam search decoding, while RankNovo's inference involves only single attention forward pass.
As we scale from 2 to 6 base models in RankNovo, the inference time increases approximately linearly, with inference speed decreasing proportionally (Table 18). This increased computational cost an inherent characteristic of reranking frameworks and represents an unavoidable trade-off compared single-model approaches. However, RankNovo effectively leverages this additional inference time to achieve superior performance levels unattainable by single models. This inference tene-performance trade-off can be flexibly adjusted by modifying the number of candidates.
In the context of denovo peptide sequencing, RankNovo's significance lies in introducing novel approach that allows researchers to optionally scale up inference time in exchange for enhanced performance. This represents the first such option the field.
E.6. Additional Results on PTM identifications
In our current 9-species-V VI benchmark, classes of post-translation modifications (PTMs) are included: Oxidation (M), Deamidation (N), and Deamidation (Q). The promising results of these modifications demonstrated that RankNovo can effectively enhance performance on PTM-containing spectra when such modifications are incorporated during training.
To better verify RankNovo's ability on more diverse PTM types, we conducted additional experiments on more diverse set of biologically significant PTMs from the dataset compiled by Zolg et al (Zolg et al., 2018). Specifically, selected three functionally important modifications: Acetylation (K), Dimethylation (K), and Phosphorylation (Y). Each PTM included 62.5K spectra split 8:1:1 for trainingyaaiidationeessnngg
Given that these new PTMs were not the original vocabulary of our models, we performed necessary fine-tuning procedures. We combined the training and validation datasets across all three PTMs, reinitialized the embedding and final linear layers to accommodate the expanded vocabulary, and fine-tuned both the six base models and RankNovo accordingly.
24

---

Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencing
PTM
Casanova
ContraNovo
ByNovo R-Casanova
R-ContraNovo
R-ByNovv
RankNovo
Acetylation (K)
0.819
0.820
0.830
0.833
0.806
0.832
0.889
Dimethylation (K)
0.455
0.459
0.458
0.458
0.401
0.457
0.487
Phosphorylation (Y)
0.476
0.473
0.520
0.491
0.519
0.522
0.589
Table 19. Performance comparison of different models various post-translational modifications (PTMs).
Our experimental results demonstrate RankNovo's consistent superiority across multiple post-translational modifications (PTMs). As shown in Table 19, RankNovo achieved significant improvements over the best base models: 5.6% for Acetylation (K) (0.889 0.833), 2.8% for Dimethylation (K) (0.487 0.457), and 6.7% for Phosphorylation (Y) (0.589 vs 0.522). These compelling performance gains validate that our deep learning reranking framework maintains its effectiveness across diverse spectrum of PTMs, highlighting the robustness and broader applicability of our approach for advanced proteomics research.
25


Reviews:
[{"review_id": "1ayAjNqDSf", "reviewer": "Reviewer_Rt9T", "summary": "This paper introduces RankNovo, an innovative deep reranking framework designed to enhance de novo peptide sequencing accuracy. RankNovo leverages the complementary strengths of multiple sequencing models to overcome the limitations of individual approaches.\n\nRankNovo represents the first deep learning-based reranking framework for de novo peptide sequencing, introducing several innovations: it models candidates as multiple sequence alignments with axial attention to extract cross-candidate features while reducing computational complexity; develops mass-focused metrics (PMD and RMD) that provide precise supervision by quantifying differences at both sequence and residue levels; achieves state-of-the-art performance on benchmark datasets, surpassing its strongest base model by 6.1% and the previous best method by 4.3% in peptide recall; demonstrates robust zero-shot generalization to unseen models; and significantly improves discrimination between amino acids with similar masses, addressing a key challenge in mass spectrometry-based peptide identification.\n\nThe dual-track architecture combines spectrum feature extraction via a Transformer encoder with peptide candidate processing through axial attention, joined by a cross-attention mechanism. By jointly optimizing PMD and RMD objectives, RankNovo offers a flexible trade-off between inference time and performance, advancing the frontier of accurate de novo peptide sequencing.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed comments. We address your concerns as follows:\n\n> The paper lacks validation on more challenging datasets with diverse post-translational modifications (PTMs).\n\nWe sincerely appreciate your insightful observation regarding the need for validating RankNovo on datasets with more diverse post-translational modifications. You've highlighted a crucial aspect of peptide de novo sequencing that directly impacts the practical utility of our approach in comprehensive proteomics studies.\n\n``` In our current Nine-Species-V1 benchmark, we included three classes of PTMs: Oxidation-M, Deamidation-N, and Deamidation-Q.``` The promising results of these modifications demonstrated that RankNovo can effectively enhance performance on PTM-containing spectra when such modifications are incorporated during training.\n\nTo address your valuable suggestion, we conducted additional experiments on a more diverse set of biologically significant PTMs from the dataset compiled by Zolg et al. [1]. ``` Specifically, we selected three functionally important modifications: Acetylation (K), Dimethylation (K), and Phosphorylation (Y).``` Each PTM included ~62.5K spectra split 8:1:1 for training/validation/testing.\n\nGiven that these new PTMs were not in the original vocabulary of our models, we performed necessary fine-tuning procedures. We combined the training and validation datasets across all three PTMs, reinitialized the embedding and final linear layers to accommodate the expanded vocabulary, and fine-tuned both the six base models and RankNovo accordingly.\n\n**Our results consistently demonstrated RankNovo's superiority:**\n\n- Acetylation (K): 5.6% improvement over the best base model (0.889 vs 0.833)\n- Dimethylation (K): 2.8% improvement (0.487 vs 0.457)\n- Phosphorylation (Y): 6.7% improvement (0.589 vs 0.522)\n\n| PTM | Casanovo | ContraNovo | ByNovo | R-Casanovo | R-ContraNovo | R-ByNovo | RankNovo |\n|---|---|---|---|---|---|---|---|\n| Acetylation (K) | 0.819 | 0.820 | 0.830 | 0.833 | 0.806 | 0.832 | 0.889 |\n| Dimethylation (K) | 0.455 | 0.459 | 0.458 | 0.458 | 0.401 | 0.457 | 0.487 |\n| Phosphorylation (Y) | 0.476 | 0.473 | 0.520 | 0.491 | 0.519 | 0.522 | 0.589 |\n\nThese compelling results further validate that our deep learning reranking framework maintains its potential across a more diverse spectrum of PTMs, underscoring the robustness and broader applicability of our approach for advanced proteomics research.\n\n[1]ProteomeTools: Systematic characterization of 21 post-translational protein modifications by liquid chromatography tandem mass spectrometry (LC-MS/MS) using synthetic peptides\n\n\n> Testing on data from a broader range of mass spectrometry instruments would better demonstrate cross-platform generalization.\n\nThank you for highlighting this limitation. While our original benchmarks (Nine-species-v1/v2) used only **Q Exactive instruments**, our training dataset (MassiveKB) **includes PSMs from various instruments, potentially enabling cross-platform generalization.**\n\nTo address your concern, we note that the **PTM datasets** mentioned in our response to Q1 were collected using **Orbitrap Fusion Lumos** instruments, different from our benchmark's Q Exactive instruments. RankNovo consistently outperformed all base models on these Orbitrap Fusion Lumos datasets across all three PTMs (Acetylation-K, Dimethylation-K, and Phosphorylation (Y)).\n\n```These results provide strong evidence of RankNovo's effectiveness beyond Q Exactive instruments, demonstrating its cross-platform generalization capabilities.``` We appreciate this suggestion which helped strengthen our evaluation.\n\n\n> The manuscript would benefit from addressing additional research directions in the literature review to better contextualize scientific contributions.\n\nThank you for this valuable suggestion. We decided to incorporate the recommended research directions into our final manuscript as follows:\n\n- **Introduction (Section 1):**\n\nAdded ensemble approaches for multiple search engines **(PepExplorer, iProphet)**\n- **Related Work - De Novo Peptide Sequencing (Section 2.1):**\n\nIncorporated PTM identification works **(Open-pFind, pNovo+)**\n- **Related Work - Candidate Reranking (Section 2.2):**\n\nExpanded with list-wise reranking algorithms **(ListNet, ListMLE)**\n- **Related Work - Axial Attention (Section 2.3):**\n\nEnhanced with efficient attention mechanisms **(Linformer, Performer)**\n\nThese additions better contextualize our contributions within existing literature and help readers understand how our work bridges deep learning techniques and proteomics applications. We appreciate your feedback which has improved the manuscript's depth and quality.", "claims_and_evidence": "The majority of RankNovo's claims are well-supported by evidence, with comprehensive experiments and analyses. The paper convincingly demonstrates state-of-the-art performance through extensive benchmarking on standard datasets, with Tables 1-2 showing clear improvements over previous methods; the effectiveness of the novel PMD and RMD metrics is verified through ablation studies in Table 3; the complementary contributions of base models are illustrated through detailed analysis in Fig. 3(B) and Fig. 7; and improved discrimination between similar-mass amino acids is supported by comprehensive data in Fig. 3(D) and Fig. 5.\n\nHowever, regarding zero-shot generalization, while Fig. 3(A) demonstrates RankNovo's ability to rerank predictions from unseen models, the paper lacks validation on more challenging datasets with diverse post-translational modifications (PTMs). Testing on specialized PTM-rich datasets, or synthetic phosphopeptide collections would provide stronger evidence of true generalization capability across the complex PTM landscape that characterizes real-world proteomics applications. This additional validation would strengthen the claim that RankNovo can generalize to difficult cases beyond the standard benchmarks.", "methods_and_evaluation_criteria": "RankNovo's methods and evaluation approach are aligned with the de novo peptide sequencing problem. The list-wise reranking approach appropriately handles multiple candidate sequences with subtle differences, while the MSA representation with axial attention efficiently captures both within-peptide patterns and cross-candidate information. The mass-centric metrics (PMD and RMD) are specifically tailored to the chemistry-driven nature of peptide sequencing, and the cross-attention mechanism effectively integrates spectrum information. The evaluation methodology is thorough, using established benchmark datasets (MassIVE-KB, 9-species-V1, 9-species-V2) with standard metrics like peptide recall and amino acid precision, while incorporating diverse base models and comprehensive ablation studies to assess different components' contributions.\n\n**Minor Limitations**\n\nLimited PTM Coverage: While the datasets include some post-translational modifications, more extensive testing on heavily modified peptides would strengthen the evaluation.\nSingle Instrument Type: Testing on data from a broader range of mass spectrometry instruments would better demonstrate cross-platform generalization.", "theoretical_claims": "The RankNovo paper primarily focuses on empirical contributions rather than making significant theoretical claims that require formal proofs. There are no mathematical theorems, lemmas, or propositions in the paper that would necessitate rigorous proof verification. The main algorithmic contributions—the list-wise reranking approach with axial attention and the definition of PMD and RMD metrics—are presented as algorithmic descriptions with pseudocode rather than theoretical results.\n\nThe paper does briefly discuss computational efficiency claims regarding the reduction from O(n²) to O(n) complexity with axial attention, but this is a well-established result in the literature. Similarly, while the authors provide theoretical motivation for their design choices, these are presented as design rationales rather than formal claims requiring proof. In summary, the paper's contributions are predominantly empirical, focusing on the practical design, implementation, and evaluation of a novel framework for peptide sequencing.", "experimental_designs_or_analyses": "Limited PTM Coverage: While the datasets include some post-translational modifications, more extensive testing on heavily modified peptides would strengthen the evaluation.", "relation_to_broader_scientific_literature": "RankNovo integrates concepts from both proteomics and machine learning domains, building upon established research trajectories. In proteomics, it extends the evolution from traditional database search methods (SEQUEST, Mascot) to deep learning approaches for de novo peptide sequencing, following Casanovo's transformer architecture and DeepNovo's neural network foundation. However, RankNovo distinctly reframes peptide sequencing as a reranking problem rather than a generation task, adapting a strategy that has proven effective in protein structure prediction and other computational biology applications.\n\nFrom a methodological perspective, RankNovo adapts several established machine learning techniques to the peptide sequencing domain. Its list-wise reranking approach parallels methods in information retrieval and NLP, while the axial attention mechanism borrows from efficient transformer variants in computer vision. The multiple sequence alignment representation repurposes traditional bioinformatics techniques, and the mass-focused metrics introduce domain-specific supervision signals. This combination addresses specific challenges in de novo peptide sequencing, particularly the discrimination between amino acids with similar masses. The paper's demonstration of zero-shot generalization connects to research on model-agnostic meta-learning, though the broader applicability of these techniques beyond peptide sequencing remains to be fully explored.", "essential_references_not_discussed": "The RankNovo paper, while generally thorough in its literature review, overlooks several important research directions that would better contextualize its contributions. In proteomics, the paper doesn't acknowledge ensemble approaches like \"PepExplorer\" (Leprevost et al., 2014) and \"iProphet\" (Shteynberg et al., 2011), which pioneered the combination of multiple search engines for peptide identification. Similarly, when discussing PTM analysis capabilities, the paper would benefit from referencing specialized frameworks like \"Open-pFind\" (Chi et al., 2018) and \"pNovo+\" (Chi et al., 2013) that specifically address the challenges of identifying post-translational modifications.\n\nFrom a methodological standpoint, the paper misses opportunities to connect with fundamental work on efficient attention mechanisms such as \"Linformer\" (Wang et al., 2020) and \"Performer\" (Choromanski et al., 2020), which pioneered approaches to reduce attention complexity from O(n²) to O(n). Additionally, when presenting its list-wise ranking approach, the paper fails to acknowledge seminal works in information retrieval like \"ListNet\" (Cao et al., 2007) and \"ListMLE\" (Xia et al., 2008) that established the mathematical foundations for list-wise ranking. These omissions don't invalidate RankNovo's contributions but limit readers' ability to understand how it builds upon established techniques across multiple disciplines.", "other_strengths_and_weaknesses": "refer to previous section", "other_comments_or_suggestions": "Refer to the previous section", "questions_for_authors": "Refer to previous section", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2H9DrXcxSi", "reviewer": "Reviewer_4F4K", "summary": "This paper presents RankNovo, the deep reranking framework that enhances de novo peptide sequencing by leveraging the complementary strengths of multiple sequencing models.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed comments. We address your concerns as follows:\n\n> It's better to provide benchmark performance of RankNovo on NovoBench.\n\nThank you for your suggestion. Following your recommendation, we expanded our evaluation to include all three **NovoBench** data sources: **Seven-Species (from DeepNovo), Nine-Species-V1, and HC-PT (from InstaNovo).** We construct Seven-Species and HC-PT as instructed in NovoBench.\n\n| Model | Nine-Species (yeast) | Seven-Species (yeast) | HC-PT |\n|-------|---------------------|----------------------|-------|\n| Casanovo | 0.48 | 0.12 | 0.21 |\n| Instanovo | 0.53 | - | 0.57 |\n| AdaNovo | 0.50 | 0.17 | 0.21 |\n| HelixNovo | 0.52 | 0.23 | 0.21 |\n| SearchNovo | 0.55 | 0.26 | 0.45 |\n| ByNovo (best base model) | 0.68 | 0.03 | 0.82 |\n| RankNovo | 0.70 | 0.04 | 0.89 |\n\nThe results on NovoBench show that ```RankNovo is consistently superior on Nine-Species and HC-PT datasets.``` On the other hand, zero-shot performance of models trained on MassiveKB (ByNovo and RankNovo) don't perform normally on Seven-Species. **Further analysis shows that the reason lies in the distribution misalignment between MassiveKB and Seven-Species.** The former is collected by high-resolution MS equipment, while the latter is composed of low-resolution data, which makes the phenomenon explainable.\n\n```We greatly appreciate your valuable suggestion and will incorporate these additional benchmark results and the discussion about data composition into our camera-ready manuscript.```\n\n> More baselines should be included for a more comprehensive comparison.\n\nThank you for this valuable suggestion. We have expanded our evaluation to include **InstaNovo, AdaNovo, π-PrimeNovo, and π-HelixNovo**. Results show **RankNovo outperforms all these baselines across all metrics on Nine-Species-V1**, a brief summary is provided below.\n\n| Model | Avg. Peptide Recall |\n|-------|---------------------|\n| Casanovo | 0.481 |\n| InstaNovo | 0.532 |\n| AdaNovo | 0.505 |\n| PrimeNovo | 0.638 |\n| HelixNovo | 0.517 |\n| RankNovo | 0.660 |\n\nAdditionally, as mentioned in our response to Q1, **we've included comprehensive comparisons with InstaNovo, AdaNovo, and π-HelixNovo on NovoBench.**\n\n**These expanded baselines and comparisons will be included in the camera-ready version.**\n\n> It's recommended to discuss works combining database search and de novo sequencing,  such as SearchNovo.\n\nThank you for this valuable recommendation. We have expanded **Section 2.1** to include **SearchNovo**, a concurrent development that combines database search with de novo sequencing. SearchNovo employs a retrieval mechanism to identify similar peptide-spectrum matches and integrates prefix peptide sequence information through a fusion layer. \nThis addition provides information on the broader research landscape. **We will include this expanded discussion in the camera-ready version.**\n\n> RankNovo requires the target peptide predicted by one of the base models and this may harm flexibility. The advantages of reranking methods compared to SearchNovo should be explained.\n\nThank you for this important question about flexibility limitations and advantages.\n\n```We view SearchNovo and RankNovo as complementary approaches with distinct advantages:```\n\n- **Inference-time vs. Training-time Improvement**\nWhile SearchNovo enhances itself through novel architectures and training techniques, RankNovo explores extracting additional value from existing model outputs at inference time. As models approach optimal performance on available training data, **the collective information in prediction data represents an untapped resource that RankNovo specifically leverages.**\n\n- **Training-free Scaling and Adaptation**\nSearchNovo primarily uses information from the single most similar PSM, limiting its capabilities post-training. In contrast, RankNovo's axial-attention modules integrate features from multiple base models and demonstrate training-free performance scaling with more candidates **(Section 4.3, Fig 3(A), Table 12)**. This allows practitioners to balance efficiency and performance as needed.\n\n- **Controlled Prediction Space with Enhanced Reliability**\nWhile RankNovo requires at least one base model to correctly sequence the peptide, **this constraint brings practical advantages**. By limiting modifications to existing predictions rather than generating novel sequences, **RankNovo minimizes the risk of introducing new errors**, **providing heightened reliability and interpretability** for real-world applications. Despite this constrained prediction space, **RankNovo achieves state-of-the-art performance across all benchmarks.**\n\nIn conclusion, these approaches address different aspects of sequencing challenge and can be integrated - SearchNovo could serve as a base model within RankNovo's framework, potentially combining respective advantages.\n\n> Fix typos.\n\nSorry for the confusion, the typos will be fixed in the final script.", "claims_and_evidence": "The claims made in the submission are supported by clear and convincing evidence.", "methods_and_evaluation_criteria": "### Dataset\n- Although the model has been tested on nine-species v1 & v2, it is better to evaluate it on the latest benchmark datasets [1], which include more diverse data.\n\n### Method\n- Table 5 shows that applying col-wise attention results in only a minor improvement (0.007 in Avg. Peptide Recall) compared to models without it while increasing computational complexity.\n- Tables 17 and 18 show that RankNovo's inference speed is approximately 6–8 times slower than that of the base model.\n- The innovation of the method is limited. It introduces the rerank to the de novo peptide sequencing. However, this task requires accurate inference rather than simply ranking peptides. **If none of the peptide sequences output by the base models are exact matches, the final output of the model will also be incorrect sequences, thus compromising the flexibility of de novo peptide sequencing.**\n\n[1] NovoBench: Benchmarking Deep Learning-based De Novo Peptide Sequencing Methods in Proteomics, NeurIPS 2024.", "theoretical_claims": "This paper does not have a theoretical part, so there is no need to check it.", "experimental_designs_or_analyses": "- The comparative methods in this paper should include more baseline for a more comprehensive comparison[1][2][3][4].\n\n\n[1] De novo peptide sequencing with InstaNovo: Accurate, database-free peptide identification for large scale proteomics experiments.\n\n[2] AdaNovo: Adaptive De Novo Peptide Sequencing with Conditional Mutual Information.\n\n[3] π-PrimeNovo: An Accurate and Efficient Non-Autoregressive Deep Learning Model for De Novo Peptide Sequencing.\n\n[4] π-HelixNovo for practical large-scale de novo peptide sequencing.", "relation_to_broader_scientific_literature": "- This paper introduces the rerank technique from NLP to the field of de novo peptide sequencing.", "essential_references_not_discussed": "- The key contribution of this paper is ranking the output of de novo peptide sequencing models conditioned on the spectral information. However, the final results can only be selected from the candidate peptides, which limits flexibility. SearchNovo [1] leverages spectral information for database searching and uses the retrieved candidate peptides to enhance de novo peptide sequencing, making it more flexible in comparison. This aspect is not discussed in the paper.\n\n[1] Bridging the Gap between Database Search and De Novo Peptide Sequencing with SearchNovo, bioRxiv  2024.", "other_strengths_and_weaknesses": "- The paper is written clearly.", "other_comments_or_suggestions": "### Typo  \n- In Table 15, the Avg. Peptide Recall values are 0.653. However, in Line 1010, it is written as 0.657.", "questions_for_authors": "- Fix typos.  \n- Evaluate on the latest benchmark datasets.  \n- Add more baseline models for comparison.  \n- Explain the advantages of the ranking technique in de novo peptide sequencing compared to the latest database-enhanced method, SearchNovo.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "c0ogfqHqPS", "reviewer": "Reviewer_wxvh", "summary": "The paper outlines a new reranking strategy, wherein the proposed meta-model is capable of ranking candidate peptide sequences for a given tandem spectrum. The proposed meta-model, RankNovo, innovates on existing approaches by (a) using axial attention to derive latent space representations, and (b) utilizing two novel metrics, PMD and RMD as supervision signals. The selected metrics enable a precise quantification of the predicted and true peptide sequences. Results on two benchmark datasets demonstrate the superiority of RankNovo over existing methods for de novo peptide sequencing from mass spectrometry data.\n\n## Update after rebuttal:\nI had assigned a score of 4 to the paper after my original review and the other reviews and author rebuttals have not changed my opinion, as I still believe that this work is a valuable contribution to this field.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed comments. We address your concerns as follows:\n\n> Is the choice of base models making benchmark comparison favorably for RankNovo? Should performance improvement be attributed to base models' capabilities or the reranking model?\n\nThank you for this important question about the contribution of base models versus our reranking approach.\n\nOur benchmark comparison is methodologically sound because: 1) We included all available baselines in our evaluation, **including the previous SOTA ContraNovo**. 2) Our methodology **follows established practices in ensemble research** [1][2][3], where comparing ensemble methods against constituent base models is standard.\n\nTo address whether RankNovo's performance stems from stronger base models or the reranking approach itself, our **ablation studies in Appendix D.2 and Table 9** are informative. When training and evaluating RankNovo using only five weaker models, our reranking approach still achieved 0.651 peptide recall - significantly **outperforming the excluded strongest ByNovo model** (0.623). ```This confirms that while high-performing base models contribute to results, the reranking methodology itself provides substantial independent value.```\n\n[1] Routing to the Expert: Efficient Reward-guided Ensemble of Large Language Models\n\n[2] Llm-blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion\n\n[3] End-to-end Training of CNN Ensembles for Person Re-identification\n\n> After Eq. 3, when forming the MSA embedding, it is not clear if the stacking is row-wise or column-wise.\n\nThank you for this important clarification question.\n\nThe stacking is performed **row-wise**. For a single peptide sequence embedding with dimensions (L, D)—where L represents the number of amino acids (after padding) and D represents the embedding dimension—the MSA embedding is constructed by row-wise stacking the embeddings of N peptide candidates to form a tensor of shape (N, L, D).\n\nThis MSA embedding construction **follows established practices** in the field, consistent with works such as MSA Transformer [1], AlphaFold2 [2], and AlphaFold3 [3].\n\n[1] MSA Transformer\n\n[2] Highly Accurate Protein Structure Prediction with AlphaFold\n\n[3] Accurate Structure Prediction of Biomolecular Interactions with AlphaFold 3\n\n> The description of the backbone of RankNovo in Section 3.4 would be easier to understand if it were presented in the form of a diagram.\n\nThank you for this constructive suggestion. While we included a model backbone illustration in **Figure 2(B)**, we acknowledge it doesn't fully capture all technical details of RankNovo's architecture.\n\nTo address this,  we've created a more detailed diagram focusing on **tensor shapes and attention mechanisms**, available at <https://anonymous.4open.science/r/RankNovo-F2FB/NewFig.png>.\n\n> I think it is worth reiterating or clarifying that RankNovo is the first DL-based reranking framework since some of the base models included already feature deep learning architectures.\n\nThank you for this important point. You're right - some base models do incorporate deep learning architectures.\n\nBut To clarify: ```RankNovo is novel as the first deep learning-based reranking framework specifically for de novo peptide sequencing. While existing models (including the base models) directly decode peptides through regression, RankNovo introduces a fundamentally different approach by evaluating multiple peptide candidates and selecting the optimal prediction.```\n\nThis reranking mechanism represents a methodological shift, allowing RankNovo to function as a **universal enhancement module** that integrates with various de novo sequencing models regardless of their architecture.\n\n> Just after Eq. 1, it is stated that “Intensity signals are projected to d dimension with a linear layer because of its relatively lower accuracy”. What does “its” refer to here – the linear layer or intensity signal?\n\nThank you for highlighting this ambiguity. \"Its\" refers to the **intensity signal**, not the linear layer.\n\nIntensity signals in MS/MS spectra inherently have lower accuracy compared to m/z signals due to measurement errors, as documented in Chang C, et al. [1]. Given this limitation, we follow Casanovo and ContraNovo by embedding intensity signals using a simple linear transformation, which is less sensitive to tiny changes than the sinusoidal encoding used for m/z signals.\n\n[1] Quantitative and In-Depth Survey of the Isotopic AbundanceDistribution Errors in Shotgun Proteomics\n\nQ6: What does ri refer to in Eq. 5?\n\nA6: In Equation 5:\n $$g = E_{i \\neq j} [P(r_i, r_j)] = \\frac{1}{n(n-1)} \\sum_{i=1}^{n} \\sum_{j=1, j \\neq i}^{n} |M(r_i) - M(r_j)|$$\nThe term $r_i$ refers to **the i-th residue (amino acid) in our vocabulary**. This equation defines the gap penalty $g$ as the average mass difference between all possible pairs of non-identical amino acids.", "claims_and_evidence": "The paper primarily makes two major claims:\n\n1.\tRankNovo surpasses the performance of existing approaches to de novo peptide sequencing. This is substantiated by extensive results in the main paper and the appendices.\n2.\tRankNovo is able to generalize to unseen models without any additional training. The robustness of RankNovo to base models is is borne out by both the ablation study and the zero-shot performance benchmark.", "methods_and_evaluation_criteria": "RankNovo is benchmarked against 7 other models, some of which were also included as based models, on benchmark datasets. Amino acid precision and peptide recall are reported as metrics when comparing performance. The use of these metrics is justified in Section 4.1, with the peptide recall seemingly the more important metric for the task of de novo sequencing, but amino acid precision being used for completeness’ sake.\n\nI found the evaluation to be fairly comprehensive and pertinent to the task at hand. But I have an issue with the choice of benchmark models. Specifically, the incorporation of base models as a benchmark appears to be “setting the stage” favorably for RankNovo; Casanovo V2, ContraNovo and ByNovo are included among the base models and consistently rank among the best performing benchmarks. Hence, it is not apparent if the superior performance of RankNovo over the other models is more attributable to the base models themselves being better, or the ability of the RankNovo framework to rerank peptides.", "theoretical_claims": "There are no theoretical claims included in this work.", "experimental_designs_or_analyses": "The experimental design appears to be sound with standard benchmark datasets used. The authors also report residue-level metrics in an effort to be consistent with existing literature. However, my qualm about the use of benchmark models as base models applies here too.", "relation_to_broader_scientific_literature": "The paper addresses a pertinent need in existing literature around de novo peptide sequencing. Scoring and ranking peptides based on the observed spectrum remains a difficult problem. Hence, a post-hoc reranking approach could be a great addition to the current toolbox for de novo peptide sequencing; rather than just another approach, RankNovo leverages the predictions of existing approaches on its way to delivering a more accurate peptide ranking.", "essential_references_not_discussed": "None that I can think of.", "other_strengths_and_weaknesses": "I found the ideas presented in this paper interesting and well-grounded. In particular, the use of RMD and PMD to provide supervision signals is logical but innovative and could potentially be utile to other works in the peptide-spectrum space. I also appreciate the thoroughness of the evaluation, with the inclusion of an ablation study and zero-shot performance.\n\nOn the other hand, the paper is a little difficult to follow in places. At several points, it was a struggle to make sense of what the authors meant, which detracted from the overall idea and innovations presented therein. I have pointed out specific instances in the next section; many of these comments pertain to dangling references or missing explanations of notations.", "other_comments_or_suggestions": "1.\tAfter Eq. 3, when forming the MSA embedding, it is not clear if the stacking is row-wise or column-wise.\n2.\tThe description of the backbone of RankNovo in Section 3.4 would be easier to understand if it were presented in the form of a diagram.", "questions_for_authors": "1.\tIn the last paragraph of the introduction, the authors state that RankNovo is “the first deep learning-based reranking framework”. I think it is worth reiterating or clarifying that it is the first DL-based reranking framework since some of the base models included already feature deep learning architectures.\n2.\tJust after Eq. 1, it is stated that “Intensity signals are projected to d dimension with a linear layer because of its relatively lower accuracy”. What does “its” refer to here – the linear layer or intensity signal?\n3.\tWhat does $\\mathbf{r_i}$ refer to in Eq. 5?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yo2Ms7T0cY", "reviewer": "Reviewer_EW3W", "summary": "This paper presents RankNovo, a deep reranking framework for de novo peptide sequencing that integrates multiple sequencing models using a listwise reranking approach and axial attention to extract informative features. The introduction of PMD (Peptide Mass Deviation) and RMD (Residual Mass Deviation) provides fine-grained supervision, and extensive experiments show that RankNovo outperforms its base models while achieving state-of-the-art performance with strong zero-shot generalization. The paper is well-structured and presents a compelling methodology, but some areas should be improved.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed comments. We address your concerns as follows:\n\n> A significance analysis is needed to strengthen the reliability of performance improvement.\n\nThank you for your suggestion. We agree that statistical significance analysis is essential for validating the reported improvements. In our main results, we presented only the mean statistical improvements without assessing their significance. ```To address this limitation, we conducted pairwise t-tests at the peptide level to evaluate metric improvements on the 9-species-v1 dataset.``` The t-values for the six base models are 82.4, 42.1, 38.6, 88.2, 43.2, and 50.7, respectively. For all models, the p-values are less than 1e-12. The results show that RankNovo's performance improvements over all base models are **statistically significant**:\n- All comparisons yielded t-statistics of at least 38.6943 (based on total 90K samples)\n- All p-values were below 1e-12, substantially lower than the conventional 0.05 threshold\nThese findings conclusively demonstrate that RankNovo achieves statistically significant improvements in both peptide recall and amino acid precision, providing stronger support for the conclusions presented in Section 4.2 (Main Results).\n\n> key parameters such as λ (and potentially others) should be analyzed through dedicated experiments to assess their impact on model performance.\n\nThank you for your valuable suggestion. While we provided λ evaluations in **Section 4.4** and **Appendix D.3**, we agree more analysis strengthens our work.\nWe conducted additional experiments testing different λ values:\n|λ|0|0.25|0.5|0.75|1|\n|----------|-------|-------|-------|-------|-------|\n|pep recall|0.650|0.656|0.660|0.657|0.652|\n\nOur findings show:\n- λ **impacts performance**, with neither extreme values being optimal, validating our choice of λ=0.5 which balances PMD and RMD contributions.\n- Even with sub-optimal λ values, RankNovo maintains peptide recall ≥0.650, still outperforming both ByNovo (0.623) and ContraNovo (0.618), demonstrating the framework's **inherent robustness**.\n```While computational constraints prevented exhaustive hyperparameter analysis, our experiments across five λ values provide strong evidence for the effectiveness of our selected configuration and the overall robustness of RankNovo's approach.```\n\n> a discussion on computational complexity and runtime is essential to provide a more comprehensive understanding of its feasibility in real-world applications.\n\nThank you for this important question.\n**Appendix E.5** has already discussed RankNovo's inference speeds. As expected, RankNovo is slower than single-model approaches—a natural trade-off for reranking frameworks. The main bottleneck isn't reranking itself but gathering peptide candidates from base models, which require sequential autoregression and beam search, while RankNovo's inference needs just one attention forward pass.\n\nRankNovo offers a flexible speed-performance trade-off addressing real-world concerns. Users can:\n1.Use fewer base models/candidates when prioritizing speed\n2.Scale up when maximum accuracy is critical\nThese findings are documented in **Appendix D.2 Table 12**. This adaptability is valuable in proteomics research and clinical settings where resource constraints and accuracy requirements vary significantly, enabling researchers to make informed decisions based on their specific constraints and objectives.\n\n> The zero-shot experiment lacks clear justification. How does it relate to the challenge of heterogeneous mass spectrometry data (Lines 80-82)? Is it testing generalization ability? Should it be framed as an OOD test?\n\nThank you for this important question about our zero-shot experimental setup. Our zero-shot experiments evaluate RankNovo's generalization to unseen base models, which indeed constitutes an OOD test as you correctly suggested. We train RankNovo using outputs from only a subset of base models, then test its ability to rerank predictions from all models, including previously unseen ones. ```This capability is crucial for practical applications. As new sequencing models emerge, users can apply our released checkpoint directly without retraining, ensuring RankNovo's long-term utility.```\n\nRegarding the heterogeneous MS data challenge, this is primarily addressed by **our reranking approach itself**. We deliberately employed six heterogeneous base models, each capable of performing well on different segments of the heterogeneous data (Fig. 1B). RankNovo functions as a meta-reranker that leverages collective knowledge while minimizing individual biases, directly addressing heterogeneous data distribution challenges, which is the advantage of ensemble learning proven in previous studies [1-3].\n\n[1]End-to-end training of CNN ensembles for person re-identification\n[2]Don't take the easy way out: Ensemble based methods for avoiding known dataset biases\n[3]Exploring model learning heterogeneity for boosting ensemble robustness", "claims_and_evidence": "Yes, The manuscript presents a comprehensive set of experimental results that include comparisons with baseline models, conduct of ablation studies, and exploration of hyperparameter variations.", "methods_and_evaluation_criteria": "Yes, it does.", "theoretical_claims": "Not a  theory paper.", "experimental_designs_or_analyses": "Table 1 presents the performance of **RankNovo** and baseline models on the **9-species-v1** and **9-species-v2** test sets.  \nFigure 3 illustrates the **zero-shot performance** of RankNovo.  \nTable 3 and Figure 4 show the results of the **ablation study**.", "relation_to_broader_scientific_literature": "The manuscript presents a clear and well-structured motivation，and and introduces two new metrics, PMD and RMD, to characterize quality differences at the peptide and residue levels.", "essential_references_not_discussed": "None.", "other_strengths_and_weaknesses": "Strengths :\n1. The paper is written very clearly and is easy to understand.\n2. It achieves SOTA performance on two datasets.\n3. The presentation is good.\n\nWeaknesses:\n1. Statistical Significance Analysis: While RankNovo outperforms other baselines as shown in Table 1, the paper does not mention whether the improvements are statistically significant. Including a significance analysis (e.g., t-tests or confidence intervals) would strengthen the reliability of the reported results.\n\n2. Lack of Hyperparameter Experiments: Although the appendix provides a hyperparameter table, key parameters such as λ (and potentially others) should be analyzed through dedicated experiments to assess their impact on model performance. A sensitivity study would help determine how robust RankNovo is to different hyperparameter choices.\n\n3. Computational Complexity and Time Efficiency: While RankNovo appears to be a promising approach, the paper lacks a detailed analysis of its computational cost. Given its reliance on multiple base models during training and inference, a discussion on computational complexity and runtime is essential to provide a more comprehensive understanding of its feasibility in real-world applications.\n\n4. Clarification of Zero-shot Experiment Motivation: The zero-shot experimental setup needs clearer justification. It is unclear how this experiment directly relates to the main challenge mentioned in Lines 80–82, which describes the difficulty arising from the heterogeneous nature of mass spectrometry data. Is the zero-shot setting intended to evaluate the model's generalization ability? If so, should it be framed as an OOD (Out-of-Distribution) test instead? A clearer connection between the experimental design and the stated challenge would improve the overall coherence of the study.", "other_comments_or_suggestions": "None", "questions_for_authors": "Please refer to Other Strengths And Weaknesses.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The submission presents RankNovo, a deep reranking framework designed to improve de novo peptide sequencing using multiple sequencing models. RankNovo introduces a unique reranking strategy using axial attention and introduces peptide mass deviation (PMD) and residual mass deviation (RMD) metrics, setting a new standard in sequencing benchmarks. Reviewers broadly recognised the novelty of the reranking approach and its demonstrated performance improvements, highlighting its robust generalisation to unseen models.

Review scores were 3, 3, 4, 2. The primary strengths of RankNovo include its ability to surpass previous methods in peptide recall and its successful application of axial attention. Nevertheless, reviewers had some reservations about the choice of benchmark models and the absence of testing on more challenging datasets with extensive post translation modification (PTM) coverage. The rebuttal was thorough, addressing these concerns by providing additional experimental results on diverse PTM datasets and clarifying model choices and configurations.

Reviewers engaged actively in the discussion. One raised their score after acknowledging the clarification of certain doubts, reflecting the alignment of opinion towards acceptance. The paper's comprehensive benchmarking and use of deep learning for peptide sequencing suggest meaningful practical implications in proteomics.

Given the resolution of initial concerns and the consensus on its significant contributions, I recommend accepting this paper. Its scalability and improvement across empirical tests make it a valuable addition to ICML.
Decision: Accept (poster)