AdaNovo: Towards Robust \emph{De Novo} Peptide Sequencing in Proteomics against Data Biases
Authors: Jun Xia
Shaorong Chen
Jingbo Zhou
Xiaojun Shan
Wenjie Du
Zhangyang Gao
Cheng Tan
Bozhen Hu
Jiangbin Zheng
Stan Z. Li
Venue: NeurIPS 2024 poster (2024)
Keywords: De Novo Peptide sequencing
Proteomics
Mass Spectrum
Primary area: machine_learning_for_other_sciences_and_fields
TL;DR: We propose a novel framework that calculates conditional mutual information (CMI) between the mass spectrum and each amino acid or peptide, using CMI for robust model training against data biases in proteomics.

Abstract: Tandem mass spectrometry has played a pivotal role in advancing proteomics, enabling the high-throughput analysis of protein composition in biological tissues. Despite the development of several deep learning methods for predicting amino acid sequences (peptides) responsible for generating the observed mass spectra, training data biases hinder further advancements of \emph{de novo} peptide sequencing. Firstly, prior methods struggle to identify amino acids with Post-Translational Modifications (PTMs) due to their lower frequency in training data compared to canonical amino acids, further resulting in unsatisfactory peptide sequencing performance. Secondly, various noise and missing peaks in mass spectra reduce the reliability of training data (Peptide-Spectrum Matches, PSMs). To address these challenges, we propose AdaNovo, a novel and domain knowledge-inspired framework that calculates Conditional Mutual Information (CMI) between the mass spectra and amino acids or peptides, using CMI for robust training against above biases. Extensive experiments indicate that AdaNovo outperforms previous competitors on the widely-used 9-species benchmark, meanwhile yielding 3.6\% - 9.4\% improvements in PTMs identification. The supplements contain the code.

AdaNovo: Towards Robust De Novo Peptide Sequencing in Proteomics against Data Biases
Jun SSaorong Chen'r, Jingbo Zhou Xiaojun Shan', Wenjie Du Zhangyang Gao ', Cheng Tan', Bozhen Hu', Jiangbin Zheng'. Stan Z. Li 'School of Engineering, Westlake University
University of California San Diego $University of Science and Technology of China {xiajun, stan .q....q... [oq.li)@wesst wwestalaee.keee... edu.
Abstract
Tandem mass spectrometry has played pivotal role in advancing proteomics, enabling the high-throughput analysis of protein composition in biological tissues. Despite the development of several deep learning methods for predicting amino acid sequences (peptides) responsible for generating the observed mass spectra, training data biases hinder further advancements of de novo peptide sequencing. Firstly, prior methods struggle to identify amino acids with Tost-Translational Modifications (PTMs) due to their lower frequency in training data compared canonical amino acids, further resulting in unsatisfactory peptide sequencing performance. Secondly, various noise and missing peaks in mass spectra reduce the reliability of training data (Peptide-Spectrum Matches, PSMs). To address these challenges, we propose AdaNovo, novel and domain knowledge-inspire framework that calculates Conditional Mutual Information (CMI) between the mass spectra and amino acids or peptides, using CMI for robust training against above biases. Extensive experiments indicate that AdaNovo outperforms previous competitors on the widely-used 9-species benchmark, meanwhile yielding 3.6% 9.4% improvements in PTMs identification. The code reproducing the results is available at:
com http:/uut....o//eeeeeeeeeeeeeeea
1 Introduction
Proteomics research focuses on large-scale studies characterize the proteome, the entire set of proteins in living organism. Tandem mass spectrometry serves as the only high-throughhut method analyze the protein composition in complex biological samples, playing an essential role in drug target discovery [12), PTMs discovery [16] and precision medicine [28 Peptide sequencing, i.e.,
predicting the peptide sequence for each observed mass spectrum, is core of proteomics [
Currently, two mainstream methods are employed peptide sequencing: database search and de novo peptide sequencing. The database search approaches [18] compare the observed spectrum against the spectra in pre-constructed PSMs database and pick the peptide sequence of the most similar spectrum the identification result. Obviously, database search cannot sequence the peptides out of the database. In contrast, de novo peptide sequencing deduces peptide sequence without prior knowledge of the database and thus it is essential in applications where the database is not available, such as antibody sequencing (26), human leukocyte antigen neoantigen sequencing (25). and identification of new proteins and peptides which are missing from the database 30
"Equal contribution. Corresponding author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

0.7
Novp
0.6
PaintNovo
HaC
HgC
0.5
Casanovo
Precision
0.4
Oxidation
0.3
OH
OH
H2N
H2N
0.1
0.0
Methionine
Methionine sulfoxide
MX-15999)
Q(+-99)
Amino Acids
(b)
Figure 1: (a): An example of PTMs (oxidation of methionine). (b): Comparisons of previous de novo sequencing methods in terms of amino acid-level precision. 'G' and 'A' denote Glyeine and Alanine, respectively. Both of them are canonical amino acids. 'M(+15.99) and 'Q(+.98)" represent oxidation of methionine and deamidation of glutamin, both of which are modified amino acids (the amino acids with PTMs). The results are reported using the human dataset in 9-species benchmark as test set.
Since the early 1990s, de novo sequencing methods based on the graph theory 2 g. Hidden Markov Model [6], or dynamic programming [3] 15,77 were developed to score peptide sequences against observed spectra. With the prosperity of deep learning, some researchers train the deep neural
networks with mass spectrum as the input and peptide sequence as the label (27 19 35 Although
these methods have achieved notable progress, as shown in Figure [ observe that they struggle to identify the amino acids with PTMs (such as the oxidation of methionine shown in Figure [
further leading to low peptide sequencing performance. On the other hand, the identification of amino acids with PTMs holds significant biological importance because PTMs plays a pivotal role in elucidating protein function and studying disease mechanisms 4. Additionally, some expected peaks in mass spectra may be missing due to instrument malfunction or multiple cleavage events occurring on the peptides, and some additional peaks may undesirably appear the spectrum, created by instrument noise or non-peptide molecules in the biological samples 17 All of these make the spectra and peptides labels for training being poorly matched.
To address above issues, we propose a novel framework, AdaNovo, to calculate the conditional mutual information (CMI) between the spectrum and each amino acid in its peptide label. This is inspired by the domain knowledge that the mass shifts of PTMs over canonical amino acids are only manifested in the mass spectrum. The CMI can measure the importance of different target amino acids by their dependence on the source spectrum. Based on the amino acid-level CMI, we can obtain the Mutual Information (MI) between the spectrum and the entire peptide to measure the matching level of each sesctrum-peptide pair in the training PSM data. Subsequently, we design robust training approach based on both the amino acid- and PSM-level CMI or MI, which re-weights the training losses of the corresponding amino acids adaptively.
The extensive experiments on the 9-species benchmark [27] indicate that AdaNovo generally out- performs state-of-the-as de novo peptide sequencing methods in amino acid-level or peptide-level precision and demonstrates significantly higher precision in identifying the amino acids with PTMs.
Background and Related Work
As shown in Figure proteins undergo initial digestion by enzymes, yielding a mixture of peptides. The peptides are then separated using liquid chromatography. Each charged peptide is analysed by mass spectrometer, which produces the first scan (MS1) spectra, displaying the mass-to- mass-to-charge (m//) ratio of the intact peptide. And then, each peptide will be fragmented in the mass spectrometer, and each generated second scan (MS2) spectrum comprises a collection of peaks. Each peak is tuple constitutes m/z value and an associated intensity value. The core of the above pipeline is the peptide sequencing, where we aim to predict the peptide sequence using the observed MS2 spectrum and the corresponding precursor information (mass and charge of the intact peptide). Finally, we can infer the entire protein sequence using assembly methods [4] There exist two lines of works for peptide sequencing.
standard protein identification workflow of shotgun proteomies [32]

---

Enzymes
Intensity
Biologiial Samples
Mixtures of Peptides
Tandem Mass Spectrometer
m/Z The first scan MS1
Peptide Sequencing
Sequence assembly
Intensity
ATASPPRQK
Protein Sequence
Peptide Sequence
The second scan MS2
Figure 2:
The identification workflow of shotgun proteomics 32 The peptide sequencing task is to predict the peptide sequence (e.g., ATASPPROK) for the observed MS2 spectrum, where peaks blue are signal peaks (real ions) and grey peaks denote noisy ones. The spectrum annotation are obtained with ProteomeXchange 31
The first line is database search, where compare the observed mass spectra against the theoretical fragmentation mass spectra peptide sequences in the database and pick the peptide sequence with the highest matching MSFragger [10] and Open-pFind [23]. However, these methods cannot sequence the peptides out of the database.
the result. Typical methods and tools include SEQUEST (5), pFind [I]]
The second line of works novo peptide sequencing, where we predict the peptide sequences for observed spectra without relying pre-constructed databases. Initially, researchers cast the de novo peptide sequencing task as finding the largest path in the spectrum graph [3] [4] or compute the best sequences whose fragment ions can best interpret the peaks in the observed MS2 spectrum using Hidden Markov Model 6 or dynamic programming algorithm [5]
With the prosperity of deep learning, DeepNovo [27] is the first method applying deep neural networks the task of de novo peptide sequencing. It regards the task as the image caption [22] in computer vision and incorporates the encoder-decoder architecture to predict the peptide sequence. To annotate high-resolution mass spectrometry data, PointNovo [99 adopts an order invariant network structure for peptide sequencing. More recently, Casanovo 35 first employs transformer encoder- decoder architecture [29] to predict the peptide sequence for the observed spectra. SearchNovo (34) integrates the strengths of database search and de novo sequencing to enhance peptide sequencing.
Although de novo peptide sequencing methods have achieved notable progress, we observe that they have difficulty in identifying the amino acids with PTMs because these amino acids occur much less frequently datasets compared other canonical ones. Additionally, mass spectrometry data contains significant amount of noise. All of these make the peptides labels being less reliable. The AdaNovo model proposed in this paper effectively alleviates both of them.
3 Methods
3.1 Task Formulation
Formally, we denote mass spectrum peaks in a MS2 spectrum as x [[mm,, H where each peak (m,,t) forms a 2-tuple representing the m/z and intensity value, and M the number of peaks that can be varied across different mass spectra. Also, we denote the precursor as {mp Cpree) }. consisting of the total mass mprre. lh and charge state Cpree E 21, Additionally, we represent the peptide sequence as y {(V1, ([y1,)2.-- J/..... .2. .....,,N))) WNN) }. where yı is the type of the i-th amino acid, N the peptide length and can varied across different peptides. y<j means the previous amino acids sequence appearing before the index in the peptide y. The de novo peptide
10) of the spectrum.

---

Reweight
[(x, x;
Lua
w apttan
Average
Reweight w
7(x, x; y)l
p(u x, 2, y,yc))
ply; y
Peptide Decoder
Peptide Decoder
MS Encoder
#1
y Y-
Mass Spectrum Peaks
Precursor
Prev. Sequence
Figure 3: Schematic diagram of AdaNovo framework.
sequencing models designed predict the probability of each amino acid yı given and y-ji
N
Ply x, 2;
II PVs 2,;,,,,;0)), j=1
where is the index of each amino acid position in the peptide sequence and 0 is the model parameter:
In general, previous models [7 19] are optimized using the cross-entropy (CE) loss:
N
(CE(O) = log (3) x,;,),
Σ
During inference, these models typically predict the probabilities of target amino acids in autore- gressive manner and generate hypotheses using heuristic search algorithms like beam search (1]
3.2 Model Architectures
As shown in Figure 3 AdaNovo consists of mass spectrum encoder (MS Encoder) and two peptide decoders (Peptide Decoder #1 and Peptide Decoder #2). All of these models are built on Transformer (29]. In order feed the mass spectrum peaks to MS Encoder, following Casanovo (35). we regard each mass spectrum peak (m,,4) (;) as a 'word' in natural language processing and obtain the peak embedding by individually encoding
m/z value (m,) and intensity value (t.) before combining them through summation. We employ the similar embedding approach for the precursor {(Mpreo: Spree)]: }. the embedding method is not our original contribution, we introduce the details in Appendix A As for the peptide sequence, the amino acid vocabulary encompasses 20 canonical amino acids, PTMs (oxidation of methionine, deamidation of asparagine or glutamine) and a special [stop] token indicates the end of decoding. Peptide Decoder #1 and Peptide Decoder #2 undergo autoregressive training, wherein they receive the previous sequence Y<j prior to amino acid y, during the prediction process. However, different from Peptide Decoder #1, Peptide Decoder #2 exclusively employs previous sequence y<j as input because we want to calculate the conditional probability p(U; y<j), which is the prerequisite for calculating the conditional mutual information between the mass spectrum (x and z) and amino acids.
3.3 Training Strategies
The training strategies consist of amino acid-level ($ 3.3. and PSM-level training methods ($ 3.3.2
3.3.1
Amino Acid-level Training Objective
As mentioned above, previous de novo peptide sequencing models struggle to identify amino acids with PTMs because they occur much less frequently in datasets compared to other canonical amino

---

acids. Therefore, we expect to emphasize the amino acids with PTMs to improve the models' ability in identifying them during inference. This resembles the up up-sampling methods in long- tailed classification where researchers emphasize samples from the tail class during training 36 20] We explain the reasons why these methods are unsuitable de novo peptide sequencing and compare AdaNovo with them in Section 4.6 On the other hand, when predicting the amino acid with PTMs yj+ we should rely more on mass spectrometry data (peaks x and precursor z) and less on the historical predictions of previous amino acids sequence yKę because the mass shifts resulting from PTMs are only manifested in the mass spectrometry data. This unique attribute of PSMs data motivates us to measure the mutual information (MI) between each target amino acid (y²) and the mass spectrum conditioned on previous amino acids, i.e, conditional mutual information (CMI) [3] between target amino acid and mass spectrum. Given that x, y 2. drawn from the underlying random variable X, Yj Z, Yzj. respectively, we can calculate the conditional mutual information (CMI) as,
(M,,, (yا)
T Tx, 2:Y, Y2t) = Bxx,y,2)
log
( (p,)) YCJ)
P(u; x,Z,Y<)) (x, MKح)
2(),),22)
log
p(y5 (Yci) . (x, ycj)
[Vú xxZ.8<i)
2xx,),,2)
log
p Y<j)
In this way, the CMI can be obtained with p (y/ x, z. y,,) and (リj y=j)). which are the output of the Peptide Decoder #1 and Peptide Decoder #2, respectively. Each data point (x, z, Bj is independently sampled from the joint distribution uniformly. Therefore, we can measure the depen-
dence between mass spectrometry (x, z) and y, conditioned on y<j using I log p(y.|xxa.y..))
PiNs &=
Moreover, to reduce the variances and stabilize the distribution of the amino acid-level CMI in each peptide, we normalize the CMI values the peptide using Z-score normalization and then scale the normalized values to obtain the amino acid-level training weight ws for yj+
1
aa j
max(81
0),
where 4d and n are the mean values and the standard deviations of all the CMI values in each peptide, and $1 is hyperparameter that controls the effect of amino acid-level adaptive training.
3.3.2 PSM-l Training Objective
As we introduced before, the training PSMs samples are of different matching levels because of the unexpected signal noise and missing peaks. To alleviate the negative effect of poorly matched mass spectrometry and peptide pairs and encourage the well-matched ones, we adopt the mutual information between them measure of matching levels. Formally,
p(y xx 2) p(y)
}}
N II (UJ (,,,,<5))
I(X, 2; = ((xY22) log
((X
log
II P ( (Y2))
N log
p( x, z. (2
N Σ 1
=E(xY,
(x
((5 Nej)
(5)
In other words, the mutual information can be derived by summarizing all the amino acid-level CMI over the peptide. Similarly, we can measure the matching level between mass spectrometrometry (x, z) and the peptide (y) using MI Σ I,. And then, we normalize all the MI values across all the PSMs in each mini-batch and scale the normalized values to obtain the PSM-level training weight aymm
MI
piem pam
pam max(s2
(6)
G7sm
where pa and pemm are the mean values and the standard deviations of the MI values of all the PSMs in each minibatch, and 82 is a hyperparameter that controls the effect of PSM-level adaptive training. We studied the influence of 9 and s2 in Section 4.7
5

---

3.3.3 Overall Training Objective
In the proposed method, re-weight each target amino acid Y/ with the following loss,
N
(((0,)
=
Σ
w
logp (V3 Yey, 21)
(7)
where 0 are the parameters MS Encoder and Peptide Decoder #1, and
Wy
Wj 8a
Pem
(8)
Additionally, Peptide Decoder #2 is trained with the following loss,
(2(0)) = Σ
logp (ys (z);
(9)
j=1
where 02 are the parameters of Peptide Decoder #2. The overall training loss is,
Lada (i)), + C2(2)).
(10)
3.4 Inference Strategies
In the inference phase, we feed the mass spectrometry to the encoder MS Encoder and the decoder Peptide Decoder #1 predicts the highest-scoring amino acid for each peptide sequence position. The decoder is then fed its preceding amino acid predictions each decoding step. The decoding process concludes upon predicting the [stop] token or reaching the predefined maximum peptide length of t 100 amino acids. We discuss the computational overhead of AdaNovo in Section 4.8
Experiments
4.1
Datasets
We employ the ninc-species benchmark initially introduced by DeepNovo [27]. This dataset amal- gamates approximately 1.5 million mass spectra from nine distinet species, all employing the same instrument but analyzing peptides from different species. Each spectra is associated with ground- truth peptide sequence, which comes from database search identification with a standard false discovery rate (FDR) set at 1%. Following previous works [27] 19 37,, adopt leave-one-our cross- validation framework. This entails traiming a model on eight species and testing it on the species held out for each of the nine species. We also split the eight species into training set and validation set with the ratio 9:1. This framework facilitates the testing of the model on peptide samples that have never been encountered before, which is precisely the advantage of de novo peptide sequencing methods over database search methods.
4.2 Evaluation Metrics
In our assessment of model predictions, we employ precision calculated at both the amino acid and peptide levels. following methodologies presented by previous works [15 (2). We first calculates the number of matched amino acid predictions, Naach which are defined as predicted amino acids that exhibit mass difference of < 0.1Da from the real amino acids and have either prefix suffix with a mass difference < 0.5Da from the corresponding real amino acid sequence in the ground truth peptide. Amino acid-level precision is then defined Nea / where Noa represents the number of predicted amino acids in predicted peptide sequences. Similarly, PTMs identification precision can be formulated Nyptm / /Nyptm where Nypêm and Nptr denote the number of matched PTMs and predicted amino acids with PTMs, respectively. For peptide prediction, a predicted peptide is deemed a correct match only of amino acids are matched. In collection of NP spectra, if model accurately predicts N Smatchh peptides, the peptide-level precision are defined NNP Kindly note that peptide-level performance measures are the primary quantifier the as / /NP model's 's practical utility because the goal is to assign complete peptide sequence to each spectrum.
Alll

---

4.3 Baselines and Experimental Settings
We compare AdaNovo with protein database search tool Peaks [5] and previous de novo peptide sequencing methods including DeepNovo (27], Casanovo [35] and PointNovo [19]. The MS Encoder, Peptide Decoder 1 and Peptide Decoder # AdaNovo are 9-layer Transformers, all of which come with 512 feed forward dimensions. During the training process, we used one Nvidia A A100 GPU with the batchsize as 32. We set the learning rate at 0.0004 and applied a linear warm-up. For gradient updates, used the AdamW optimizer [9]. The hyperparameters 81 and 82 are tuned within the range (0.05, 0.1, 0.33 using the validation set.
4.4
Main Results
Table I: Empirical comparison of previous models on 9-species benchmark. The best and the second best results are highlighted with bold and underlined, respectively. The reproduced performance of Casanovo is denoted as Casanovo (rep.). Following Casanovo B5]. train 5 models with different random initializations and report the standard deviation when the test dataset is Mouse or Human. The other standard deviations not reported because it would be too computationally expensive.
Species
DeepNovo
PointNovo
Peytidelevel procision
Casaova
Casmoou (rop.)
AdaNovo
DeepNowe
Poin
Amino acid-level precision
Casanovo
Casmooo (rop.)
AdaNovo
Mouse
0.286
0.355
01.433 + 0.019 D.:40 0.010
01493 - 0.815
0.623
0.666
(1.502
0.021
0.612 0.015
0.667 0.018
Homan
0.293
0.35)
0.367 0.0011 0.343 0,016
0,373 0.012
0.600
0.506
0.424 0.019
0,585 0,010
0.618
0.013
Yeast
0.462
0.534
0.750
Honeyhoe
Tomato
0.454
0.13
0.400
0.463
0.552
0.731
0.73
0.471
0.720
0.767
Rice hese
0.466
0.511
0437
0.549
0.546
0.679
I.730
0.442
0.727
0.719
Bacillus
0.449
0.518
0.540
0.513
0.561
0.742
0.768
0.573
0.711
0,788
Clam bacteria
0.253
0.298
0.57
0.347
0.397
0.002
0.589
0.405
0.612
0.656
Average
0.376
0.439
0453
0.499
0.499
0.673
0.687
0.191
0.672
0,716
DeepNova
PointNovo
Casanovo
AdaNovo
75
70
PTM-level Precision (%)
65
60
55
60
45
40
Mouse
Human
Yeast
M. mazei
Honeybee
Tamalo
Rice bean
Bacillus
Clam bacleria
Datasels
Figure 4: Empirical comparison of de novo sequencing models in terms of PTMs identification.
AdaNovo frequently outperforms previous methods on 9-species benchmark and notably excels in PTMS identification. As can be observed in Table AdaNovo outperforms competitive models on most (8 out of 9) species in peptide-level precision compared to DeepNovo, PointNovo and CasaNovo. At amino acid-level, AdaNovo also outperforms baselines on most datasets. Also, as demonstrated in Figure 4 we compare AdaNovo with other methods terms of identifying amino acids with PTMs because AdaNovo is designed to accurately identify the amino acids with PTMs. The results in the table indicate that AdaNovo consistently exceeds other competitors by significant margins (3.6% 9.4%) in identifying amino acids with PTMs, verifying the effectiveness of the amino acid-level adaptive training objective. We further evaluate the stability of AdaNovo Appendix
4.5 Ablation Study
Ablations on amino acid-level and peptide-level adaptive training strategies. To investigate the influence of the amino acid-level and peptide-level adaptive training strategies, we remove each of them from AdaNovo and evaluate the models' performance using the experimental settings in

---

Table 2: Ablations on amino acid-level (AA-level) and peptide-level training strategies. The results are reported using the Human datasets as test set.
Model
AA. Prec.
Peptide Prec.
PTM Prec.
Casunovo
0.585
0.343
0.300
AdaNovo (w/o PSM-level objective)
0.607
0.360
0.478
AdaNovo (w/ AA-leeell objective)
0.594
0.349
0.314
AdaNovo
0.618
0.373
0.483
The results shown in Table 2 indicate that both modules are necessary and effective for the AdaNovo model. More specifically, when we remove the AA-level training strategy in AdaNovo, the precision of the amino acids with PTMs identification drops significantly because the amino acid-level training strategy is designed for identifying amino acids with PTMs.
Section 4.3
Table 3: Models' Performance mass spectrum dataset with synthetic noise. The results reported using the Clam bacteria as test set.
Model
AA. Prec.
Peptide Prec.
CasaNovo
0.582
0.297
AdaNovo (w/o PSM-level objective)
0.617
0.336
AdaNovo (w/o AA-level objective)
0.644
0.372
AdaNovo
0.656
0.397
Performance on mass spectra with synthetic noise. To verify the effectiveness of the PSM-level adaptive training strategy, we randomly choose 20% spectrum the training datasets, and add synthetic noise peaks or remove original peaks with higher intensity values, We report the results in Table 3 from which we can observe that the performance would degrade sharply when we remove the PSM-level training strategy. This indicates that PSM-level adaptive training strategy can enhance models' robustness against data noise in mass spectrum.
4.6 Comparisons with Alternative Methods for identifying amino acids with PTMs
Table 4: Comparisons with alternative methods in terms of identifying amino acids with PTMs. All results are reported using the yeast dataset as test set.
Model
AA. Prec.
Peptide Prec.
PTM prec.
Casanovo
0.753
0.568
0.552
Re-weight (Upsampling)
0.762
0.576
0.564
+ Focal loss
0.745
0.543
0.550
AdaNovo (wlo PSM-level objective)
0.793
0,594
0.616
AdaNovo
0.825
0.612
0.665
In this section, we show the performance of AdaNovo only with amino acid-level loss (denoted as 'AdaNovo w/o PSM-level objective') and compare to some alternative methods terms of identifying amino acids with PTMs. The first alternative is re-weight each amino acid yj with wy = NNotat/NN y;. where Ntotal and N,, represent the total number of amino acids and the number of amino acids in the yj category in the dataset, respectively. The second alternative is the focal loss [33, we replace the cross entropy loss of Casanovo [35] with the focal loss,
f= f= - op(y; x, xx)))<s))² logply; |x,,,<<),
where a and are hyperparameters to adjust the loss weight. The results shown in Table indicate that both AdaNovo and the first alternative can help improve Casanovo's ability. Moreover, AdaNovo outperforms the first alternative by notable margin probably because the training and testing datasets are derived from different species, there exists significant difference in the distribution of PTMs quantities. Therefore, the above w, obtained with the train set is not suitable for test set. Also, AdaNovo inspired by the domain knowledge that the mass shift of PTMs only be manifested in the mass spectra, thus shows superiority over the re- re-weighting methods in mailed classification.

---

4.7 Sensitivity Analysis
0.37
0.61
0.35
0.59
0.33
0.57
0.3
0.05
0.1
0.05
0.1
g>>
0.05
0.05
S2
0.3
S2
0.3
(a) Human (Peptide-level)
(b) Human (AA-level)
Figure The effects of the two hyperparameters 81 and 82 for AdaNovo.
In this section, we investigate the effects of the two hyperparameters $1 and 82, which determines the influence of amino acid-level and PSM-level training strategy. As shown in Figure 5 we tune both 81 and 82 within the range [0.05, 0.1, 0.3) and observe that values of these two hyperparameters significantly affect the final performance of the model. Additionally, the optimal hyperparameters for peptide-level metrics may be sub-optimal to amino acids-level metrics. necessary to finely adjust the values of 8 and $2 based on the dataset, striking the balance between amino acid-level and PSM-level training strategies.
4.8 Costs of Computing and Storage
Table 5: Comparisons with competitive methods in terms of computational overhead. The training and inference time are evaluated Honeybee dataset with the same Nvidia A100 GPU.
Model
#Params (M)
Training time (h) Inference time (h)
Casanovo
47.35
56.52
7.14
AdaNovo
66.31
60.17
7.09
In this part, compare AdaNovo with Casanovo in terms of the number of model parameters, training time and inference time. The results shown in Table S Although AdaNovo outperforms Casanovo in peptide sequencing and PTMs identification, it inevitably introduced extra parameters (Peptide Decoder #2), resulting in a 40.04% increase in parameter count (from 47.35M to 66.31M). Also, under the same hardware settings (1 Nvidia 1100-XXM-800
GPU), the training time of AdaNovo increased by 6.3% (from 56.52h to 60.1 7h) over Casanovo. However, AdaNovo and Casanovo share the similar inference speed, which is important in real-world applications.
5 Conclusion and Future Work
In this paper, we discern that data biases limit progress in de novo peptide sequencing by hindering the accurate identification of PTMs and reducing the reliability PSMs due low PTM frequency and spectral noise. To address these issues, we introduce novel approach involving the calculation of conditional mutual information between the spectrum and each amino acid, followed by re- weighting the training loss of corresponding amino acids. Extensive experiments on 9-species datasets affirm that AdaNovo surpasses previous de novo sequencing methods, showcasing superior performance in both amino acid- and peptide-level precision. Notably, AdaNovo exhibits a distinct advantage in identifying amino acids with PTMs. Despite the significant progress made by AdaNovo, identifying previously unseen PTMs remains challenging, prompting the need for further research in the future.

---

6 Acknowledgement
This work was supported by National Natural Science Foundation of China Project No. 623B2086 and No. U21A20427, the Science & Technology Innovation 2030 Major Program Project No. 20212D0150100, Project No. WU2022A009 from the Center of Synthetic Biology and Integrated Bioengineering of Westlake University, and Project No. WU2023C019 from the Westlake University Industries of the Future Research. Finally, we thank the Westlake University HPC Center for providing part of the computational resources. Xiaojun Shan was not supported by any funds from China.
References
[1] Ruedi Aebersold and Matthias Mann. 12266928) 2003.
Mass spectrometry-based proteomics. Nature,
[2] Christian Bartels. Fast algorithm for peptide sequencing by mass spectroscopy. Biomedical & enviroumental mass spectrometry, 1966)3363-368 1990.
[3] Vlado Dančík, Theresa A Addona, Karl R Clauser, James Vath, and Pavel A Pevzner. De novo peptide sequencing via tandem mass spectrometry. Journal of computational biology, 6(3-4):327-342, 1999.
[4] Yonathan Lissanu Deribe, Tony Pawson, and Ivan Dikic. Post-translational modifications in signal integration. Nature structural & molecular biology, 17(6):666 (1f6:666-672,222 2010.
[5] Jimmy Eng, Ashley L McCormack, and John R Yates. An approach to correlate tandem spectral data of peptides with amino acid sequences in a protein database. Journal of the american society for mass spectrometry, 5(11):976-98) 1994.
[6] Bernd Fischer, Volker Roth, Franz Roos, Jonas Grossmann, Sacha Baginsky, Peter Widmayer, Wilhelm Gruissem, and Joachim M Buhmano. Novohmm: hidden markov model for de novo peptide sequencing. Analytical chemistry, 77(22):265-7273, 2005.
[7] Ari Frank and Pavel Pevzner. Pepnovo: de novo peptide sequencing via probabilistic network modeling. Analytical chemistry. 77(4):964 973, 2005.
[8] Ari M Frank. Predicting intensity ranks of peptide fragment ions. Journal of proteome research, 8(5):2226-2200, 2009.
[9] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
[10] Andy T Kong, Felipe V Leprevost, Dmitry Avtonomov, Dattatreya Mellacheruvu, and Alexey I Nesvizhskii Msfragger: ultrafast and comprehensive peptide identification in mass spectrometry-baaee proteomics. Nature methods, 14(5):513-520, 2017.
[11] Dequan Li, Yan Fu, Ruixiang Sun, Charles Ling, Yonggang Wei, Hu Zhou, Rong Zeng, Qiang Yang, Simin He, and Wen Gao. pfind: novel database-searching software system for automated peptide and protein identification via tandem mass spectrometry. Bioinformatics, 21(13):3049-388 2005.
[12] Eugene Lin, Chieh-Hsin Lin, and Hsien-Yure Lane. Relevant applications of generative adversarial networks in drug design and discovery: molecular de novo design, dimensionality reduction, and de novo peptide and protein design. Molecules, 25 2020.
[13] Tsung- Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, pages 2980-2988, 2017.
[14] Fan Liu, Dirk TS Rijkers, Harm Post, and Albert JR Heck. Proteome- wide profiling of protein assemblies by cross-linking mass spectrometry. Nature methods, 12(12): 2015.
[15] Bin Ma, Kaizhong Zhang, Christopher Hendrie, Chengzhi Liang, Ming Li, Amanda Doherty- Kirby, and Gilles Lajoie. Peaks: powerful software for peptide de novo sequencing by tandem mass spectrometry. Rapid communications in mass spectrometry, 17(20)2337-2344 2003.
10

---

[16] Matthias Mann and Ole N Jensen. Proteomic analysis of nost-translational modifications. Nature biotechnology, 21(3):255- 261, 2003.
[17] Kevin McDonnell, Enda Howley, and Florence Abram. The impact of noise and missing fragmentation cleavages on de novo peptide identification algorithms. Computational and Structural Biotechnology Journal, 20: 402-11224, 2022.
[18] Alexey Nesvizhskii. A survey of computational methods and error rate estimation procedures for peptide and protein identification in shotgun proteomics. Journl of proteomics, 73(11))2092-
2123, 2010.
[19] Rui Qiao, Ngoc Hieu Tran, Lei Xin, Xin Chen, Ming Li, Baozhen Shan, and Ali Ghodsi. Com- putationally instrmment-resolution wwsouuutioonddddeeeeenn de novo peptide sequencing for high-resolution devices. Nature Machine Intelligence, 3(5):420 2021.
[20] Mengye Ren, Wenyuan Zeng, Bin Yang, and Raquel Urtasun. Learning to reweight examples for robust deep learning. In International conference on machine learning, pages 4334-4344 PMLR, 2018.
[21] CARNEGIE-MELLEE UNIV PITTSBURGH PA DEPT OF COMPUTER SCIENCE. Speech Understanding Systems. Summary of Results of the Five-Yeas Research Effort at Carnegie- Mellon University. 1977.
[22] Matteo Stefanini, Marcella Cornia, Lorenzo Baraldi, Silvia Cascianelli, Giuseppe Fiameni, and Rita Cucchiara. From show to tell: A survey on deep learning-based image captioning. IEEE Iransactions pattern analysis and machine intelligence, 45(1 (5(3):539-559 2022.
[23] Jinshuai Sun, Jiahui Shi, Yihao Wang, Shujia Wu, Liping Zhao, Yanchang Li, Hong Wang, Lei Chang, Zhitang Lyu, Junzhu Wu, et al. Open-pfind pfind enhances the identification of missing proteins from human testis tissue. Journal of proteome research, 18(12):4189 (112)))689-419, 2019.
[24] J Alex Taylor and Richard S Johnson. Implementation and uses of automated de novo peptide sequencing by tandem mass spectrometry. Analytical chemistry, 73(11):233 1:2594--26000 2001.
[25] Ngoc Hieu Tran, Rui Qiao, Lei Xin, Xin Chen, Baozhen Shan, and Ming Li. Personalized deep learning of individual immunopeptidomes identify neoantigens for cancer vaccines. Nature Machine Intelligence, 2(12):764-771 2020.
[26] Ngoc Hieu Tran, M Ziaur Rahman, Lin He. Lei Xin, Baozhen Shan, and Ming Li. Complete de novo assembly of monoclonal antibody sequences. Scientific reports, 6(1):31730, 2016.
[27] Ngoc Hieu Tran, Xianglilan Zhang, Lei Xin, Baozhen Shan, and Ming Li. De novo peptide sequencing by deep learning. Proceedings of the National Academy of Sciences, 1 114431):8247- 8252, 2017.
[28] Anuli Christiana Uzozie and Ruedi Aebersold. Advancing translational research and precision medicine with targeted proteomics. Journal of proteomics, 189:1-10, 2018.
[29] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in newral information processing systems, 30, 2017.
[30] Rui Vitorino, Sofia Guedes, Fabio Trindade, Inés Correia, Gabriela Moura, Paulo Carvalho, Manuel AS Santos, and Francisco Amado. De novo sequencing of proteins by mass spectrometry. Expert Review of Proteomics, 17(7-8):555-607, 2020.
[31] Juan Vizcaíno. Eric Deutsch, Rui Wang, Attila Csordas, Florian Reisinger, Daniel Ríos, José A Dianes, Zhi Sun, Terry Farrah, Nuno Bandeira, et al. Protcomexchange provides globally coordinated proteomics data submission and dissemination. Nature biotechnology, (2(3):2232226 2014.
[32] Dirk A Wolters, Michael P Washburn, and John R Yates. An automated multidimensional protein identification technology for shotgun proteomies. Analytical chemistry, 73(23):5683-5669 2001.
11

---

[33] Aaron D Wyner. A definition of conditional mutual information for arbitrary ensembles. Information and Control, 38(1):11-59 1978.
[34] Jun Xia, Sizhe Liu, Jingbo Zhou, Shaorong Chen, and Stan Li. Bridging the gap between database search and de novo peptide sequencing with searchnovo. bioRxiv, pages 2024-10, 2024.
[35] Melih Yilmaz, William Fondrie, Wout Bittremieux, Sewoong Oh, and William S Noble. De novo mass spectrometry peptide sequencing with a transformer model. In International Conference on Machine Learning, pages 25514-25522 PMLR, 2022.
[36] Yifan Zhang, Bingyi Kang, Bryan Hooi, Shuicheng Yan, and Jiashi Feng. Deep long-tailed learning: A survey, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
[37] Jingho Zhou, Shaorong Chen, Jun Xia, Sizhe Liu, Tianze Ling, Wenjie Du, Yue Liu, Jianwei Yin, and Stan Li. Novobench: Benchmarking deep learning- based de novo peptide sequencing methods proteomics. arXiv preprint arXiv:2406. /906, 2024.
A Peak Embedding Methods
In order to feed the MS2 peaks to MS Encoder, we regard each mass spectrum peak (m,, f.) as 'word' in natural language processing and obtain its embedding by individually encoding m/z value and intensity value before combining them through summation. For a given peak, we consider its m/z as its position and employ positional encoding with reference
29
mi
d for j S 2
S6j sin
(11)
2
11
m4
d 2²
fij cos
for j
(12)
2j/d
VA1V12
where fly is the value of fo in the j-th dimension, is the embedding size of fi, and nı and n2 is an user-defined scalar and can be set to any value. Specifically, we set nı where mmax = 10,000 and mmin = 0.001 in our work. The input embeddings furnish a detailed portrayal of high-precision m/z information. Analogous the consideration of relative positions in the initial transformer model [29]. these embeddings potentially facilitate the model's 's attention to m/z variations between peaks. Such attention to detail crucial for the accurate identification of amino acids within the peptide sequence. The intensity information t; is directly encoded through linear layer Wg and mapped to the d-dimensional space Rd
and n2 2
9 Wgh
(13)
where Wg E RRd denote the parameters of the linear layer. Subsequently, the input peak (m,, t:) embedding h; generated through summation,
h = 9: + fr.
(14)
The input to adanovo's MS Encoder is the embedding of each peak, h = (h, { where M is the number of peaks mass spectra. Similarly, for the precursor z = [mpprec: Cpree)] to be fed into the Peptide Decoder #1, we embed myree using the same sinusoidal position embedding as the peaks in the spectrum. Additionally, the charge state Gpree is embedded using the embedding layer in Py Torch.
12

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Please see the abstract and the introduction section
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The proposed method cannot identify never-before-seen PTMs (see section 5)
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
13

---

Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Please section
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
14

---

Answer: [Yes]
Justification:
We have released the code in the supplementary materials.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https guwdes/ddeeeeee Cooeeeoodeesssseeeeee //nips.com / ssiniiiiccypyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: Please see section
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: expensive.
The error bars are not reported because it would be too computationally
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
15

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Please see section 4.8
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
This research conforms with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We have discussed the potential positive societal impacts in section It has no negative societal impact.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
16

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: [TODO]
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have cited related papers.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
17

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We release the code in the supplement.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: [TODO]
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: [TODO]
Guidelines:
The answer NA means that the paper does involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
18


Reviews:
[{"review_id": "J2j8EKlOw9", "reviewer": "Reviewer_XeYm", "summary": "The paper introduces AdaNovo, a novel framework for de novo peptide sequencing that significantly improves the identification of post-translational modifications (PTMs) and enhances robustness against data biases in proteomics. AdaNovo utilizes Conditional Mutual Information (CMI) to reweight training losses based on the dependence of target amino acids on mass spectrum data, addressing common issues like noise and missing peaks in mass spectra. The framework shows superior performance on a 9-species benchmark, especially in identifying amino acids with PTMs, compared to previous methods. The paper also discusses the computational efficiency and scalability of AdaNovo, suggesting it as a potent tool for advancing proteomic research.", "questions": "see weakness", "limitations": "see weakness", "rating": 3, "confidence": 3, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "1. AdaNovo significantly improves the identification of Post-Translational Modifications (PTMs), crucial for detailed proteomic analysis.\n2.  It effectively handles noise and biases in mass spectra data, ensuring reliable peptide sequencing results.", "weaknesses": "1.  In eq.3 use MI (X , Z; Yj | Y<j ) but in model it is CMI(X , Z; Yj | Y<j ), and should not distinguish between italic rv and bold rv.\n2. CMI(X , Z; Yj ) = MI (X , Z; Yj | Y<j )? ?Please provide a more detailed process.\n3. There is a lack of verification whether the Transformer's predictions align with the model's hypothesized outputs，Additionally, although CMI can technically be calculated, the inputs used in your model do not conform well to standard data formats.\n4. The paper does not provide comparisons with other methods that utilize strategies for handling repetitive long-tail data,", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> *Q1: In eq.3 use MI (X , Z; Yj | Y<j ) but in model it is CMI(X , Z; Yj | Y<j ), and should not distinguish between italic rv and bold rv.*\n\nThanks for your careful review! **Kindly note that the conditioned mutual information in model is formulated as CMI(X , Z; Yj) or MI (X , Z; Yj | Y<j ) rather than CMI(X , Z; Yj | Y<j )**. In eq.3, we can rewrite CMI(X , Z; Yj) as MI (X , Z; Yj | Y<j ) because condition mutual information is defined as the mutual information between two random variables (X , Z) and Yj conditioned on a third Y<j. In other words, CMI is a kind of mutual information. Here, we use the well-known concept of mutual information to help readers understand the calculation process of conditional mutual information.\n\nAdditionally, the bold rv denotes specific values while the italic rv denotes the variables. **It is necessary to distinguish between values (bold rv) and variables (italic rv) because the conditional mutual information is defined as a measure of the mutual dependence between two variables instead of specific values**.\n\n\n> *Q2: CMI(X , Z; Yj ) = MI (X , Z; Yj | Y<j )?  Please provide a more detailed process.*\n\n**Yes, CMI(X , Z; Yj ) = MI (X , Z; Yj | Y<j )**. Here, CMI(X , Z; Yj) denotes the conditional mutual information between (X, Z) and Yj (conditioned on Y<j). This step can be derived using the definition of Conditional Mutual Information, which is the mutual information of two random variables (X , Z) and Yj given the value of a third Y<j. In other words, CMI is a kind of mutual information. Here, we use the well-known concept of mutual information to help readers understand the calculation process of conditional mutual information.\n\n> *Q3: The paper does not provide comparisons with other methods that utilize strategies for handling repetitive long-tail data.*\n\n**We have compared with such methods in section 4.6 of the original paper**.\n\n> *Q4: There is a lack of verification whether the Transformer's predictions align with the model's hypothesized outputs. Additionally, although CMI can technically be calculated, the inputs used in your model do not conform well to standard data formats.*\n\nWe have elborated on how we feed the mass spectrum to the transformer in Appendix A. More specifically, we regard each mass spectrum peak (m_i, I_i) as a word/token in natural language processing and obtain its embedding by individually encoding its m/z value m_i and intensity value I_i before combining them through summation. The entire mass spectrum can be regarded as a sentence. The precursor can also be encoded using similar way. More details can be found in Appendix A.\n\nAdditionally, we feed mass spectrum ($\\mathbf{x}$), precuosor ($\\mathbf{z}$) and previous sequence ( $\\mathbf{y_{<j}}$ ) to the MS Encoder and Peptide Decoder #1 to predict the next amino acid ($y_j$). Therefore, the output of Peptide Decoder #1 is $p(y_{j}|\\mathbf{x}, \\mathbf{z}, \\mathbf{y_{<j}})$. Similarly, Peptide Decoder #2 takes previous sequence ($\\mathbf{y_{<j}}$) as input and outputs $p(y_{j}|\\mathbf{y_{<j}})$. And then, we can use $p(y_{j}|\\mathbf{x}, \\mathbf{z}, \\mathbf{y_{<j}})$ and $p(y_{j}|\\mathbf{y_{< j}})$ to calculate the conditional mutual information using Eq. (4).\n\n ---\nWe greatly appreciate your helpful comments, as they will undoubtedly help us improve the quality of our article. If our response has successfully addressed your concerns and clarified the ambiguities, we respectfully hope that you consider raising the score. Should you have any further questions or require additional clarification, we would be delighted to engage in further discussion. Once again, we sincerely appreciate your time and effort in reviewing our manuscript. Your feedback has been invaluable in improving our research.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sO4lZgxzXb", "reviewer": "Reviewer_xCSd", "summary": "The paper presents AdaNovo, a novel framework designed to address the biases in training data used for de novo peptide sequencing in proteomics. The main contribution is the calculation of Conditional Mutual Information (CMI) between mass spectra and amino acids, enabling robust training that mitigates the negative impacts of Post-Translational Modifications (PTMs) frequency bias and spectral noise. Extensive experiments demonstrate that AdaNovo outperforms existing methods, especially in identifying amino acids with PTMs, thus enhancing peptide sequencing performance.\n\nThe math seems clearly described to me, and well motivated. The results in table 1 highlights the improved performance. In particular, \n\nFor this review, please note that I'm not familiar with bioinformatics, and in particular the field of predicting protein sequences from mass spec.", "questions": "For Figure 4, can you supply the dataset size and the performance? Or perhaps include performance in table 1.\nHow does the amount of PTMs impact the performance vs other classifiers? Can you train models where you gradually remove the amount of PTMs and showcase the performance dropoff of your model vs. casanova?", "limitations": "I'm not familiar enough with the topic to understand if there are ethical concerns.", "rating": 7, "confidence": 1, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The use of Conditional Mutual Information (CMI) for addressing training data biases in de novo peptide sequencing is novel.\nThe experimental design is thorough, with extensive benchmarks across multiple species datasets, demonstrating significant improvements over state-of-the-art methods.\nThe paper is well-organized, with clear sections on background, methodology, experiments, and conclusions.\nThe results show substantial improvements in PTM identification, which is critical for proteomics research and applications in drug discovery and precision medicine.", "weaknesses": "This is a narrow field of proteomics, there are other problems that could be considered to truly showcase the power of the proposed methodology.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> *Q1: For Figure 4, can you supply the dataset size and the performance? How does the amount of PTMs impact the performance of Casanovo and AdaNovo?*\n\nThanks for your helpful comments! We provide the dataset size and performance in **Table Re1**. We would add these results in Figure 4 of the revised version.\n\nAdditionally, following your valuable advice, we remove 50% and 100% peptides with PTMs and report the results in **Table Re2**, from which we can observe that the performance (peptide-level precision) of Casanovo and AdaNovo significantly improves after removing the peptides with PTMs in the test set. These results indicate that the amounts of PTMs have a significant impact on the performance of de novo sequencing models and AdaNovo is less sensitive to the amounts compared with Casanovo.\n\n\n**Table Re1**: The absolute numbers for peptides with/without PTMs in 9 species datasets.\n| Species|Rice bean |Honeybee |Bacillus|Clam bacteria| Human| Mouse| M. mazei| Yeast| Tomato|\n|----|----|----|----|----|----|----|----|----|----|\n| The number of PSMs |37775  |314571 |291783 |150611 |130583|37021  |164421 |111312| 290050 | \n| The Ratios of Peptide with PTMs |27.8% |20.5% |11.8% |31.1% |27.6%|29.0%  |22.2% |22.3%| 26.4% | \n| PTMs precision of AdaNovo| 69.2% |52.4% |56.4% |57.8% |48.1% |54.8%  |54.6% |66.6%| 59.7% |\n\n**Table Re2**: The peptide-level precision of Casanovo and AdaNovo before and after removing peptides with PTMs in test set.\n| Species|Bacillus |M. mazei |Clam bacteria| \n|----|----|----|----|\n| Casanovo (before removing peptides with PTMs)|0.513|0.474|0.347 |\n| Casanovo (after removing 50% peptides with PTMs)|0.536|0.491| 0.356|\n| Casanovo (after removing 100% peptides with PTMs)|0.572|0.520| 0.385|\n| AdaNovo (before removing peptides with PTMs)|0.561|0.523|0.397|\n| AdaNovo (before removing 50% peptides with PTMs)|0.573|0.529|0.415|\n| AdaNovo (after removing 100% peptides with PTMs)|0.598|0.545|0.423|\n\n\n> *Q2: This is a narrow field of proteomics, there are other problems that could be considered to truly showcase the power of the proposed methodology.*\n\nThe initial phase of drug discovery involves pinpointing disease biomarker proteins or drug target proteins. De novo sequencing serves as a pivotal step in identifying these proteins within the realm of proteomics. **Despite remarkable progress in AI for drug design under known targets, the primary bottleneck in the drug discovery pipeline persists in uncovering these crucial protein targets**. Our objective is to encourage researchers in the AI community focus more attention this task, thereby advancing the development of AI-driven drug discovery and development (AIDD). \n\n**Moreover, many works [1,2,3] in computational mass spectra including Casanovo [2] have been published in top-tier machine learning conference including NeurIPS and ICML because the tasks in this field can be easily formulated as machine learning problem.**\n\n[1] Prefix-tree decoding for predicting mass spectra from molecules (Goldman, Samuel, et al., NeurIPS 2023)   \n[2] De Novo Mass Spectrometry Peptide Sequencing with a Transformer Model (Yilmaz et al., ICML 2022)    \n[3] Efficiently predicting high resolution mass spectra with graph neural networks (Murphy et al., ICML 2023)  \n\n ---\nWe greatly appreciate your insightful comments, as they will undoubtedly help us improve the quality of our article. If our response has successfully addressed your concerns and clarified any ambiguities, we respectfully hope that you consider raising the score/confidence. Should you have any further questions or require additional clarification, we would be delighted to engage in further discussion. Once again, we sincerely appreciate your time and effort in reviewing our manuscript. Your feedback has been invaluable in improving our research.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dqmkXAesBu", "reviewer": "Reviewer_XCqV", "summary": "This work introduces AdaNovo, a framework for improving peptide sequencing by addressing biases in training data. It calculates conditional mutual information (CMI) between mass spectra and amino acids/peptides, enhancing robustness against noise and improving PTM identification. Besides, the model consists of a Transformer-based architecture and employs adaptive training strategies. Extensive experiments show AdaNovo outperforms existing methods on a 9-species benchmark, with significant gains in PTM identification.", "questions": "1. As shown in Table 1, PointNovo performs on par with Casanovo, what are the costs of computing and storage of PointNovo? The authors only compare the costs of computing and storage of Casanovo and their AdaNovo in Table 5.\n\n2. For the results in Table 4, why using focal loss would give worse results compared to cross-entropy loss? Besides, what is the detailed number of amino acids in each category in the dataset? It would be better to provide more statistical details of the dataset in the Appendix.", "limitations": "The authors have addressed the limitations of the work.", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The introduction of AdaNovo, which leverages conditional mutual information (CMI) to tackle biases in training data, is an effective approach demonstrated by the superior performance in experiments.\n\n2. The paper is well-written and clearly structured.", "weaknesses": "1. The extra cost of memory compared to Casanovo is significant (~40%) as also indicated by the authors, which could limit the max length of predicted peptide sequences.\n2. AdaNovo is built on the Transformer encoder-decoder architecture, which was initially employed by Casanovo to predict the peptide sequence for the observed spectra. Thus I feel the novelty of the proposed model is limited.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> *Q1: The extra cost of memory compared to Casanovo is significant, which could limit the max length of predicted peptide sequences.*\n\nThanks for your insightful reciews! **In mass spectrometry, proteins are enzymatically broken down into peptides for analysis, with peptide lengths typically ranging from 2 to 50 amino acids. This is considerably shorter than full protein sequences, which can contain hundreds or thousands of amino acid residues.** Therefore, the memory constraints mentioned might not be a limiting factor for AdaNovo in predicting peptide sequences.\n\nMoreover, **we can reduce memory consumption by scaling down Decoder #2**. In the rebuttal process, we scale down Decoder #2 to transformer with 3 layers and 6 layers. The experiments shown in **Table Re1** indicate a marginal performance drop while significantly reducing memory consumption. Thank you very much for your comments! They have helped us identify issues we hadn't thought of before, significantly improving the efficiency of AdaNovo. We will update the experimental results in the revised version.\n\n**Table Re1**: The performance of scaling down Decoder #2 from 9 layers to 3 layers on Clam bacteria dataset.\n| Models| #Params (M) |Peptide-level precision | Amino acid-level precision| \n|----|----|----|----|\n| Casanovo |47.35 | 0.347| 0.617 | \n| AdaNovo with 3-leyer Decoder #2 |53.67 |0.389 | 0.642  |\n| AdaNovo with 6-leyer Decoder #2 |59.99 |0.392 |0.648  | \n| AdaNovo with 9-leyer Decoder #2 |66.31 |0.397 | 0.656 | \n\n\n\n> *Q2: What are the costs of computing and storage of PointNovo?*\n\nThanks for your valuable advice! We compare the costs of computing and storage of PointNovo, Casanovo and AdaNovo in **Table Re2**. As can be observed, PointNovo contains less parameters than Casanovo and AdaNovo with similar training and inference speed. However, as shown in Table 1 of the paper, its performance is significantly inferior to Casanovo and AdaNovo.\n\n**Table Re2**: Costs of Computing and Storage on the same device (Nvidia A100 GPU).\n| Models| #Params (M) |Training time (h) |Inference time (h)| \n|----|----|----|----|\n| PointNovo | 4.78| 57.49|7.56| \n| Casanovo | 47.35|56.52 |7.14 | \n| AdaNovo | 66.31|60.17 | 7.09 | \n\n> *Q3: What is the detailed number of amino acids in each category in the dataset? In Table 4, why using focal loss would give worse results than cross-entropy loss? *\n\nThanks for your insightful reviews! We provide the detailed number of amino acids in each category in **Table Re2** (Due to the limited sapce, we only show the number of 6 canonical amino acids + 3 PTMs and we will show all the numbers in the revised version). Additionally, focal loss is designed to address moderate class imbalance. However, in our case, canonical amnio acids vastly outnumbers PTMs, focal loss may not be able to effectively handle the imbalance, leading to suboptimal performance. Also, a previous work [3] also report that focal loss is inferior to cross-entropy loss in de novo sequencing. Thanks once again for your valuable advice! \n\n**Table Re2**: The detailed number of amino acids in each category.\n|  Amnino acids   | G      | A      | S      | P      | V      | T     | M(+15.99) | N(+. 98) | Q(+.98) |\n|--------|--------|--------|--------|--------|--------|--------|--------|--------|--------|\n| Rice bean | 45578  | 41453  | 21550  | 34618  | 34586  | 26844   | 3604    | 1572    | 265     |\n| Honey bee   | 318184 | 330329 | 285195 | 269162 | 330057 | 252258 | 3567    | 10096   | 5460 \n|\n\n\n> *Q4: AdaNovo is built on the Transformer encoder-decoder architecture. Thus I feel the novelty of the proposed model is limited.*\n\n**AdaNovo's innovation lies in its training strategy (conditional mutual information-based adaptive training) specifically tailored for spectrum-peptide matching data to mitigate the data biases (various noise and missing peaks in mass spectra and low-frequency PTMs), rather than the Transformer encoder-decoder model architecture**. The de novo peptide sequencing task we have explored, transitioning from mass spectra to amino acid sequences, shares similarities with fields like image captioning [1] (translating images into descriptive texts) and protein inverse folding [2] (deriving amino acid sequences from protein structures), where the encoder-decoder architecture is widely adopted. In these domains, innovation often stems from training strategies rather than the model architectures themselves [1,2].\n\n[1] From Show to Tell: A Survey on Deep Learning-Based Image Captioning (Stefanini et al., TPAMI 2023)    \n[2] ProteinInvBench: Benchmarking Protein Inverse Folding on Diverse Tasks, Models, and Metrics (Gao et al., NeurIPS 2023)   \n[3] De Novo Mass Spectrometry Peptide Sequencing with a Transformer Model (Yilmaz et al., ICML 2022)\n\n---\nWe greatly appreciate your insightful and helpful comments, as they will undoubtedly help us improve the quality of our article. If our response has successfully addressed your concerns and clarified any ambiguities, we respectfully hope that you consider raising the score. Should you have any further questions or require additional clarification, we would be delighted to engage in further discussion. Once again, we sincerely appreciate your time and effort in reviewing our manuscript. Your feedback has been invaluable in improving our research.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qqV4EFMBw2", "reviewer": "Reviewer_sJoV", "summary": "In the field of proteomics, tandem mass spectrometry has been crucial for analyzing protein composition in biological tissues. However, existing methods struggle to identify amino acids with Post-Translational Modifications (PTMs) due to their lower frequency in training data compared to canonical amino acids, leading to suboptimal peptide sequencing performance. Additionally, noise and missing peaks in mass spectra reduce the reliability of training data (Peptide-Spectrum Matches, PSMs. To address these challenges, the authors introduce AdaNovo, a novel framework that uses conditional mutual information to mitigate these biases. AdaNovo outperforms existing methods on a widely-used benchmark, showing significant improvements in identifying PTMs.", "questions": "See Weaknesses.", "limitations": "The authors have addressed the limitations and potential societal impact.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The motivation behind this work is strong, addressing the unique challenges of high noise levels in mass spectrometry data and the difficulty in identifying PTMs. The method is novel and simultaneously alleviates two key issues.\n\n2. The experimental results are promising, particularly in accurately identifying PTMs. Moreover, the authors provide the code for reproducing these results.\n\n3. The article is well-presented and easy to follow. For instance, the authors first introduce the process of protein identification based on mass spectrometry, which aids understanding for researchers in the AI field.", "weaknesses": "1.In the inference stage, the decoder predicts the highest-scoring amino acid for each peptide sequence position. However, the beam search has been verified to be more effective way to decode text sequence. Have the authors tried this decoding strategy in AdaNovo?\n\n2.What is the meaning of FDR = 1% in line 176? How to calculate the FDR?\n\n3.Minor issue: The space between the citation and main text is not consistent across the manuscript.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> *Q1: Have the authors tried beam search in AdaNovo?*\n\nThanks for your helpful reviews! Following your valuable advice, we apply the beam search (beam size = 5) in AdaNovo and observe consistent improvements over greedy search in **Table Re3** (peptide-level) and **Table Re4** (amino acid-level). Thanks again for such helpful advice! \n\n**Table Re3**: Comparison between greedy search and beam search in terms of **peptide-level precision**.\n| Models| Mouse |Human |Yeast| \n|----|----|----|----|\n| AdaNovo (greedy search)|0.493|0.373 | 0.612 | \n| AdaNovo (beam search) |0.523|0.395 | 0.637 | \n\n**Table Re4**: Comparison between greedy search and beam search in terms of **amino acid-level precision**.\n| Models| Mouse |Human |Yeast| \n|----|----|----|----|\n| AdaNovo (greedy search)|0.667 |0.618 | 0.825 | \n| AdaNovo (beam search) |0.690 |0.643 | 0.836 | \n\n\n> *Q2: What is the meaning of FDR = 1% in line 176? How to calculate the FDR?*\n\nFDR (False Discovery Rate) is a measure used to assess the reliability of peptide or protein identifications in database search. The typical method to calculate FDR involves using a decoy database. Here’s a simplified overview of the process:\n**1. Database Search**: Peptide sequences are identified by matching observed mass spectra to theoretical mass spectra derived from a protein sequence database. \n**2. Decoy Database**: A decoy database is created by reversing or shuffling the protein sequences in the original database. This decoy database contains sequences that mimic the characteristics of the real database but do not correspond to actual proteins.\n**3. Scoring**: Peptide identifications from the real and decoy databases are scored based on parameters such as peptide mass accuracy, retention time, and fragmentation pattern.\n**4.FDR Calculation**: The FDR is then calculated based on the number of accepted identifications from the decoy database compared to the accepted identifications from the real database. This ratio provides an estimate of the proportion of false identifications among all accepted identifications. The FDR calculation can be expressed as:\n$$\n\\text{FDR} = \\frac{\\text{Number of decoy identifications}}{\\text{Number of target identifications}}\n$$\n\nA common approach is to set a threshold for the accepted FDR, such as 1% or 5%, to control the rate of false identifications. This method allows researchers to estimate the proportion of false identifications among their identifications, providing a measure of the reliability of their results.\n\n> *Q3: Minor issue: The space between the citation and main text is not consistent across the manuscript.*\n\nThanks for your careful reviews! We will mitigate this issue in the revised version. \n\n ---\nWe greatly appreciate your insightful and helpful comments, as they will undoubtedly help us improve the quality of our article. If our response has successfully addressed your concerns and clarified any ambiguities, we respectfully hope that you consider raising the score. Should you have any further questions or require additional clarification, we would be delighted to engage in further discussion. Once again, we sincerely appreciate your time and effort in reviewing our manuscript. Your feedback has been invaluable in improving our research.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Nj1mF22H4z", "reviewer": "Reviewer_xGNa", "summary": "The paper proposed a new method in protein sequencing for tandem mass spectra, especially in solving post-translational modifications. \n\nSpecifically, two (say conditional and unconditional) decoders are designed for protein sequences generation, from which conditional mutual information is calculated and further used to weigh the training samples and calculate losses.", "questions": "1. Is \"AdaNovo w/o decoder #2 and any reweighting\" actually Casanovo? If not, please include this as a part of the ablation in Table 2-5. \n\n2. Is 72% accuracy on AA level enough in real applications? Maybe some comparisons with database-searching or physics-based methods help answer this question.\n\n3. Section 3.3.1 and 3.3.2 have confusing names. Consider changing them into amino-acid-level / peptide-level weighting. \n\n4. Eq (5) and (7) seem somewhat arbitrary. Is there any evidence that the setup is optimal?", "limitations": "Limitations are discussed in Section 5 which is somewhat formalistic.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The proposed CMI weighting framework is interesting and seemingly fits the scenario.\n\n2. Good empirical evidence for model effectiveness.\n\n3. The method may be generalized to other scenarios with imbalanced data distribution.", "weaknesses": "1. The studied problem is interesting, while may not be of interest for the major audience in NeurIPS. It is also hard for readers who are not familiar with the domain to evaluate this work.\n\n2. Casanovo appears to be the most important baseline in this paper. The authors would better articulate the relation between AdaNovo and Casanovo.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> *Q1: Is \"AdaNovo w/o decoder #2 and any reweighting\" actually Casanovo? *\n\nThanks for your helpful reviews! Yes, \"AdaNovo w/o decoder #2 and any reweighting\" is Casanovo indeed. Therefore, **we have performed the ablition study in the original version (Table 2-5).**\n\n> *Q2: Is 72% accuracy on AA level enough in real applications? Maybe some comparisons with database-searching methods help answer this question.*\n\nThe accuracy of AdaNovo is sufficient for practical applications. In fact, the earliest deep learning model for de novo peptide sequencing, DeepNovo[1], has already been integrated into the commercial software PEAKS. Additionally, what we want to emphasize is that the database search method can only identify proteins already present in the database, whereas de novo peptide sequencing can identify new proteins that are out of the database. **Therefore, database searching methods are not suited for our scenarios where the peptide sequences in test set do not exist in the database.**\n\n> *Q3: Section 3.3.1 and 3.3.2 have confusing titles. Consider changing them into amino-acid-level / peptide-level weighting.*\n\nThanks for your valuable advice! We will mitigate this issue in the revised version.\n\n\n> *Q4: The normalizations in Eq (5) and (7) seem somewhat arbitrary. Is there any evidence that the setup is optimal?*\n\nThanks for your insightful reviews! The normalization in Eq (5) and (7) is carefully designed. Specifically, we employ the widely-used Z-Score Normalization to standardize variables by transforming them into a standard normal distribution with a mean of 0 and a standard deviation of 1. The cofficients $s_1$ and $s_2$ are designed to control the effects of amino acid-level and peptide-level adaptive training, respectively. Additionally, the inclusion of +1 in the formula typically ensures that the resulting weight is non-negative and that each PSM is fully utlized for training.\n\n>*Q5: The authors would better articulate the relation between AdaNovo and Casanovo*\n\nCasanovo [2] initially employed Transformer encoder-decoder architecture to predict the peptide sequence for the observed spectra. **In comparison to Casanovo, AdaNovo's innovation lies in its training strategy specifically tailored for spectrum-peptide matching data to mitigate the data biases (various noise and missing peaks in mass spectra and low-frequency PTMs), rather than the Transformer encoder-decoder architecture**. Actually, the de novo peptide sequencing task we have explored, transitioning from mass spectra to amino acid sequences, shares similarities with fields like image captioning [5] (translating images into descriptive texts) and protein inverse folding [6] (deriving amino acid sequences from protein structures), where the encoder-decoder architecture is widely adopted. In these domains, innovation often stems from training strategies rather than the model architectures themselves.\n\n>*Q6: The studied problem is interesting, while may not be of interest for the major audience in NeurIPS.*\n\nMany works [2,3,4] in computational mass spectrum including Casanovo [2] have been published in top-tier machine learning conference including NeurIPS and ICML recently because the tasks in this field can be easily formulated as machine learning problem. \n\nMoreover, the initial phase of drug discovery involves pinpointing disease biomarker proteins or drug target proteins. De novo sequencing serves as a pivotal step in identifying these proteins within the realm of proteomics. Despite remarkable progress in AI for drug design under known targets in top-tier AI conference, the primary bottleneck in the drug discovery pipeline persists in uncovering these crucial protein targets. Our objective is to encourage researchers in the AI community focus more attention on this task, thereby advancing the development of AI-driven drug discovery and development (AIDD).\n\n[1] De novo peptide sequencing by deep learning (Tran et al., PNAS 2017)  \n[2] De Novo Mass Spectrometry Peptide Sequencing with a Transformer Model (Yilmaz et al., ICML 2022)    \n[3] Efficiently predicting high resolution mass spectra with graph neural networks (Murphy et al., ICML 2023)    \n[4] Prefix-tree decoding for predicting mass spectra from molecules (Goldman, Samuel, et al., NeurIPS 2023)    \n[5] From Show to Tell: A Survey on Deep Learning-Based Image Captioning (Stefanini et al., TPAMI 2023)    \n[6] ProteinInvBench: Benchmarking Protein Inverse Folding on Diverse Tasks, Models, and Metrics (Gao et al., NeurIPS 2023)\n\n---\nWe greatly appreciate your insightful and helpful comments, as they will undoubtedly help us improve the quality of our article. If our response has successfully addressed your concerns and clarified any ambiguities, we respectfully hope that you consider raising the score. Should you have any further questions or require additional clarification, we would be delighted to engage in further discussion. Once again, we sincerely appreciate your time and effort in reviewing our manuscript. Your feedback has been invaluable in improving our research.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "p8V5q96xLu", "reviewer": "Reviewer_DL85", "summary": "The authors introduce a novel approach for the robust de novo sequencing of peptides from mass spectrometry experiments. The novelty on the application side is the focus on post-translationally modified peptides that are commonly ignored in pure de novo sequencing approaches, however, biologically of highest relevance in many applications. Methodologically this is achieved by computing the Conditional Mutual Information between mass spectra and amino acid sequences. Evaluation is carried out in comparison against commonly used learning based tools in the field on a commonly used benchmark dataset for de novo sequencing.", "questions": "* While I understand that computing standard deviations for all experiments can be computationally prohibitive, could you also provide them at least for Mouse and Human for all tools and indicate why you believe that AdaNovo outperforms other tools?\n* Can you quantify how many of the peptides in the 9 species datasets are actually identical between species or how you protect against data leakage?\n* Can you indicate absolute numbers for peptides with/without PTMs in the 9 species benchmark\n* Can you delineate your approach from existing approaches to identify PTMs and argue why it is beneficial to include this step in de novo sequencing rather than running it separately.", "limitations": "The authors indicate limitations in identifying never observed PTMs (they may want toexplicitly  acknowledge that some open search tools actually are able to do that).", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "•\tThe authors identify a highly relevant biological question, the robust identification of post-translational peptides that is only very rarely studied in the context of de novo sequencing \n•\tThe proposed solution based on conditional mutual information well fits the problem at hand\n•\tThe authors benchmark against state-of-the-art-tool on a commonly used data set and can show improvements there", "weaknesses": "•\tThe initial hypothesis (Post-Translational Modifications due to their lower frequency in training data compared to canonical amino acids, further resulting in unsatisfactory peptide sequencing) is somewhat flawed. PTMs are rare: even most commonly occurring ones such as phosphorylation (while common on the protein level) end up being extremely sparsely occuring (<1%) on the peptide level under natural conditions. Missing PTMs is suboptimal, but the effect on the peptide sequencing level overall has to be ignorable.\n•\tWhat is commonly done (e.g. https://www.nature.com/articles/s42256-022-00467-7) is to use machine learning models to identify the location of PTMs without necessarily requiring the sequence. This is a different problem than the de novo sequencing and the authors may want to acknowledge those two different schools of thought and also may want to consider benchmarking accordingly.\n•\tThe benchmark data set is likely not adequate for learning tasks and is potentially subject to major data leakage. A high proportion of peptides between highly related organisms such as mouse and human is shared (e.g. https://www.researchgate.net/figure/Human-and-Mouse-Proteins-with-Identical-Amino-Acid-Sequences_tbl1_14316492. This is even more pronounced if you consider that highly expressed proteins (such as house keeping proteins) tend to be even better conserved. Thus, many peptides of one species are contained in the training set, but then also as part of another related species, part of the test set. This is not covering the problem of identifying novel proteins, particularly in the settings where a user would like to apply de novo sequencing (usually underexplored organisms without closely related ones with available genomes)\n•\tThe lack of error bars (due to computational complexity) makes it hard to judge the claim that the method provided by the authors outperforms other approaches. The standard deviations given actually indicate that this may not be the case and that there may be no statistically significant advance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Response to Reviewer DL85 (2/2)\n\n> *Could you provide standard deviations for Mouse and Human for all tools?*\n\nThanks for your helpful reviews! Following your valuable advice, we reproduce the performance of DeepNovo and PointNovo with standard deviations over 5 different random initializations. The results shown in **Table Re4** indicate that AdaNovo outperforms previous methods with statistically significant margins. \n\n**Table Re4**: Empirical comparison of previous models on Mouse and Human datasets in terms of **amino acid-level/peptide-level precision (with standard deviations)**.\n| Models| DeepNovo |PointNovo |CasaNovo |AdaNovo|\n|----|----|----|----|----|\n| Mouse| 0.627 ± 0.009/0.293 ± 0.012 |0.625 ± 0.010/0.352 ± 0.014 | 0.612 ± 0.015/0.449 ± 0.010 |**0.667** ± 0.018/**0.493** ± 0.015\n| Human| 0.603 ± 0.012/0.290 ± 0.019| 0.596 ± 0.014/0.342 ± 0.010|  0.585 ± 0.010/0.343 ± 0.016 | **0.618** ± 0.013/**0.373** ± 0.012\n\n> *The authors indicate limitations in identifying never observed PTMs.*\n\nThank you for your professional reviews! **The open search tools such as MSFragger [6] can identify the PTMs the human has discovered, as included in the UniMod database [7]. However, what we mean are the PTMs that have not been found by humans.** We will further clarify the distinction in the revised version.\n\n[1] Ad hoc learning of peptide fragmentation from mass spectra enables an interpretable detection of phosphorylated and cross-linked peptides (Altenburg et al., Nature Machine Intelligence)    \n[2] Large-scale prediction of protein ubiquitination sites using a multimodal deep architecture (He at al., BMC Systems Biology)   \n[3] De novo peptide sequencing by deep learning (Tran et al., PNAS)   \n[4] Computationally instrument-resolutionindependent de novo peptide sequencing for high-resolution devices (Qiao et al., Nature Machine Intelligence)   \n[5] De Novo Mass Spectrometry Peptide Sequencing with a Transformer Model (Yilmaz et al., ICML 2022)   \n[6] MSFragger: ultrafast and comprehensive peptide identification in mass spectrometry–based proteomics. (Kong et al., Nature Methods)         \n[7] Unimod: Protein modifications for mass spectrometry. (Creasy et al., Nature Methods)\n\n---\nWe greatly appreciate your insightful and helpful comments, as they will undoubtedly help us improve the quality of our article. If our response has successfully addressed your concerns and clarified any ambiguities, we respectfully hope that you consider raising the score. Should you have any further questions or require additional clarification, we would be delighted to engage in further discussion. Once again, we sincerely appreciate your time and effort in reviewing our manuscript.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: There were highly divert scores and opinions about this paper. On the one hand, there were negative comments regarding the unclear notation of conditional mutual information, the novelty compared to existing approaches like Casanovo and the high memory complexity. On the other hand, there were several positive comments about  the convincing motivation, promising experimental results, and good presentation. In the end, to me as an AC, this is a typical "borderline" case, which good arguments both for and against this paper. However, in this case, I think that some points of criticism (particularly those regarding the problematic notation of CMI) could be addressed in a convincing way in the rebuttal, and that in the end, the positive aspects outweigh the negative ones. Therefore, I finally recommend acceptance of this paper.
Decision: Accept (poster)