Bidirectional Representations Augmented Autoregressive Biological Sequence Generation
Authors: Xiang Zhang
Jiaqi Wei
Zijie Qiu
Sheng Xu
Zhi Jin
ZhiQiang Gao
Nanqing Dong
Siqi Sun
Venue: NeurIPS 2025 poster ()
Keywords: Protein Sequencing
Primary area: machine_learning_for_sciences

Abstract: Autoregressive (AR) models, common in sequence generation, are limited in many biological tasks like de novo peptide sequencing and protein modeling by their unidirectional nature, failing to capture crucial global bidirectional token dependencies. Non-Autoregressive (NAR) models offer holistic, bidirectional representations but face challenges with generative coherence and scalability.
To transcend this, we propose a hybrid framework enhancing AR generation by dynamically integrating rich contextual information from non-autoregressive mechanisms. Our approach couples a shared input encoder with two decoders: a non-autoregressive one learning latent bidirectional biological features, and an AR decoder synthesizing the biological sequence by leveraging these bidirectional features. A novel cross-decoder attention module enables the AR decoder to iteratively query and integrate these bidirectional features, enriching its predictions. This synergy is cultivated via a tailored training strategy with importance annealing for balanced objectives and cross-decoder gradient blocking for stable, focused learning.
Evaluations on a demanding 9-species benchmark of de novo peptide sequencing task show our model substantially surpasses AR and NAR baselines. It uniquely harmonizes AR stability with NAR contextual awareness, delivering robust, superior performance on diverse downstream data. This research advances biological sequence modeling techniques and contributes a novel architectural paradigm for augmenting AR models with enhanced bidirectional understanding for complex sequence generation.
Our code is available on GitHub: https://github.com/BEAM-Labs/denovo

Bidirectional Representations Augmented Autoregressive Biological Sequence Generation
Xiang 12 Jiaqi Wei
Zijie Qiu
Sheng Xu Siqi Sun 13
Zhi Jin
Zhiqiang Gao
Nanqing Dong²
Fudan University 2 University of British Columbia
Shanghai Artificial Intelligence Laboratory 4 Zhejiang University
xzhang23dualberta.co ca. siqisundfudan edu.cn
Abstract
Autoregressive (AR) models, common in sequence generation, are limited in many biological tasks like de novo peptide sequencing and protein modeling by their unidirectional nature, failiig capture crucial global bidirectional token depen- dencies. con--uuurrgressivee (NAR) models offer holistic, bidirectional represen- tations but face challenges with generative coherence and scalability. To transcend this, we propose hybrid framework enhancing AR generation by dynamically integrating rich contextual information from non-autoregressive mechanisms. Our approach couples shared inpul encoder with two decoders: non-autoregressive one learning latent bidirectional biological features, and an AR decoder synthesiz- ing the biological sequence by leveraging these bi-directional features. A novel cross-decoder attention module enables the AR decoder to iteratively query and integrate these bidirectional features, enriching its predictions. This synergy is cultivated via a tailored training strategy with importance annealing for balanced objectives and cross dess-de gradient blocking for stable, focused learning. Eval- uations on a demanding 9-species benchmark of de novo peptide sequencing task show our model substantially surpasses AR and NAR baselines. It uniquely harmonizes AR stability with NAR contextual awareness, delivering robust, supe- rior performance diverse downstream data. This research advances biological sequence modeling techniques and contributes a novel architectural paradigm for augmenting AR models with enhanced bidirectional understanding for complex sequence generation. Our code is available in GitHub.
Introduction
Biological cintences--ncludiin DNA, RNA, and proteins-encone the fundamental information Modeling these sequences requires capturing not only local motifs but also global bidirectional dependencies, as distant tokens and elements often interact in ways that critically determine biological function [1 23 For example, amino acid residues may form structural or functional interactions across long sequence distances from both directions, and motifs in DNA/RNA frequently depend upstream and downstream context (5I). Thus, methods limited to unidirec- tional modeling struggle to fully capture the biological reality where sequence meaning arises from bidirectional contextual information.
of life 35 49
Autoregressive (AR) models have become a dominant paradigm in biological sequence generation due to their capacity for tractable sequential modeling (14). However, their inherent unidirectionality imposes a fundamental constraint, limiting their ability to capture the global sequence semantics
Equal contributions. Work done while interning at Fudan University.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

477
A78
Out
1.00
Out
Dum
0.00
3.44
Figure I: Peptide Precision-Coverage Curves for Various Species. Across all subplots, the green lines are consistently positioned above the blue and orange lines, illustrating the superior performance of our model in peptide recall over varying coverage levels. (X-axis: coverage of peptides according to confidence scores, Y-axis: Peptide recall)
often crucial for accurately interpreting biological sequences. In contrast, Non- Von-Autoregressive (NAR) Transformers (NAT') employ holistic bidirectional self-attention mechanisms, enabling them to learn potent bidirectional representations. This characteristic allows NAT's to better reflect underlying biological realities and potentially achieve superior performance [46). Despite these representational advantages, NATs introduce practical and theoretical challenges. Notably, managing variable sequence lengths often necessitating the use of auxiliary length predictors that can be prone to errors 19]. navigating complex bi-directional optimization landscapes (e.g., using NATT-speiffic loss 10 can lead to training instability and reduced scalability compared to their AR counterparts 26 These inherent difficulties have, to date, hindered the full realization of NATs' bidirectional
processing strengths in the domain of biological sequence modeling.
To transcend the prevailing dichotomy between the generative stability of AR models and the rich contextual understanding afforded by NAR, we introduce CROSsNOVO, novel hybrid framework. This architecture empowers autoregressive generation by dynamically integrating potent contextual information derived from non-autoregressive mechanisms. Our approach couples a shared encoder for input condition encoding, with two specialized decoders for sequence decoding. The first, non-autoregressive decoder, is tasked with learning latent bidirectional contextual features directly from the input. Concurrently, an autoregressive decoder synthesizes the sequence, its generation process critically informed by these contextual features. The cornrrsonn of this integration a novel cross-decoder attention module, which enables the autoregressive decoder to iteratively query and incorporate the rich bidirectional representations captured by its non-autoregressive counterpart. This symbiotic relationship is cultivated through a training strategy incorporating importance annealing for balanced multi objeti-b optimization and pross-decoder gradient blocking to ensure focused representational learning and maintain training stability.
In this work, we focus on the challenging task of de novo peptide sequencing as test bed for evaluating our framework. Peptide sequencing from tandem mass spectrometry (MS/MS) data is pivotal in proteomics [2], impacting research from fundamental biology to drug development 23].
falter with novel sequences, in de novo antibody characterization [3] neoantigen discovery 17,, and metaproteomics [T3] De novo sequencing. inferring sequences directly from spectra, is thus indispensable. The problem is well-structured, computationally demanding, and highly consequential for protein sequence modeling, making it an ideal benchmark for exploring hybrid AR-NAR architectures.
However, traditional database search methods [8 25
45 6 43
On extensive 9-species benchmarks [31 and NAT baselines by uniquely harmonizing AT generative stability with NAT contextual awareness, yielding robust, superior performance. Ablation studies validate our architectural innovations, includ- ing the cross-decoder attention. This work significantly advances biological sequence modeling and
49 42
CROSSNOVO substantially surpasses AT

---

Step1: Muccher-Student Preinsining with Shared Encodar
Traning Diatllig Knowledge
Oidrectooal Perception
Spactrum Encoder
NAT Modele
Non-Autoregresttve Decoder
Shared Spectrum Encoder
Autoregressive
MJB real luteettee Loyas
Stared
Vanayy
JewSTion
Unidiectional Percaption
Speinum Parwry
Spectrum Encoder
AT Modets
Autoregreasive
diveciioo Rizelegge
of
Foed Fonaar Layers
Specttrum Paak
Mon Autornngggeeieee
Antoregrossive Probatly Matris
Probably
Updating Paramitera with CE Losa
Figure 2: The architecture of CROSSNOVO. Step 1 involves joint training with shared encoder in multitask learning framework, enabling the simultaneous training of Autoregressive (AT) and -on-Autoregressive (NAT) decoders. This approach exploits the synergistic advantages multitask learning to enhance performance. Upon convergence of both decoders, Step 2 introduces novel knowledge distillation process, transferring insights from the NAT module to the AT module through cross-decoder attention mechanism. Cross-decoder gradient blocking is employed throughout to optimize the training process.
de novo peptide sequencing, offering a novel paradigm for augmenting AT models with bidirectional understanding for complex sequence generational tasks.
2
Related Work
Autoregressive and Non-Antoregressive Transformers. Transformer architectures 32 underpin
modern sequence modeling. Autoregressive (AT) variants generate tokens sequentially, ensuring high quality but suffering from inference latency (28
42 34] Non-Autoreerressive Transformers (NAT's)
[1 33) generate tokens in parallel for speed, though sometimes trading off accuracy [46] (6).
In de nove peptide sequencing, prevalent AT models 440 [5] can suffer from error propagation. Our
work pioneers a novel approach where a non-autoregressive decoder's insights into bidirectional con-
text are used not for direct generation, but to substantially enrich and guide primary autoregressive decoder, thereby addressing this limitation.
De Novo Peptide Sequencing, Early de novo methods [20 9 have largely been superseded by deep learning. DEEPNOVO [31] initiated this shift using CNN-LSTMs. Transformer models now define the state-of-the-art. AT systems like Casanovo 40 41 and its derivatives (e.g.. AdaNovo [36], HelixNovo [9), InstaNovo 7 SearchNovo [37], ContraNovo [[3], RankNovo (28) have focused architectural refinements within the AT paradigm. PrimeNovo [46), RefineNovo 48 47,
and XuanjiNovo [16] explored NAT generation for speed, though NAT' can present challenges with
sequence length prediction and complex optimization [19]
CRossNovo introduces a distinct and novel framework that transcends the conventional AT/NAT dichotomy. We propose a unique autoregressive architecture fundamentally enhanced by rich bidi- rectional latent representations. These representations are innovatively distilled from an auxiliary non-autoregressive process, allowing our model to integrate the precision of AT decoding with the holistic contextual understanding of NATs. This pioneering hybrid formulation achieves superior accuracy by uniquely leveraging the strengths of both approaches for de novo peptide sequencing.
Method
This section elucidates the architectural design and training paradigm of CROssNOVO, a novel hybrid framework engineered for de novo peptide sequencing. We first establish the problem formulation and
3

---

notation. Subsequently, we dissect the core architectural innovations of CROSsNOvO, comprising shared spectral feature extractor, distinct autoregressive (AT) and non-autoregressive (NAT) peptide decoders, and crucially, the sophisticated mechanisms architected for their synergistic knowledge integration.
3.1 Problem Formulation and Notation Overview
The core challenge of de novo peptide sequencing is the direct inference of an amino acid sequence A = (a1, a an)) from an observed mass spectrum We formally define the input S as composite of three critical information sources: (i) the peak list I (imzz,gg)]5 detected mass-to-charge to-rrar ratios (mz) and their respective intensities (g); (ii) the precursor mass m R and (iii) the precursor charge state с E Z Our objective is to learn a mapping f + A that accurately predicts the ground-truth sequence.
representing
3.2 Model Backbone of CROSSNOVO
The architectural blueprint of CROSSNOvO centers on a novel tripartite structure: a foundational shared spectrum encoder coupled with two specialized decoders- an autoregressive (AT) head for high-fidelity sequential generation and a non-autoregressive (NAT) head engineered to capture global sequence context. This dual-decoder system is designed to synergistically harness the complementary strengths of autoregressive precision and von-autoregressive holistic understanding.
Shared Spectrum Encoder. The core of CROSsNOVO's representation learning lies in its shared spectrum encoder, a Transformer-based architecture tasked with transforming raw spectral data into rich, latent representation. We conceptualize the input peak list I as sequence of tokens. For each peak (mz;, its constituent mass-to-charge ratio mz, and intensity g; are independently projected into d-dimensional embeddings using domain- sinusoidal encoding.
ev (iin(())((C ( (1)mi 3 )), for S cos((v))((
Plmin 2 }), otherwise
2x
where is the float value (either mz; or g,) to be encoded, C (v)aax/(V)min is scaling factor, with max and (V) being the pre-defined bounds for the values. These per-value embeddings subsequently fused, typically via summation, yield initial peak embedding sequence
E(0) (ej (0) e (0)) This sequence is then refined through a stack of canonical self-attention layers:
E SelfAttenttonLaayr((EE-)) for l,...,b.
The resultant high-level feature sequence E(6) (e (b) (b)
e constitutes the pivotal shared spectral context, concurrently utilized by both the AT and NAT decoders.
AT Peptide Decoder. For sequential peptide construction, CROSSNOvO incorporates an autore- gressive (AT) decoder, which instantiates a Transformer decoder architecture carefully tailored for the nuances of peptide sequence generation. Conditioned previously generated amino acids (or ground-truth tokens during teacher forcing), represented as embeddings ()),, the AT decoder itera- tively predicts the next amino acid a, This process unfolds over L decoder layers, where each layer performs causal self-attention, producing h = CausalSelfAttn(h, (1- (bir'i)), H), to maintain autoregressive integrity. This is followed by cross-attention the shared spectrum features E()) yielding ((()) = CrossAttt(hh"" (!) E())), and finally a feed-forward network. The output represen- tation h, (2) for each token from the terminal layer sequence H is then mapped to probability distribution over the amino acid vocabulary using Pr(- S, a softmax Poftmax(WWW"" (Whh") The token a, is then sampled from this distribution. To further bolster its predictive accuracy, and drawing inspiration from established biochemical constraints [15], our AT decoder is uniquely augmented with prefix and suffix mass information at each generation step, thereby injecting vital biological context directly into the decoding process.
NAT Peptide Decoder. Complementing the AT decoder, CROSsNoVo integrates non- autoregressive (NAT) decoder specifically engineered to learn a holistic, global understanding the peptide sequence from the spectrum. Diverging from the sequential nature of the AT component,

---

the NAT decoder generates the entire peptide sequence in single pass. Architecturally. it mirrors Transformer decoder but strategically omits the causal mask its self-attention mechanisms, com- puting ve SelfAttn(v (1-1), V((-1) to enable true bidirectional context aggregation. Adopting common strategy for NAT models A6]. we operate with predefined maximum sequence length Tmax. A pivotal design choice for fostering effective knowledge distillation to the AT decoder (detailed in Section 3.4) the NAT decoder's input: it exclusively receives positional embeddings p(0) (Pi..... PTima>>: ) without any target sequence token information. These embeddings processed through L' layers, each comprising this non-causal self-attention and subsequent cross-
attention to the shared spectral features E(a), given by V; ((() CrossAttn(y;; (!) E())). The resulting
latent representations v(2) from the final NAT layer are then linearly transformed yield predictions for all T max positions concurrently.
3.3 Novel Joint Training Strategy for CROSSNOVO
central contribution of CROSSNOVO lies in its sophisticated joint training strategy, meticulously designed to concurrently optimize both the AT and NAT decoders while fostering unique unidi- rectional knowledge flow from the NAT to the AT component. This co-training paradigm leverages a shared encoder and optimizes the entire architecture via carefully constructed multitask loss function.
Multitask Learning Loss. The combined loss function Leotal is a weighted sum of the AT and NAT decoder losses:
Froisl = ARTCAT + (1 >AT)
(3)
The autoregressive loss LAr is the standard cross-entropy loss:
CAT log P(A S:
Σ logp(ac acci S;0)
(4)
where 0 represents the model parameters.
For the NAT decoder, we employ the Connectionist Temporal Classification (CTC) loss 10 to effectively handle variable-length sequences and learn robust bidirectional representations. Given maximum generation length T Tmax- the NAT decoder produces a sequence y = (y YTm.. ). The CTC mechanism introduces blank token and defines reduction function I that removes repeated tokens and then blank tokens (e.g., [(AATe = ATTG). The NAT loss aims to maximize the sum of probabilities of all paths y that be reduced to the target sequence A:
T Tmax
LuAT -loog
Ply, A
Σ
II
Importance Annealing for Balanced Optimization. Recognizing that the NAT decoder's strength lies in learning rich contextual priors while the AT decoder excels at fine-grained sequential prediction, we introduce a scheduling mechanism termed importance annealing for the weighting coefficient ÀAT- This technique, drawing conceptual parallels with curriculum learning strategies, dynamically modulates the relative contributions of the AT and NAT losses throughout training. Here, the weighting coefficient (i) progressively increases with training iteration i according to the relation >Ar(i) i/Twai, where is the current training iteration and Tiotal is the total number of iterations. This strategic annealing ensures that the NAT decoder initially plays dominant role in shaping robust spectral representations, with the optimization landscape gradually transitioning prioritize the AT decoder's objective for refining precise, high-fidelity sequence generation later stages.
3.4 Novel Cross-Decoder Attention for Knowledge Transfer
The principal innovation enabling synergistic AT-NAT interaction within CROSsNOVO is our pro- posed crosy-decoder attention mechanism. This novel module facilitates the direct injection of rich, bidirectionally-inoorne decoder's generation process. This transfer is typically activated during dedicated second fine- tuning stage or at inference. Specifically, re-engineer the AT decoder's 's cross- noss-attention submodule:
contextual knowledge, pre-learned by the NAT decoder, into the AT
5

---

Algorithm CROSSNOVO: AT Fine-tuning with Cross-Decoder NAT Knowledge Transfer
Inputs: Dataset D; Pre-trained parameters (Oencs OAT, (NAT) from Stage I; Fine-tuning epochs
En; Learning rate 7h Max NAT length T Tmax.
2: if En then
3: Freeze parameters enc and INAT-
4:
for epoch e: = En do for each (S, A) in 'D do
5:
6:
E <- Encoder(S; v
D Use frozen encoder
7:
E(h); E
(NAT)
NAT
features from frozen NAT decoder
V (L') blocked -- (B(((4))
8:
P Cross-decoder gradient blocking Augmented context for AT, cf. Eq. 8, 10 (with distinct
9:
Caug -
(1.)
blocked
E
positional encodings)
LaT-Ft < ConpitceATiossAugnnntd/c Caug; 0xT)
11:
Update 0xr using Vo.r with 7/-
Return Fine-tuned parameters 0xr (and unchanged Pene, θNTT).
instead of solely attending to the primary spectrum features E(b), the AT decoder's query states h now attend an augmented context. This augmented context is formed by concatenating with the original spectrum features p((::Tmaa)
the NAT decoder's final layer latent representations v(L")
E
P[Tmux immss+kk as shown in Equation 6
h
update CrossAttn (n/²), [viE') ((l:Tmmm)
E
P(Tmax 11: (.Tmxxx+))
1)
(6)
Here, @ denotes concatenation along the sequence dimension. The distinct positional encodings (denoted by subscript pf-)) ensure the AT decoder can differentiate and appropriately leverage these heterogeneous information sources. Optimization during this knowledge transfer phase is driven exclusively by the AT loss, CAT-
Cross-Decoder Gradient Blocking for Stable Learning. A critical consideration when fusing representations from independently (or jointly but distinctly) optimized modules is the potential for undesirable gradient interference. If the AT decoder's loss LAz were allowed to backpropagate
through the NAT-derived features v((L') v( pf1.Tmar could perturb the carefully learned representations optimized under the NAT-specific CTC loss (LHAt), phenomenon we term 'representational drift.' To preempt this, we institute a cross-decoder gradient blocking strategy. As formalized in Equation 7 the NAT features (L')
p(1:(Tmas) are treated as immutable, pre-computed contextual inputs by the AT decoder, effectively detaching them from the AT decoder's computation graph with respect gradient flow back into the NAT decoder (e.g., via 'torch. Tensor.detach())
hr
update
CrossAttn
(6,0)
GB
(L')) P(I.Tmaac)
+ E
+1:
where GB(-) denotes the gradient blocking operation. This isolation ensures that the AT decoder can effectively assimilate the distilled knowledge from the NAT branch without corrupting its source, thereby promoting stable training dynamics and enhancing the performance of CROSsNOvO.
3.5 Consideration of Bidirectional Knowledge Transfer
While our proposed knowledge transfer is unidirectional (NAT-AAT,, one might contemplate reciprocal AT-NAT pathway. Although architecturally plausible- for instance, by enabling the NAT decoder attend AT hidden states such a design introduces a substantial risk of data contamination. Specifically, due to the teacher-foreing regimen commonly employed during AT decoder training, its internal representations are directly exposed to ground-truth target tokens. Allowing the NAT decoder to access these states would inadvertently provide it with privileged information, circumventing the fundamental challenge of de novo prediction from spectral data alone. Our AT transfer mechanism, contrast, is intrinsically robust against such leakage.

---

The NAT decoder, by its design of solely using positional inputs for its sequence generation task (before its features are passed), learns representations purely from the spectrum and own output distribution's 's constraints. Consequently, the features it provides to the AT decoder are untainted by ground-truth sequence information from the AT, ensuring the integrity of the training process. This carefully designed unidirectional distillation allows CROSsNovO to effectively leverage the NAT"s bidirectional context while rigorously upholding the principles of fair and challenging model training.
4 Experiments
4.1
Experiments Setup
Dataset. Following prior work [4 46) for fair comparison, trained CROSSNOVO the
MassIVE-KB dataset [33,, which contains 30 million high-resolution peptide-spectrum matches (PSMs) from diverse instruments. For validation and benchmarking against leading methods [40] A1
46 50 we used the 9-species-vl (approx. 1.5. spectra from nine experiments) and the 9-species-v2 revised datasets. The latter offers more and higher-quality spectra with broader data distribution and stricter annotation than its predecessor.
Implementation Details. All inputs (peaks and amino acids) were embedded into 400 dimensions. The shared spectrum encoder, NAT decoder, and AT decoder of CROSsNOVO each comprise Transformer layers with 8 attention heads and 1024 hidden dimensions. We trained CROSSNoVo on eight NVIDIA A100 80GB GPUs using the AdamW optimizer [18] with an initial learning rate of 5 x 10 a linear warm-up phase, and subsequent cosine decay schedule for training stability.
Evaluation Metries. Following standard practice, we evaluated performance at amino acid (AA) and peptide levels. AA-level accuracy: An amino acid is considered correctly predicted if its mass deviation from the actual amino acid less than 0.1 Da, and its prefix or suffix mass differences do not exceed 0.5 Da relative to the corresponding segment of the ground truth peptide. Accuracy is then My, where MLA is the count of correctly predicted AAs and TA is the total number of predicted Tu
AAs. Peptide-level precision: A peptide is considered accurately predicted if all its constituent amino acids match their true counterparts. Precision is Mar, where M Mpeg is the count of correctly Typp predicted peptides and pep is the total number of peptides evaluated in the dataset.
Baselines. We benchmarked CROSSNOVO against a diverse set of baselines spanning three major methodological paradigms: Database (DB): Represented by PEAKS [20]. which performs spec- tral matching against a reference protein database, often using enzymatic digestion constraints (e.g., tryptic peptides) refine the search. Autoregressive (AT): This dominant paradigm in- cludes early deep learning models like DEEPNOVO (31) and POINTNoVO [27]. More recent state- of-the-art Transformer-based AR models include Casanovo [40]. HelixNovo [39], InstaNovo (7]. Casanovo V2 [41) (incorporating beam search), and ContraNovo [1] (utilizing contrastive learning and amino acid mass embeddings). Non-Autoregressive (NAT): A newer paradigm exemplified by PrimeNovo [46]. As the first NAR-based model for peptide generation, PrimeNovo demonstrated competitive accuracy with superior inference efficiency, highlighting the potential of NAR strategies, which CROssNovO's 's hybrid architecture distinctively builds upon.
4.2 Results
9-Species-v1 Benchmark Dataset. Table highhtghts the superior performance of CROssNOVO, establishing new state-of-thhe-rt results at both amino acid and peptide levels. Our model outperforms prior autoregressive (AR) methods across all species and metrics, with amino acid recall improving from 0.785 to 0.811 and peptide recall from 0.621 to 0.654. The recall-coverage graph in Figure [ further illustrates its dominance, consistently outperforming all baseline models at every coverage level. Moreover, our knowledge distillation techniques successfully bridge the gap between AT and NAT. Our model not only exceeds NAT in amino acid precision across all species but also surpasses NAT in peptide recall for all but two species, where it remains highly competitive. Furthermore, we observe that our combined training module has granted CROSsNovo the advantages of both AT and NAT in predicting peptides of different species. Specifically, in Human and Mouse, where models performed significantly better than NAT models, CROSsNOVo extends this advantage by further outperforming NAT by 9%. In other species where NAT models performed
Performance
7

---

Table I: Comparison of the performance of CROSSNOVo and baseline methods on the 9-species-vl
test set.
Metries
Architect
Methods
Mouse
Human
Year
M.mazei
Honeybee
Tomato
Rice bean
Bacillur
C. bacterio
Average
DB
0.639
0.673
0.633
0.728
0.586
NAT
Prime.
0.784
0.729
0.802
0.501
0.763
0.815
0.822
0.846
0.734
0.788
Amino
Deep.
0.623
0.610
0.750
0.064
0.630
0.731
0.679
0.742
0.602
0.673
Acid
Point.
0.606
0.712
0.644
Precision
Casa.
0.689
0.586
0.684
0.679
0.629
0.721
0.668
0.749
0.603
0.667
AT
Insta,
0.703
0.636
0.691
0.712
0.660
0.732
0.711
0.739
0.519
0.689
Casa.V2 V2
0.760
0.676
0.752
0.755
0.706
0.785
0.748
0.790
0.681
0.739
Helix.
0.765
0.665
0.768
0.784
0.757
0.721
0.793
0.816
0.681
0.750
Contra
0.798
0.271
0.797
0.799
0.745
0.810
0.807
0.828
0.711
0.785
Ours
0.816
0.800
0.814
0.826
0.785
0,830
0.831
0.856
0.740
0.811
DB NAT
Peaks
0.197
0.277
0.428
0.356
0.287
0.403
0,362
0.387
0.203
0.322
Prime.
0.567
0.574
0.607
0.650
0.603
0.697
0.702
0.721
0.531
0.638
Deep
0.286
0.293
0.462
0.422
0.330
0.454
0.436
0.449
0.253
0.376
Pepticle Recall
Point.
0.355
0.351
0.534
0.478
0.396
0.513
0.511
0.518
0.298
0.439
AT
Casa.
0.478
0.406
0.330
Helix.
0.483
0.392
0.568
0.560
0.473
0.560
0.623
0.596
0.388
0.517
Insta
0.471
0.455
0.559
0.528
0.466
0.732
0.564
0.576
0,416
0.530
Contra.
0.567
0.622
0.674
0.630
0.576
0.672
0.677
0.688
0.486
0.621
Ours
0.596
0.661
0.698
0.660
0.610
0.695
0.716
0.726
0.518
0.654
Table 2: Comparison the performance CROSSNOVO and baseline methods on 9-species- v2 test set. The bold font indicates the best performance.
Metrics
Architect
Methods
Mouse
Human
Yeasr
M.mazei
Honeybee
Tomato
Rice bean
Bacillas
Ciaacteria
Average
NAT
Prime.
0.839
0.893
0,932
0,908
0.862
0,909
0.931
0.921
0.827
0.891
Amino Acid
C2
0.813
0.872
0.915
0,877
0.823
0,891
0.891
0.888
0.791
0.862
Precision
Contrn.
0.913
0.901
0.807
0.882
0.939
0,939
0,927
0.906
0.837
0.849
0.737
Poptide Recall
Casa V2
0.555
0.712
0.837
0.754
0.669
0.783
0.772
0.793
0.558
0.714
Contrn.
0.616
0.820
0.854
0.780
0.711
0.794
0.799
0.815
0.575
0.752
Ours
0.651
0.850
0.885
0.819
0.751
0.816
0.847
0.850
0.607
0.786
better than AT models, CROSSNOVo leverages NAT strengths to increase its prediction accuracy by 1-3%.
Overall, our model's ability to integrate the strengths of both AR and NAR paradigms makes it robust and adaptable solution. This dual capability ensures its effectiveness across diverse species, making it a valuable tool for wide-ranging biological applications.
Performance 9-Species-v2 Benchmark Dataset. We evaluated CROSsNOvo the 9-Species- v2 benchmark, known for its diverse modifications and high qughiqualtty spectra. As detailed in Table 2] CROSsNOvO achieves ofute-of-the--a art performance, securing the highest average amino acid precision (0.906) and peptide recall (0.786) across all nine species. This demonstrates that CROSS- Novo's architecture, which integrates bidirectional context into its autoregressive framework, yields substantial improvements over existing methods.
Performance of Amino Acids with Similar Masses. Ac- curately differentiating between amino acids with very similar masses is crucial for achieving precise outcomes in peptide sequencing. We conducted rigorous evaluations to assess the performance of CROSsNovo correctly predicting these amino acids. Specifically, our objective was determine the efficacy of CROSsNovo in distin- guishing these challenging cases. For a comprehensive comparison, we also evaluated the performance of all AT models. The results, depicted in Figure 3 demonstrate CROssNOVO's exceptional performance. The consistently better performance in identifying these easily mistaken amino acids further showcases the accuracy of CROssNovo more fine- -egigineeed level.
0.8
CooaNorc Chntrallsn CHouuANV
0.4
Aming Age Figure 3: The precision comparison of CROSSNOVO with all AT models amino acids with similar masses.
8

---

Sensitivity to Beam Size. We an- alyzed the effect of beam size on CROssNovO's performance using the 9-species-vl henchmark. Table shows that while larger beam sizes initially im- prove AA Precision and Peptide Recall, these benefits plateau, and recall can slightly decrease with very large beams, possibly due to exposure bias [29]. A beam size of offers an optimal trade-off between prediction accuracy and computational cost. Detailed results are in the Appendix (Section "Influence of Various Beam Sizes").
Table 3: Effect of different beam sizes on CROSSNOVO.
Beam Size
Metric
3
5
7
11
AA Precision
0.784
0.804
0.811
0.810
0.810
0.811
Peptide Recall
0.634
0.651
0.654
0.654
0.654
0.653
Ablation Study. The ablation study in Table 4 evaluates the effects of differ- ent proposed modules on performance. CROSsNovo achieves its highest preci- sion scores when both the cross decoder with gradient blocking and the shared encoder are utilized. In contrast, omit- ting the shared encoder while retaining the cross decoder and gradient blocking significantly reduces precision. Addi- tionally, the absence of gradient blocking leads to training failure due to gradient explosion, as indicated by xx. These findings underscore the essential role of the cross-decoder with gradient blocking and the shared encoder in stabilizing and enhancing model performance.
Table 4: Results of the ablation study showing the effects of different model configurations. The "x' indicates training failure due to gradient explosion.
Gradient Cross
Shared Encoder
Amino acid Precision
Peptide Precision
Blocking
Decoder
0.795
0.643
X
0.698
0.546
0.654
Downstream Task. To demonstrate the generalizability of the proposed algorithm and its applicability, We further apply CROSSNOVO to a downstream task of identifying peptides in animal antibody data [4]. Obtaining the sequence information antibodies is crucial for understanding the structural basis of
HG-AspN
LG-LysC
HC-Chymakypin
antibody-antigee binding. recognition, and interac- tion [5]. However, existing methods for antibody protein sequencing rely on mRNA extraction from hybridoma cells [24], which can be challenging. De novo peptide sequencing offers a more accurate alter- native by predicting peptides using MS/MS.
0:30
LC-AwsN
can
Azx
We utilize a publicly available human antibody dataset (30]. which includes the light chain (LC) and heavy chain (HC) antibody proteins, each di- gested into peptides using various enzymes. We ap- ply CROSsNovo and several state-of-the-e baseline models evaluate this dataset. None of these models were trained on antibody data, and all perform purely zero-shot inference. As shown in Figure CROssNoVO significantly outperforms the baseline models in human antibody sequencing, achieving up to a 5% improvement in both peptide recall and AA precision (Appendix Table (7). We also analyzed performance on mouse antibodies, with results detailed in the Appendix.
HC-Trypsnn
HC-LysC
Casanono V2
Contraovv
Ours
Figure 4: The peptide recall comparison of all models Human antibody data.
5 Conclusion
In conclusion, our research presents CROSsNOVO, a novel approach that significantly advances de novo peptide sequencing. By effectively integrating bidirectional latent knowledge from NAT AT, CROssNovo leverages the strengths of both AT and NAT models. Through innovative architectural modifications, CROssNovo demonstrates superior performance across diverse species, surpassing both AT and NAT baselines.
Acknowledgement
This project was fully supported by the Shanghai Artificial Intelligence Laboratory (S.S.).

---

References
[1] Ruedi Aebersold and Matthias Mann. Mass spectrometry-basse proteomics.
Nature,
422(6928):188-207 2003.
[2] Ruedi Aebersold and Matthias Mann. Mass spectrometry-based proteomics.
Nature,
12266928)1188-07, 2003.
[3] Denis Beslic, Georg Tscheuschner, Bernhard Y Renard, Michael G Weller, and Thilo Muth. Comprehensive evaluation of peptide de novo sequencing tools for monoclonal antibody assem- bly. Briefings in Bioinformatics. 5:1-12, dec 2022.
[4] Denis Beslic, Georg Tscheuschner, Bernhard Y Renard, Michael G Weller, and Thilo Muth. Comprehensive evaluation of peptide de novo sequencing tools for monoclonal antibody assem- bly. Briefings in Bioinformatics, 24( (1(1):bbc3442, 2023.
[5] Mark L Chiu, Dennis R Goulet, Alexey Teplyakov, and Gary L Gilliland. Antibody structure and function: the basis for engineering therapeutics. Antibodies, 8(4):55, 2019.
[6] Jürgen Cox and Matthias Mann. MaxQuant enables high peptide identification rates, indi- vidualized p.p.b.-range mass accuracies and proteome-wide protein quantification. Nature Biotechnology. 26(12):1367-1377 2008.
[7] Kevin Eloff, Konstantinos Kalogeropoulos, Oliver Morell, Amandla Mabona, Jakob Berg Jespersen, Wesley Williams, Sam PB van Beljouw, Marcin Skwark, Andreas Hougaard Laustsen, Stan JJ Brouns, et al. De novo peptide sequencing with instanovo: Accurate, database-free peptide identification for large scale proteomics experiments. bioRxiv, pages 2023-08, 2023.
[8] J JK Eng, A L McCormack, and J R Yates. An approach to correlate tandem mass spectral data of peptides with amino acid sequences a protein database. Journal of the American Society for Mass Spectrometry, 5(1 989,9- nov 1994.
[9] Ari Frank and Pavel Pevzner. Pepnovo: de novo peptide sequencing via probabilistic network modeling. Analytical chemistry, 77(4):964 2005.
[10] Alex Graves, Santiago Fernández, Faustino Gomez, and Jürgen Schmidhuber. Connectionist. temporal classification: labelling unsegmented sequence data with recurrent neural networks. In Proceedings of the 23rd international conference on Machine learning, pages 369-376, 2006.
[11] Alex Graves and Navdeep Jaitly. Towards end-to-end speech recognition with recurrent neural networks. International conference on machine learning, pages 1764-1772. PMLR, 2014.
[12] Jiatao Gu, James Bradbury, Caiming Xiong, Victor OK Li, and Richard Socher. Non- autoregressive neural machine translation. arXiv preprint 1711::12222221111111 2017.
[13] Robert L. Hettich, Chongle Pan, Karuna Chourey, and Richard J Giannone. Metaproteomics: harnessing the power of high performance mass spectrometry to identify the suite of proteins that control metabolic activities in microbial communities. Analytical chemistry, 85(9):4203 4214, may 2013.
[14] Ming Hu, Chenglong Ma. Wei Li, Wanghan Xu, Jiamin Wu, Jucheng Hu, Tianbin Li, Guohang Zhuang, Jiaqi Liu, Yingzhou Lu, et al. A survey of scientific large language models: From data foundations agent frontiers. arXiv preprint arXiv:2508.21 2025.
[15] Zhi Jin, Sheng Xu, Xiang Zhang, Tianze Ling, Nanqing Dong, Wanli Ouyang, Zhiqiang Gao, Cheng Chang, and Siqi Sun. Contranovo: A contrastive learning approach to enhance de novo peptide sequencing. In Proceedings of the AAAI Conference on Artificial Intelligence. volume 38, pages 152, 2024.
[16] A Jun, Xiang Zhang, Xiaofan Zhang, Jiaqi Wei, Te Zhang, Yamin Deng, Pu Liu, Zongxiang Nie, Yi Chen, Nanqing Dong, et al. Massnet: billion-scale ai-friendly mass spectral corpus enables robust de novo peptide sequencing. bioRxiy, pages 2025-06, 2025.
10

---

[17] Korrawe Karunratanakul, Hsin- Tang, David W Speicher, Ekapol Chuangsuwanich, and Sira Sriswasdi. Uncovering Thousands of New Peptides with Sequence Mask-Search Hybrid De Novo Peptide Sequencing Framework. Molecular & cellular proteomics MCP, 18(12):2478- 2491, dec 2019.
[18] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
[19] Puyuan Liu, Xiang Zhang, and Lili Mou, A character-level length-control algorithm for non- autoregressive sentence summarization. Advances in Neural Information Processing Systems, 35:29101-29112, 2022.
[20] Bin Ma, Kaizhong Zhang, Christopher Hendrie, Chengzhi Liang, Ming Li, Amanda Doherty- Kirby, and Gilles Lajoie. Peaks: powerful software for peptide de novo sequencing by tandem mass spectrometry. Rapid communications in mass spectrometry. 17(20))237-22444 2003.
[21] Xuezhe Ma. Chunting Zhou, Xian Li, Graham Neubig, and Eduard Hovy. Flowseq: Non-autorrereessve conditional sequence generation with generative flow. arXiv preprint arXiv:1909.02480 2019.
[22] Clara Meister, Tim Vieira, and Ryan Cotterell. If beam search is the answer, what was the question? arXiv preprint arXiv:2010.02650, 2020.
[23] Cheuk Chi A Ng. Yin Zhou, and Zhong-Ping Yao. Algorithms for de-novo sequencing of peptides by tandem mass spectrometry: A review. Analytica Chimica Acta, page 341330, 2023.
[24] Weiwei Peng, Matti F Pronker, and Joost Snijder. Mass ppectrometry-basee de novo sequencing of monoclonal antibodies using multiple proteases and dual fragmentation scheme. Journal of Proteome Research, 20(7):3559-333 -3666, 2021.
[25] D N Perkins, D J Pappin, D M Creasy, and J S Cottrell. Probability- protein identification by searching sequence databases using mass spectrometry data. Electrophoresis, 20, 1999.
[26] Lihua Qian, Hao Zhou, Yu Bao, Mingxuan Wang, Lin Qiu, Weinan Zhang, Yong Yu, and Lei Li. Glancing transformer for non-autoregressive neural machine translation. arXiv preprint arXiv:2008.07905 2020.
[27] Rui Qiao. Ngoc Hieu Tran, Lei Xin, Xin Chen, Ming Li, Baozhen Shan, and Ali Ghodsi. Com- putationally caaneenttesoouutioo-nneeeeeeee devices. Nature Machine Intelligence, 4(5):420-425, 2021.
de novo peptide sequencing for high-resolution
[28] Zijie Qiu, Jiaqi Wei, Xiang Zhang, Sheng Xu, Kai Zou, Zhi Jin, Zhiqiang Gao, Nanqing Dong, and Siqi Sun. Universal biological sequence reranking for improved de novo peptide sequencing. arXiv preprint arXiv:2505. 2025.
[29] Mare' Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba. Sequence level training with recurrent neural networks. arXiv preprint arXiv:1511.06732. 2015.
[30] Ngoc Hieu Tran, M Ziaur Rahman, Lin He, Lei Xin, Baozhen Shan, and Ming Li. Complete de novo assembly of monoclonal antibody sequences. Scientific reports, 6(1):31730, 2016.
[31] Ngoc Hieu Tran, Xianglilan Zhang, Lei Xin, Baozhen Shan, and Ming Li. De novo peptide sequencing by deep learning. Proceedings of the National Academy of Sciences, 11441))8247-- 8252, 2017.
[32] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[33] Mingxun Wang, Jian Wang, Jeremy Carver, Benjamin S Pullman, Seong Won Cha, and Nuno Bandeira. Assembling the community-scale discoverable human proteome. Cell systems, 7(4):412442,, 2018.
11

---

[34] Jiaqi Wei, Bin Jiang, and Yanxia Zhang. Identification of blue horizontal branch stars with multimodal fusion. Publications of the Astronomical Society of the Pacific, 135(1050):845001 2023.
[35] Jiaqi Wei, Yuejin Yang, Xiang Zhang, Yuhan Chen, Xiang Zhuang, Zhangyang Gao, Dongzhan Zhou, Guangshuai Wang, Zhiqiang Gao, Juntai Cao, et al. From for science to agentic science:
A survey on autonomous scientific discovery. arXiv preprint arXiv:2508. 14111, 2025.
[36] Jun Xia, Shaorong Chen, Jingho Zhou, Tianze Ling, Wenjie Du, Sizhe Liu, and Stan Z. Li. Adanovo: Adaptive emph (De ]ovo} peptide sequencing with conditional mutual information. arXiv preprint arXiv:2403.0m 300013. 2024.
[37] Jun Xia, Sizhe Liu, Jingbo Zhou, Shaorong Chen, Zicheng Liu, Yue Liu, Stan Z Li, al. Bridging the gap between database search and amd' (De Novo} peptide sequencing with searchnovo. In NeurIPS 2024 Workshop on AI for New Drug Modalities, 2025.
[38] Yisheng Xiao, Lijun Wu, Junliang Guo, Juntao Li, Min Zhang, Tao Qin, and Tie-yan Liu. A survey on non-autoreereessvve generation for neural machine translation and beyond. IEEE 2023.
Transactions on Pattern Analysis and Machine Intelligence, 45(10)):1407-11111
[39] Tingpeng Yang. Tianze Ling, Boyan Sun, Zhendong Liang, Fan Xu, Xiansong Huang, Linhai Xie, Yonghong He, Leyuan Li, Fuchu He, al. Introducing n-helixnovo for practical large-scale de novo peptide sequencing. Briefings in Bioinformatics, 25(2):bbae021, 2024.
[40] Melih Yilmaz, William Fondrie, Wout Bittremieux, Sewoong Oh, and William S Noble. De novo mass spectrometry peptide sequencing with transformer model. In International Conference on Machine Learning, pages 25514-25522. PMLR, 2022.
[41] Melih Yilmaz, William Fondrie, Wout Bittremieux, Carlo F Melendez, Rowan Nelson, Varun Ananth, Sewoong Oh, and William Stafford Noble. Sequence-to-sequene translation from mass spectra to peptides with a transformer model. BioRxiv, pages 2023-01, 2023.
[42] Chenyu You, Haocheng Dai, Yifei Min, Jasjeet S Sekhon, Sarang Joshi, and James S Dun- can. Uncovering memorization effect in the presence of spurious correlations. Nature Communications, 16(1):5424, 2025.
[43] Jing Zhang, Lei Xin, Baozhen Shan, Weiwu Chen, Mingjie Xie, Denis Yuen, Weiming Zhang, Zefeng Zhang, Gilles A Lajoie, and Bin Ma. PEAKS DB: de novo sequencing assisted database search for sensitive and accurate peptide identification. Molecular & cellular proteomics: MCP,
1 11(4): 110::M111.011587 apr 2012.
[44] Wen Zhang, Yang Feng, Fandong Meng, Di You, and Qun Liu. Bridging the gap between training and inference for neural machine translation. arXiv preprint arXiv: 02444, 2019.
[45] Xiang Zhang, Juntai Cao, Jiaqi Wei, Chenyu You, and Dujian Ding. Why prompt design matters and works: A complexity analysis of prompt search space in llms. arXiv preprint arXiv:2503. 10084, 2025.
[46] Xiang Zhang, Tianze Ling, Zhi Jin, Sheng Xu, Zhiqiang Gao, Boyan Sun, Zijie Qiu, Nanqing Dong, Guangshuai Wang, Guibin Wang, et al. T-primenovo: An accurate and efficient non- autoregressive deep learning model for de novo peptide sequencing. bioRxiv, pages 2024-05, 05, 2024.
[47] Xiang Zhang, Jiaqi Wei, Zijie Qiu, Sheng Xu, Nanqing Dong, Zhiqiang Gao, and Siqi Sun. Curriculum learning for biological sequence prediction: The case de novo peptide sequencing. arXiv preprint arXiv:2506. 1 13485, 2025.
[48] Xiang Zhang, Jiaqi Wei, Zijie Qiu, Sheng Xu, Nanqing Dong, ZhiQiang Gao, and Siqi Sun. Curriculum learning for biological sequence prediction: The case of de novo peptide sequencing. In Forty-second International Conference on Machine Learning, 2025.
[49] Jingbo Zhou, Shaorong Chen, Jun Xia, Sizhe Sizhe Liu, Tianze Ling, Wenjie Du, Yue Liu, Jianwei Yin, and Stan Li. Novobench: Benchmarking deep learning- emph (De Novo} sequencing methods in proteomics. Advances in Neural Information Processing Systems, 37:104776-104791. 2024.
12

---

[50] Xie-Xuan Zhou, Wen-Feng Zeng, Hao Chi, Chunjie Luo, Chao Liu, Jianfeng Zhan, Si-Min He, and Zhifei Zhang. pdeep: predicting ms/ms spectra peptides with deep leariing Analytical chemistry, 89(23):12000-1266) 2017.
[51] Yuhao Zhou, Yiheng Wang, Xuming He, Ruoyao Xiao, Zhiwei Li, Qiantai Feng, Zijie Guo, Yuejin Yang, Hao Wu, Wenxuan Huang, et al. Scientists' first exam: Probing cognitive abilities of mllm via perception, understanding, and reasoning. arXiv preprint 10521, 2025.
13

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justificatinn right after your answer (even for NA).
The checklist answers
integral part of your paper submission. They visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No]" ", is perfectly acceptable to answer "[No] provvided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used").
general, answering "[No] " or "[NA]" is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract and introduction clearly state the main contributions: the proposal of CROSSNOVO, hybrid AT-NAT framework for de novo peptide sequencing; its key innovations (shared encoder, dual decoders with cross-decoder attention, specialized training strategy with importance annealing and gradient blocking); and its superior performance over AT and NAT baselines on 9-species benchmark. These claims are reflected in the methods section (Section 3) and experimental results (Section 4, Tables 2 Figure I].
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings. is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
14

---

2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
The paper includes a "Limitations" section Appendix A. discusses that while the 9-species benchmark is extensive, it does not cover all phylogenetic diversity, and performance on highly distinct or rarely investigated species remains an avenue for future work.
Justification:
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification: The paper proposes a novel architecture and empirical results. While it uses established mathematical formulations for its components (e-g., attention, loss functions described in Section 3 and Appendix B), it does not present new theoretical results in the form of theorems lemmas requiring formal proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
15

---

4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: Section 4.1 "Experiments Setup" details the datasets used (MassIVE-K) 9- species-vl, 9-species-v2), implementation details (embedding sizes, layer numbers, attention heads, hidden dimensions, optimizer, learning rate, GPU type), evaluation metrics, and baselines. Section 3 provides detailed description of the model architecture and training methods. Appendix sections further elaborate on CTC loss (B), PMC decoding (C), and beam size effects (D). This information should be sufficient to reproduce the main experimental results.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The abstract states, "Our code available for reproduction: Link Here. The datasets used (MassIVE-KBB, 9-species benchmarks, antibody datasets) are publicly available and cited (Section 4.1, Section 4.2, Appendix E). Instructions for reproducing results would presumably be part of the provided code repository.
The answer NA means that paper does not include experiments requiring code.
16

---

Please see the NeurIPS code and data submission guidelines (https://nips.cc httiai/gguiieee/eoeuuiiiiiaa
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ wttppp.ccppbllcc///oocccc gwwss///gue////eee CCoeeeummddiSmiim ssi onoolicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Section 4.1 details the datasets for training (MassIVE-KB) and valida- tion/testing (9-species-v1, 9-species-v2). Hyperparameters (embedding dimensions, layers, heads, hidden dimensions, learning rate, optimizer AdamW, LR schedule) are specified. The choice of beam size is justified in Section 4.2 and Appendix D. How other hyperparameters were chosen (e.g.. grid search) is not explicitly stated, but the values used are provided.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: While the paper presents extensive empirical comparisons across multiple benchmarks, each primary experiment was conducted a single time to obtain the final results due to cost consideration.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
17

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative еπхо rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
Section "Implementation Details" mentions the experiments compute re-
sources.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics httttp////aaurrpps.cc///////////
Answer: [Yes]
Justification:
This paper conform NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer:
[Yes]
Justification: This paper discuss positive and negative societal impacts the Appendix.
Guidelines:
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
18

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system Iearns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper develops model for peptide sequencing, which does not fall into the category of high-risk generative models or datasets requiring specific misuse safeguards deseribed in the guidelines.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All datasets and baseline models properly cited, with their respective licenses acknowledged the text.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in package should be provided. For popular datasets, paperswithcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
19

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: New assets introduced in the paper are well documented in the paper.
Guidelines:
The answer NA means that the paper does release assets.
Researchers should communicate the details the dataset/code/model as part their submissions via structured templates, This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The research is focused on developing a computational model for analyzing biological data (mass spectra) and does not involve crowdsourcing direct research with human participants.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: As the research does not involve human subjects, IRB approval is not applica- ble.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
20

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The paper does not involve the use of Large Language Models (LLMs) as core method.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM for what should or should not be described.
21

---

A Appendix
A.1
Limitations
The current study validated CROSSNOVO using an extensive 9-species benchmark, which provides substantial evidence for its robust performance and generalization capabilities across these organisms. However, focused limitation of this work is that our evaluation, while covering multiple species, did not extend an exhaustive representation of the vast phylogenetic diversity present across all kingdoms life. The species included in the benchmark, though diverse, may not fully encapsulate all proteomic complexities or unique peptide sequence characteristics that could be present organisms from exceptionally divergent evolutionary lineages or those adapted to extreme and understudied environments. Consequently, while CROSSNOVO demonstrates strong cross-species performance within the considerable scope of the current benchmarks, its specific performance nuances when applied to proteomes from such highly distinct or rarely investigated species remain an avenue for potential future investigation. This would serve to further confirm the breadth of its applicability across the widest possible biological landscape.
A.2 CTC Loss for Protein Sequence Prediction
Connectionist Temporal Classification (CTC) loss [TO] is a pivotal objective function for sequence-to- sequence modeling, especially when the precise alignment between input (e.g., mass spectra I) and output (e.g,, peptide sequence A) is unknown or variable. Our approach leverages CTC to train model that predicts amino acid sequences from spectral data without requiring explicit alignment supervision. This section details the CTC formulation as employed in our work.
A.2.1 Problem Definition
Given input spectrum Z and a target amino acid sequence AA (aj, a2..... ,,),, where U the length of the target peptide, CTC aims to maximize the sum of probabilities of all possible alignment paths (or "blank-augmented"" sequences) z (21, (2 22,,.., ........ .., 2r) of length T' (typically the length of the
model's output feature sequence) that can be reduced to A. The reduction operation Bzz) involves merging consecutive identical non-ble non-blank tokens and removing all blank tokens (c).
The conditional probability of the target sequence A given the input I is:
P(AI)-
Σ
P(z[Z).
(8)
2:B(2)=A
Assuming conditional independence of outputs given the input at each time step t:
T
P II I).
(9)
where PT) is the probability of observing token 2, at time step f. provided by the neural network. Direct summation over all paths is intractable. Thus, we employ dynamic programming approach.
A.2.2 Dynamic Programming Formulation
Let A' = (c, @1, ..,..... and between every token, resulting in sequence of length U' = 2U + 1. We define forward
... C, au, c) be the target sequence interspersed with blanks the beginning
variable (s) as the total probability of all paths that emit the prefix Ai.s using the first time steps of the model output.
The initialization for f. is:
(1 P(:1 = E(T),
(10)
an P( a1 ZT)
(11)
a1(s) = 0, >
(12)
The recursion for <>> and 2 is:
ar(s) (a)-1(s) (s) + an-1(s
1)) P(ze = A,(I),,
(13)
22

---

A = c or A', = A (allowing repeats of characters if separated by blank). If A * e and A (i.e., A. is a distinct character that must be taken, or the previous character in A' was different):
o ) (at-1 (s) + at at - 1) at (s - = A(
(14)
Appropriate boundary conditions (e.g., a (0) = 0 for t > 0, and 04 04-1 handled.
0f < 2) must be
The total probability P(A[I) is the sum of probabilities of paths ending in either the last blank Alr or the last amino acid A 1 at time T:
P(A or (U') aT - 1))
(15)
A.2.3 Loss Function
The CTC loss is the negative log-likelihood of this probability:
Lere log P(A[Z).
(16)
This formulation enables end-to-end training of our peptide sequencing model by efficiently marginal- izing over all possible alignments. Numerical stability is maintained by performing computations in log-space.
A.3 Novel Precise Mass Control (PMC) Decoding for Non-Autoregressive Transformers
We introduce Precise Mass Control (PMC), novel knapsack-style dynamic programming algorithm specifically engineered for decoding peptide sequences from non-autoregressive transformer (NAT) models. core challenge in applying NATs to de novo peptide sequencing is ensuring that generated sequences adhere to strict physical and experimental constraints. PMC addresses this by guarantecing that the decoded peptide sequence P rigorously matches the experimentally observed precursor mass Myr within user-defined tolerance ōmms- This direct integration of mass constraints into the decoding process is erucial for de novo sequencing, as it substantially prunes the vast search space of possible amino acid combinations, thereby significantly enhancing prediction accuracy and chemical validity. PMC offers principled way reconcile the parallel output generation of NATs with the precise, sequential constraints inherent in peptide mass spectrometry.
A.3.1 Problem Formulation
Let Px(y[T) denote the probability distribution over the vocabulary of amino acids VAA (augmented with a blank token e) for each position / E {1,..., Teag as predicted by NAT sequence model conditioned on input Z. Given a target precursor mass mar and a mass tolerance òmass the objective of PMC is to find peptide sequence P = (pi.....PI) This peptide P is derived by applying CTC-like collapse function, B(-), to an underlying path y (V1...., Vr,)), such that B The goal is to maximize the sum of propbbbbabilities of the path y:
Toq
y arg max
log (r ()
subject to the mass constraint:
Mpx 8 mass
Σ
u(p)) Mpe + 6masss
(17)
where u(p,) represents the monoisotopic mass of the amino acid pj in the peptide P.
A.3.2 Mass Discretization for Dynamic Programming
To facilitate the dynamic programming approach, continuous mass values are discretized. We define a discretization function (-) that maps continuous mass discrete mass bin index. Consequently, the monoisotopic mass of each amino acid aa E VAA is mapped to its discretized counterpart, u' ('(aa) = fasee (u(a))) The target precursor mass Mşr is also discretized to m'r = faise (mr)), and the maximum allowable discretized mass for any peptide prefix is denoted M The tolerance dmuss defines target mass range [mfoower m mypperl where m/ower faise (1ip Ōmass) and
mapperr faise (mpp + dmass).
23

---

A.3.3 Dynamic Programming State
The core of PMC is DP table, D. An entry D[t[[m]]aas (alaatt_______________________ stores the maximum accumulated pee-robbbbiityy of a path prefix y1..... 1/+ such that:
1. The CTC-collapsed peptide derived from y1...., Ut has total discretized mass m.
2. The last non-e token encountered in the path Ml...... Yr was jast__ne E VAA U (null}. The null value used for initialization or all preceding tokens were e.
Storing last_y_m is essential for correctly applying CTC collapse rules, particularly for handling repeats and insertions. Each entry D|t||m]|[ast_ j__n____ee________ stores a tuple: pos__abbbbblity, collapsed _epplddepppp___eeeee (pptidd_sequence). While we describe the top-1 (Viterbi) version for clarity, this DP formulation can be extended to beam search by storing the top-B candidates in each cell.
At t = O, before processing any tokens: D[o][o] [null = (0.0, sippt_sequennee Initialization
All other entries [m]]mm]] [mm]|mstttyy losst___mm y_ne] are initialized (-00, cempty_sequencee The mass m corresponds to an empty peptide.
Recursion The DP table is populated iteratively for each time step 1,..., Tey: For each previous discretized mass mold E {O.... ] For each previous last non-e token prev_y_ne e VAA {mull]: If Dit 1] Motd] [prev y __n__y__eee_____ loo llllooprobabbltty > -oo (i.e., state is reachable): Let (logP and peptide
D(t [prev_ (e [I) Let logP cand logP ald + log (T).
For each token ya VAA U {E) with probability
yt = e (blank token): The collapsed peptide and its mass remain unchanged. The last non-E token also remains prev_y__v srev_ynnee
Mhee Mold
peptide new peptide old
next y _ne prev y_ne
If logP cand > | [mnww] [next ne]e.oooo probabiiibbbbtyyy.. update D[t] Meew [next_y y y_ne] (logP cand: peptide
2. If VAA and prev_y_ne (repeat of last non-c token): This corresponds to stutter in the y-sequence (e.g., A AA). The CTC-collapsed peptide and its mass do not change. The last non-e token is updated to yt.
Mnew Mold
peptide new = peptide old
next_y_ne= 1/1
If logP cand D[t][m]ne]]] [next Inclo_ync.lloooogggoooooooooooo (logP cands wandtaapnttii peptide
pol..ooooooooooobbbiiiitt.. probability, update D[t][mnew]]]] [[]]xt [next_y y_ne]
3. If y, E VAΛ and y * prev_y vvev__me y_ne (new amino acid added to peptide): The amino acid y is appended to the collapsed peptide. Its mass is added. The last non-e token becomes yt.
Myew mold + u1())
peptide = peptide, ye (where denotes concatenation)
next___nne V/+
If myew M and logP cand hotp://eeooettty________oooobaao update DEt [new _next_y_neel__p________eee_eeeee = (logP cands peptide )
A.3.4 Final Sequence Selection
After populating the DP table up to Tsey the optimal peptide sequence P is determined. Le_____e_________ me_final E VAA U {null} and all dis- cretized masses mina such that m/ower < m finat S Wupper The peptide sequence p is the collapsed laipd__ppppp___eeeeeeeeee sequence from the entry D (eToyy[(m [last_y Besst_y________e____ee ___ne_inna]] that has the highest log probabiiiy among these candidates.
We iterate through all possible final non-c tokens last_y_ne_fime
24

---

This PMC algorithm uniquely embeds precursor mass constraints directly within CTC-compatible decoding framework for NATs. By ensuring adherence to experimental observations, PMC signifi- cantly refines the output of NAT's for challenging tasks like de novo peptide sequencing, leading to more accurate and physically plausible molecular identifications.
A.4 Influence of Various Beam Sizes
Based on the experimental results presented in Table increasing the beam size generally enhances both Amino Acid Precision and Peptide Recall across various species. At both the amino acid and peptide levels, accuracy tends to improve with larger beam sizes, though it stabilizes later, exhibiting only minor increments or remaining unchanged.
For Amino Acid Precision, increasing the beam size from to 3 significantly boosts average precision from 0.784 to 0.804, a rise 0.020. Further increases up beam size of 11 yield smaller gains, with the highest precision of 0.811 observed at beam sizes of 5 and 11, but the rate of improvement diminishes. In terms of Peptide Recall, increasing the beam size from to 3 raises average recall from 0.634 to 0.651, increase of 0.017. Beyond beam size of 3, improvements are marginal, with the highest recall of 0.654 achieved at beam sizes of 7, and Some species exhibit slight recall decreases at larger beam sizes, likely due to the Seq2Seq model's exposure bias 229
44 22). The
model, trained with Teacher Forcing, consistently receives the correct prior output during training but must generate its during inference, leading to potential deviations from the optimal solution. Larger beam sizes can mitigate this issue, but excessively large sizes might cause overfitting and hinder generalization.
While larger beam sizes can enhance prediction performance, they also increase inference costs. To balance effectiveness and speed, we selected beam size of for our experiments. With this size, model achieves high performance metrics, with an average precision of 0.811 and an average recall of 0.654, showing minimal differences compared to larger beam sizes. Additionally, compared to beam sizes of 9 or 11, a beam size of offers faster inference speed and lower computational demands, maintaining prediction performance while optimizing efficiency. Therefore, considering both performance and computational costs, a beam size considered optimal, achieving an effective balance between accuracy and efficiency.
Table Comparison of Amino Acid Precision and Peptide Recall for 9-Species-vl Various Beam
Sizes
Metrics
Species
1-Beam
3-Beam
5-Beam
7-Beam
9-Beam
11 11-Beam
Bacillus
0.829
0.850
0.856
0.854
0.854
0.855
Clambacteria
0.713
0.728
0.740
0.734
0.734
0.734
Honeybee
0.758
0.779
0.785
0.785
0.785
0.786
Human
0.766
0.792
0.800
0,800
0.802
0,802
Amino
M.mazei
0.801
0,819
0.826
0.824
0.825
0.824
Acid
Mouse
0.794
0.813
0.816
0.816
0.816
0.817
Precision
Ricebean
0.793
0.820
0.831
0.827
0.828
0.828
Tomato
0.812
0.826
0.830
0.830
0.832
0.832
Yeast
0.791
0.809
0.814
0.815
0.815
0.816
Average
0.784
0.804
0.811
0.810
0.810
0.811
Bacillus
0.706
0.725
0.726
0.727
0.726
0.726
Clambacteria
0.502
0.517
0.518
0,519
0.519
0,518
Honeybee
0.591
0.607
0.610
0.610
0.610
0.610
Human
0.632
0.657
0.661
0.663
0.664
0.663
Peptide Recall
M.mazei
0.642
0.658
0.660
0.660
0.660
0.660
Mouse
0.577
0.593
0.596
0.596
0.596
0.595
Ricebean
0.686
0.712
0.716
0.717
0.717
0.717
Tomato
0.682
0.694
0.695
0.694
0.694
0.694
Yeast
0.684
0.696
0.698
0.698
0.698
0.698
Average
0.634
0.651
0.654
0.654
0.654
0.653
25

---

A.5 Precision-Coveragg Curves
To evaluate the efficacy of our model, we utilize Precision-Coverage curves, which offer insights into performance across various species. full visual representation of CROSSNoVO's outstanding performance is depicted in the SrciisinnCoovrraee curve shown in Figure [] The horizontal axis represents coverage, while the vertical axis represents peptide recall. The blue line indicates the performance of Casanovo V2, the orange line represents ContraNovo, and the green line shows the performance of our model. Across all subplots, the green lines are consistently positioned above the blue and orange lines, illustrating the superior performance of our model in peptide recall over varying coverage levels. This consistent outperformance suggests potential for more accurate peptide identification, which could enhance biological insights.
B A.7 Cross-Attention Alignment Analysis
In our extended analysis of the interaction between the Autoregressive Transformer (AT) decoder and the Non-Auuoregressive Transformer (NAT) feature map, we observed consistent and interpretable alignment pattern. Specifically, when the decoder generates token position its cross- attention weights are predominantly concentrated around the corresponding /-th token region in the NAT feature representation. This spatial-tempon correspondence suggests that the AT decoder has learned to selectively focus on the NAT's bidirectional contextual embedding at each generation step. rather than distributing attention uniformly across the entire sequence.
Such behavior provides strong empirical evidence that the NAT decoder functions as a powerful auxiliary encoder, supplying global contextual information that complements the AT's locally condi- tioned generation process. Through this mechanism, the AT decoder effectively integrates the NAT's bidirectional representation as guiding signal, thereby enhancing both token-level consistency and long-range structural coherence in sequence generation.
B.1 Downstream Tasks
Table 6: Comparison of the performance of CROSsNovo and baseline methods on WIgGl-Mouse. The bold font indicates the best performance.
HC Chymotrypsin
LC
Metrics
Methods
Average
AspN
Trypsin
AspN
Casa.
0.714
0.591
0.723
0.668
0.674
Amino Acid Precision
Contra.
0.750
0.612
0.650
0.649
0.665
Ours
0.769
0.640
0.747
0.724
0.720
Casa.
0.557
0.483
0.636
0.456
0.533
Peptide
Contra.
0.649
0.545
0.671
0.519
0.596
Recall
Ours
0.662
0.577
0.699
0,581
0.630
Table 7: Comparison of the performance of CROSsNovo and baseline methods on IgGI-Human Bold values indicate the best performance.
HC
LC
Metrics
Methods
Average
AspN
Chymo.
GluC
LysC
Proteinase
Trypsin
AspN
LysC
Amino
Casa. V2
0.520
0.472
0.605
0.757
0.354
0.759
0.666
0.778
0.642
Acid
Contra.
0.580
0.565
0.642
0.790
0.348
0.787
0.702
0.793
0.676
Precision
Ours
0.613
0.617
0.694
0.814
0.367
0.803
0.719
0.807
0.702
Casa. V2
0.265
0.274
0.399
0.569
0.206
0.595
0.325
0.625
0.446
Peptide
Contra.
0.396
0.372
0.437
0.653
0.274
0.675
0.499
0.646
0.529
Recall
Ours
0.415
0.421
0.512
0.701
0.275
0.699
0.544
0.676
0.560
26

---

Data. The Human IgG1 antibody dataset (IgGI-Human) consists of mass spectrometry data collected using the LTQ Orbitrap instrument. lonization was performed through higher-energy collisional dissociation (HCD), and the resulting peptide fragments were captured at resolution of 17,50M FWHM. The dataset features peptides digested by a variety of proteolytic enzymes, including trypsin, chymotrypsin, asp-N, lys-C, glu-C, and proteinase K. We perform the evaluation donwloaded data with no processing.
The Mouse IgGl antibody dataset
was similarly analyzed using an LTQ Orbitrap mass spectrometer with HCD ionization and the same resolution of 17,500 FWHM. In this dataset, the mouse peptides were digested with trypsin, asp-N, and chymotrypsin to generate a comprehensive proteomic profile. We also used downloaded data for evaluation of all tested models.
Results, As shown in Tables 7 and CrossNovo consistently achieves superior performance across both amino acid-level precision and peptide-level recall when compared to the baseline methods. Specifically, for the Mouse dataset, CrossNovo demonstrates a notable improvement in peptide recall, achieving up 6% increase for the AspN enzyme on the light chain (LC) protein. Similarly, in the Human dataset, the peptide recall improvement is up to 4.5% for the AspN enzyme on the LC protein.
HC-AspN
1/2g
LC-AspN
HC Chymatypsh
6.72
HC-Trypsin
Figure 5: The comparison of performance of models in mouse antibody data.
The performance gain is particularly pronounced for light chain proteins across both species, with CrossNovo showing higher overall precision and recall. The average amino acid precision for the Mouse dataset reaches 0.720, while the peptide recall boosted to 0.630. For the Human dataset, CrossNovo attains an average precision of 0.702, with peptide recall of 0.560, further underscoring its effectiveness over baseline approaches.
These differences in performance are even more apparent in Figure where CrossNovo's en- hancements, particularly on the light chain proteins, can clearly visualized. The consistency in performance improvements across both datasets highlights CrossNovo's 's ability to handle diverse proteolytic enzymes with high accuracy, especially in cases involving the light chains.
B.2 Pust- TasssTranslationll Modification (PTM) Fine-Tuning Evaluation
To further evaluate the generalization ability of CROssNovo and PRIMENOVO, extend experiments to nost-translational modification (PTM) prediction tasks. Each PTM model was fine- tuned independently on the curated 21-PTM dataset, containing spectra annotated with 21 types of peptide modifications. For conciseness, we report results on the first five PTMs in alphabetical order: Acetylation, Biotinylation, Crotonylation, Butyrylation, and Dimethylation.
Analysis. As shown in Table, both models maintain high classification accuracy after PTM-specific fine-tuning. CROssNOVo consistently achieves superior or comparable results across all met- rics, particularly in amino acid and peptide recall for complex modifications like Butyrylation and Dimethylation. This demonstrates the advantage of cross-decoder attention and bidirectional contex- tual transfer in recognizing subtle mass shifts introduced by PTMs. Furthermore, fine-tuning on the
27

---

Table 8: Performance comparison on five representative PTMs after fine-tuning on the 21-PTM dataset. Bold values indicate the best performance for each column.
Metrie
Model
Acetylation
Biotinylation
Crotonylation
Butyrylation
Dimethylation
PrimeNovo
0.98
0.99
0.98
0.97
0.95
Classification Accuracy
CrosNNov
0.98
0.98
0.98
0.98
0.97
AA Recall
PrimeNovo
0.95
0.83
0.90
0.82
CrossNovo
0.96
0.83
0.86
0.85
0.85
PrimeNovo 0.89
0.70
0.79
0.65
0.66
Peptide Recall
CrossNovo
0.90
0.72
0.78
0.69
0.68
diverse 21-PTM dataset highlights the scalability of our architecture to biochemical modifications, extending its applicability beyond canonical peptide sequencing.
B.3 Broader Impact
Advancements in de novo peptide sequencing have far-reaching societal implications, particularly in healthcare, biotechnology. and life sciences. CROSsNOVO significantly enhances the accuracy and robustness of sequence prediction, directly benefiting applications such neoantigen discovery for personalized cancer immunotherapy, rapid characterization of therapeutic antibodies, and profiling of microbial communities in environmental and clinical metaproteomics. By improving the capacity to identify novel or mutated peptides without reliance on existing databases, our method contributes to overcoming current bottlenecks in drug discovery, vaccine development, and discase diagnostics. Moreover, CROssNOvO's ' hybrid architecture- balancing predictive stability with rich contextual understanding enables more reliable deployment in translational pipelines where interpretability and robustness are critical. However, as with any powerful bioinformatic tool, careful consideration must be given to issues of data privacy, equitable access to computational resources, and responsible clinical translation to ensure these technologies are used ethically and inclusively.
28


Reviews:
[{"review_id": "uURitIPx9F", "reviewer": "Reviewer_2Zcz", "summary": "This study addresses the de novo peptide sequencing problem in mass spectrometry-based proteomics, a critical challenge in proteomic research. Prior approaches have relied primarily on autoregressive (AR) or non-autoregressive (NAR) models, which face limitations in capturing bidirectional sequence dependencies (AR) or ensuring generative coherence and scalability (NAR). To overcome these issues, we introduce a hybrid framework integrating AR and NAR paradigms through: (1) A shared spectral encoder coupled with dual decoders (AR and NAR) featuring cross-decoder attention mechanisms to leverage complementary strengths. (2) Importance annealing to dynamically balance contributions from the two decoding strategies and cross-decoder gradient blocking to prevent interference during training.\nThis study provides validation across a 9-species benchmark dataset and antibody-specific peptide datasets, demonstrating significant performance improvements over state-of-the-art methods.", "questions": "1. Previous studies have explored hybrid autoregressive (AR) and non-autoregressive (NAR) decoders for various traditional NLP tasks, such as speech recognition. However, the unique advantages or innovations of this hybrid approach in the specific context of peptide sequencing remain unclear. What distinct features does the proposed method offer to address the challenges inherent to peptide sequencing that set it apart from existing hybrid frameworks?\n2. Extending the analysis to post-translationally modified (PTM) peptides—similar to the work in PrimeNovo (https://d-nb.info/1360284540/34)—would be a valuable addition. Investigating how the method performs on PTM peptides could reveal its applicability to more complex proteomic scenarios.\n3. Attention map and feature vector similarity study would benefit the study.", "limitations": "In summary, the paper’s key strength lies in its presentation of an effective approach that integrates autoregressive (AR) and non-autoregressive (NAR) architectures within a single model, leading to significant performance gains. A notable weakness, however, is the lack of clarity regarding the uniqueness or novelty of this hybrid AR/NAR framework when applied specifically to the peptide sequencing task.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The method undergoes rigorous evaluation and demonstrates significant performance improvements in de novo peptide sequencing tasks.\n2. The study is overall well-written and clearly presented.\n3. This study can benefit the audience in the peptide sequencing domain.\n\nWeakness:\n1. Previously, hybrid autoregressive and non-autoregressive decoder was studied by many other papers for various traditional NLP tasks, including speech recognition. It is not clear what is the uniqueness of the method for peptide sequencing task?\n2. It would be interesting to look into the post-translational modified peptide task as PrimeNovo did (https://d-nb.info/1360284540/34)", "was_revised": true, "final_justification": "The reviewers did a nice job and addressed all my concerns.", "rebuttal": "# **Reviewer Comment:**  \n\nWeakness: \n> *“Previously, hybrid autoregressive and non-autoregressive decoder was studied by many other papers for various traditional NLP tasks, including speech recognition. It is not clear what is the uniqueness of the method for peptide sequencing task?”*\n\nQuestions:\n> What distinct features does the proposed method offer to address the challenges inherent to peptide sequencing that set it apart from existing hybrid frameworks?\n\n\n**Author Response:**  \nWe appreciate the reviewer’s insight and the opportunity to clarify the novelty and motivation of our hybrid design in the context of peptide sequencing. To provide a comprehensive response, we will take a step back to dive into the details of our initial design motivations and rationale below. \n\nIn natural language tasks, information typically flows uni-directionally — for example, in translation, summarization, or dialogue systems, where earlier context is usually sufficient for predicting the next token. This aligns with how humans generate and interpret language sequentially (from left to right), and as a result, **autoregressive (AT)** models like GPT and LLaMA dominate performance benchmarks across NLP tasks. **Non-autoregressive (NAT)** models in NLP, while explored, are predominantly motivated by inference speed, not accuracy, and they generally underperform AT models unless substantial constraints or post-processing are applied (e.g. [Liu et al. 2022] ). \n\nEven when hybrid approaches are proposed (e.g., Mask-Predict or iterative refinement or joint decoder learning in NLP), these often use NAT components to improve efficiency or as auxiliary objectives (e.g., Hao et al., 2020) often with no actual bi-directional information directly being accessed during prediction, as they don't provide additional performance and information gain due to already powerful AT modelling power in natural language as stated above.\n\nBiological sequences, however, differ fundamentally from language. Each token (amino acid) in a peptide does not have a fixed or dominant direction of information flow. Instead, `its identity and functional implications often depend on both the residues front and after`. This is evident from the prevalence of bi-directionally trained protein language models (e.g., ESM-2, ESM-3), which have achieved state-of-the-art performance in many protein-related tasks by leveraging full-sequence context (even generational tasks, where ESM generates accurate protein sequences in given regions using NAT architecture).\n\nIn the**de novo peptide sequencing task**, the NAT model, such as PrimeNovo, also outperforms AT baselines in accuracy as shown by previous study, partially because it enables `full bidirectional context`, **which is naturally aligned with the underlying biological dependencies** in spectral data. However, we found that NAT models face challenges in length generalization, training stability, and scaling to longer sequences due to architectural constraints and reliance on fixed-length decoding (PrimeNovo predefines generation length up to 40, while AT allows much longer in our experiments).\n\nOur contribution is to bridge these complementary strengths __(AT's scalability, training stability and higher performance in certain species like human, NAT's bi-directional features)__ through a novel and *task-specific* hybrid architecture: by introducing a dedicated NAT decoder to learn bidirectional latent features **direction**, and a separate AT decoder that performs sequence generation by attending to these features via a **cross-decoder attention mechanism**. This attention mechanism **directly** integrates the bidirectional NAT representations into the AT decoding steps, enhancing prediction accuracy while retaining the generative flexibility, optimization benefits, and length generalization of AT models. \n\nTo the best of our knowledge, this is the first work to introduce such a direct, modular, and fine-grained information flow from a NAT decoder into an AT decoder for sequence generation. **Unlike prior hybrid approaches in NLP or speech, where NAT modules are often auxiliary or limited to inference-time ensembling and speed improving** (as autoregressive already performs better than non-autoregressive), our model `transfers the bidirectional context directly into AT decoding as a core predictive mechanism` for `performance gain`. We believe this bi-directional information transfer paradigm is particularly beneficial for biological sequences, where no strict left-to-right generational assumption holds.\n\nAs a result , while hybrid AT/NAT ideas exist in NLP,  the motivation and implementation in our work are biologically motivated and architecturally different and novel, tailored to the fundamental differences in peptide sequences compared to natural language. We hope this resolves your concerns with regards to our hybrid design and we will further make this clear in our manuscript during revision!\n\n\n# **Reviewer Comment:**  \n> *“Extending the analysis to post-translationally modified (PTM) peptideswould be a valuable addition. ”*\n\n**Author Response**  \nWe sincerely thank the reviewer for this valuable suggestion. In response, we have extended our analysis to include post-translationally modified (PTM) peptides following the setup described in the PrimeNovo paper.\n\nSpecifically, we augmented our amino acid vocabulary to include additional PTM tokens and fine-tuned our pretrained **CrossNovo** model on the same **21PTM** dataset used in PrimeNovo. Due to the limited rebuttal time, we  finetuning our model on each one of the first five PTMs (in alphabetical order): **Acetylation**, **Biotinylation**, **Crotonylation**, **Butyrylation**, and **Dimethylation**, following settings in PrimeNovo.\n\nThe following table presents side-by-side comparisons between **PrimeNovo** and **CrossNovo** on these PTMs across three metrics: **Classification Accuracy**, **Amino Acid Recall**, and **Peptide Recall**, where PrimeNovo numbers are quoted from their paper and CrossNovo are fine-tuned on each one of the PTM data split using the same setting as PrimeNovo paper.\n\n| **Metric**                  | **Model**   | **Acetylation** | **Biotinylation** | **Crotonylation** | **Butyrylation** | **Dimethylation** |\n|----------------------------|-------------|------------------|-------------------|--------------------|------------------|--------------------|\n| **Classification Accuracy** | PrimeNovo   | 0.98             | **0.99**          | 0.98               | 0.97             | 0.95               |\n|                            | CrossNovo   | **0.98**         | 0.98              | **0.98**           | **0.98**         | **0.97**           |\n| **AA Recall**              | PrimeNovo   | 0.95             | **0.83**          | **0.90**           | 0.82             | 0.84               |\n|                            | CrossNovo   | **0.96**         | **0.83**          | 0.86               | **0.85**         | **0.85**           |\n| **Peptide Recall**         | PrimeNovo   | 0.89             | 0.70              | **0.79**           | 0.65             | 0.66               |\n|                            | CrossNovo   | **0.90**         | **0.72**          | 0.78               | **0.69**         | **0.68**           |\n\nAs shown above, CrossNovo achieves competitive or improved performance in nearly all categories, particularly **Peptide Recall**, which are critical for accurate peptide sequence prediction. \n\nDue to the short rebuttal window we were only able to evaluate 5 of the 21 PTMs (chosen based on the aliphatic order). Still, the consistent performance improvements suggest that our approach can serve as a strong foundation for handling PTM peptides in more complex proteomic scenarios. We plan to extend the evaluation to all 21 PTMs after rebuttal to further strengthen our findings. Thank you again!\n\n# **Reviewer Comment:**  \n> *“Attention map and feature vector similarity study would benefit the study.”*\n\n**Author Response:**  \nWe thank the reviewer for this insightful suggestion.  We performed an analysis of attention patterns in CrossNovo's AT decoder and compared them to those observed in Contra. and Casa. AT decoder.\n\n`While the rebuttal policy prevents us from including figures or external links, we summarize our findings here`:\n\nIn Casa, we found that a significant proportion of attention weight is narrowly concentrated on the first token of the input, similar to what PrimeNovo noticed.  A better behaved attention pattern was noted in ContraNovo with more diversity across positions in input.\n\nIn comparison, our model demonstrates a much more diverse and context-sensitive attention pattern. The attention is distributed across wider range of peak inputs and more often allocated to spectral positions corresponding to peaks that are b- or y-ion fragments. \n\nWe also analyzed the **cross-decoder attention map**, which enables the AT decoder to incorporate contextual knowledge from the NAT decoder. Interestingly, we observed a consistent pattern: when the AT decoder generates a token at position *t*, its attention is predominantly aligned with the rough predictive region of t-th token position in the NAT feature map. This alignment suggests that the AT decoder has learned to focus on the NAT’s bidirectional contextual embedding at the corresponding position, leveraging it as a complementary signal during generation. We believe these observations offer some more insights in the effectiveness of designs and we will further discuss in the revision, thank you again for this!\n\n**We thank the reviewer again for the  constructive feedback. Please feel free to let us know if there are any further questions or concerns!!**\n\n# Reference\n\nLiu et al., (2022, May). Learning Non-Autoregressive Models from Search for Unsupervised Sentence Summarization.\n\n\nHao et al. (2020) Multi-task learning with shared encoder for non-autoregressive machine translation."}, {"review_id": "bXZ1Dse137", "reviewer": "Reviewer_rxjr", "summary": "The author proposed a novel framework combining both autoregressive (AT) and non-autoregressive (NAT) tranformers for de novo peptide sequencing. The two-stage training firstly pretrain both AT and NAT decoder jointly, and then finetune the AT decoder with NAT features to improve the overall performance of the AT model. The proposed CrossNovo model achieves strong performance compared with baselines.", "questions": "1. Related work should probably not include the model proposed in this work (line 92-97).\n1. The problem formulation is clear but can be improved for readers who are not familiar with the domain knowledge. From my understanding, the length of the amino acid sequence $\\mathbf{A}$ to be predicted is unknown, which is why AT is preferred over NAT method for this problem. It would be nice if the authors can mention that.\n1. The authors mention augmenting the AT decoder with prefix and suffix mass information at each generation step (line 140-143). Can the authors elaborate?\n1. Figure 3 and Figure 6: missing x and y axis labels.\n1. Ablation study can be improved. First of all, the author should elaborate more about different ablation configurations. The shared encoder config is not explained clearly in the paper. More importantly, I am specifically curious about one possible configuration: if you train the NAT decoder first with spectrum encoder, and then train the AT decoder from scratch as in the second stage of training (cross attending to both spectrum encoder and NAT decoder features), how would the performance be. To me, the NAT decoder functions more as another “encoder” or “feature extractor” that provides bi-directional features to the AT decoder, hence I’m curious about the ablation of such configuration. \n1. Can authors provide comparison of number of parameters of the proposed CrossNovo model and the baselines (such as CasaNovo V2 and ContraNovo)?", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengh**\n1. Clear explanation of the proposed model architecture. \n1. Strong performance in benchmark compared to comprehensive list of baselines.\n\n**Weakness**\n1. Discussion of baselines is lacking for readers who are not familiar with the specific task.\n1. Ablation study can be improved.\n1. Formatting of manuscript and figures can be improved. \nSee *Questions* section for more details about weakness.", "was_revised": true, "final_justification": "Interesting work that organically combines Autoregressive and Non-Autoregressive generation for specific domain application. Suggest to accept.", "rebuttal": "# **Reviewer Comment:**  \n> *“Discussion of baselines is lacking for readers who are not familiar with the specific task.”*\n\n**Author Response:**  \nWe thank the reviewer for pointing this out. Our original intention was to provide a detailed description of each baseline method in the main Experiments section. However, due to the strict page limit, we condensed the descriptions into a brief summary **(lines 246–255)** to prioritize core model designs.\n\nWe agree that for readers unfamiliar with the de novo peptide sequencing domain, a more comprehensive explanation of the baselines would improve clarity and context. In the revised version of the paper, we will add a detailed comparison and methodological overview of each baseline in the `Appendix`. This will include their architectural differences, design motivations, and key performance characteristics to better situate our our contributions relative to prior work. \n\n# **Reviewer Comment:**  \n> *“The problem formulation is clear but can be improved for readers who are not familiar with the domain knowledge. From my understanding, the length of the amino acid sequence to be predicted is unknown, which is why AT is preferred over NAT method for this problem. It would be nice if the authors can mention that.”*\n\n**Author Response:**  \nWe thank the reviewer for this helpful observation—your understanding is exactly correct.\n\nAs we briefly noted in the paper, a key strength of the AT approach lies in its natural ability to generalize to sequences of arbitrary and unseen lengths. `This is particularly important for de novo peptide sequencing, where the length of the amino acid sequence is not known a priori`. In contrast, **NAT** models, such as PrimeNovo, must predefine a maximum generation length ( 40 tokens in their setting). As a result, they are unable to generate valid sequences longer than this limit, leading to failure cases in important long-peptide scenarios.\n\nAdditionally,` AT models benefit from simpler and more stable optimization during training`. NAT models—especially those trained with CTC or other alignment-based objectives—often suffer from convergence issues and sensitivity to initialization and hyperparameters.\n\nWe will revise the **Introduction** to explicitly state these advantages of AT models, both to better motivate our hybrid design and to improve clarity for readers who may be less familiar with the unique challenges of this domain.\n\n# **Reviewer Comment:**  \n> *“The authors mention augmenting the AT decoder with prefix and suffix mass information at each generation step (lines 140–143). Can the authors elaborate?”*\n\n**Author Response:**  \nThank you for this question—this is an important design detail that we are happy to clarify.\n\nIn de novo peptide sequencing, each input spectrum includes a known **precursor mass**, which represents the total molecular mass of the peptide being decoded. To help the model make more informed predictions and remain within the valid mass constraint, we augment the decoder at each generation step with two auxiliary signals:\n\n- **Prefix mass**: the cumulative mass of all previously predicted amino acids.\n- **Suffix mass**: the remaining mass, calculated as `(precursor mass – prefix mass)`.\n\nProviding this mass information helps the model reason about how much mass budget is left when selecting the next amino acid, reducing the chance of invalid or over-extended predictions.\n\nThese values are continuous and domain-specific, so we encode them using **sinusoidal positional encoding** (adapted to float values) before injecting them as additional inputs at each decoding step. `While detailed implementation is available in our released cod, the motivation is to give the model explicit biochemical context for more accurate and mass-consistent decoding.`\n\nWe will clarify this design choice more explicitly in the revised manuscript for better transparency and understanding!\n\n# **Reviewer Comment:**  \n> *“Figure 3 and Figure 6: missing x and y axis labels.”*\n\n**Author Response:**  \nWe apologize for this oversight. Figures 3 and 6 are **Recall-Coverage curves**, where:\n\n- The **x-axis** represents the **percentage of prediction coverage**, determined by filtering predictions based on confidence scores.\n- The **y-axis** represents the **peptide-level recall** (also referred to as peptide precision in the main text).\n\nWe greatly appreciate the reviewer’s careful reading and attention to detail. We will revise both figures in the final version to clearly include axis labels.\n\n# **Reviewer Comment:**  \n\n> *Ablation study can be improved. First of all, the author should elaborate more about different ablation configurations. The shared encoder config is not explained clearly in the paper. More importantly, I am specifically curious about one possible configuration: if you train the NAT decoder first with spectrum encoder, and then train the AT decoder from scratch as in the second stage of training (cross attending to both spectrum encoder and NAT decoder features), how would the performance be. To me, the NAT decoder functions more as another “encoder” or “feature extractor” that provides bi-directional features to the AT decoder, hence I’m curious about the ablation of such configuration.*\n\n**Author Response:**\nWe thank the reviewer for this insightful and thoughtful comment. We agree that the ablation study could benefit from more elaboration, and we appreciate the opportunity to clarify and expand on our configurations.\n\nIn our current ablation (Table 4), the **“shared encoder”** configuration refers to removing the shared spectrum encoder and instead assigning independent encoders to the AT and NAT branches. This design assumes that AT and NAT decoders might focus on different aspects of the input spectrum and thus benefit from specialized encodings. In this setup, the only pathway for information exchange between NAT and AT is the **cross-decoder attention** mechanism. Notice that, `when both the shared encoder and cross-decoder connection are removed, the two decoders become completely disconnected`, which does not entail any row in Table 4 (i.e. either Cross-Decoder or Shared-Encoder needed to be included  .\n\nWe also thank the reviewer for the **very interesting suggestion** regarding a staged training scheme where the NAT decoder is trained fully first as a feature extractor, and then the AT decoder is trained from scratch by cross-attending to both the fixed spectrum encoder and NAT features. We share the reviewer's view that the NAT decoder in our framework effectively acts as an auxiliary information encoding unit, providing enriched bi-directional representations of potential sequence. This configuration could lead to improved stability or representation quality, especially if the NAT branch is allowed to mature independently before the AT training begins.\n\nDue to the substantial computational demands of training de novo sequencing models (our training involves over 30 million spectra and requires more than **1,600 A100 GPU hours** for each configuration to be trained), we initially prioritized the most structurally distinct configurations for the ablation in Table 4. Some of these, like Row 2, reflect early-stage development result. But highly expensive computational cost prevents us to try all configs. \n\nThat said, we fully agree with the reviewer’s intuition, and the proposed experiment aligns well with our training design. We can simulate this configuration by modifying the **importance annealing schedule** in our paper: instead of gradually decreasing the weight of the NAT loss (λ), we would fix the NAT loss at 100% for the initial phase (say first 50 epoch), then gradually introduce the AT loss later—effectively staging the training in the way the reviewer suggests.\n\nWe have started preparations to run this variant and will include the results in the final version of the paper once it's finished. We again thank the reviewer for this valuable suggestion!\n\n# **Reviewer Comment:**  \n> *Can authors provide comparison of number of parameters of the proposed CrossNovo model and the baselines (such as CasaNovo V2 and ContraNovo)?*\n\n**Author Response**:\nWe thank the reviewer for raising this important point. Below we provide a comparison of model sizes between our proposed model and the main baselines:\n\n- **CrossNovo (AT only)**: 33 million parameters  \n- **CROSSNOVO (AT + NAT total)**: 45 million parameters  \n  - 9 Transformer layers per decoder, 400 hidden dimensions  \n  - `NAT decoder serves as an auxiliary module for contextual feature extraction during training and cross-attention`\n\nFor comparison:\n- **PrimeNovo (NAT)**: 33 million parameters  \n  - Same number of layers and hidden dimensions as our NAT component\n- **Casanovo V2 (AT)**: ~38 million parameters  \n  - 9 layers, 512 hidden dimensions  \n- **ContraNovo (AT)**: ~43 million parameters  \n  - 9 layers, 512 hidden dimensions, plus an additional contrastive module\n\n\n\nWe hope this comparison provides the information the reviewer needs, we will make this further clear in the revision!\n\n\n**We sincerely thank the reviewer for the thoughtful and detailed feedback, and we are happy to address any further questions if you have!**"}, {"review_id": "vsIltoERXo", "reviewer": "Reviewer_nsFd", "summary": "This paper proposes a novel hybrid architecture for de novo peptide sequencing, which innovatively combines the advantages of autoregressive and non-autoregressive models. The core contribution of the paper lies in the design of a dual decoder architecture. Experiments have shown that it outperforms existing methods significantly on 9 species benchmark tests.", "questions": "1. How sensitive are the parameters?\n\n2. Will the proposed dual decoder architecture significantly increase computational costs? Can you provide a comparison of inference time with existing methods?\n\n3. Can CROSSNOVO's hybrid architecture design principles be extended to other sequence modeling tasks, such as RNA sequence analysis?\n\n4. Can the author analyze the performance of CROSSNOVO on peptide sequences of different lengths?", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. The paper clearly explains the limitations of AT and NAT models in de novo peptide sequencing.\n\n2. The model architecture, training strategy, and knowledge transfer mechanism are described in detail and organized clearly.\n\n3. The validation of this method in downstream tasks such as antibody sequencing has demonstrated its potential for practical biological research.\n\nWeaknesses:\n\n1. The paper did not fully explore the impact of model size on performance, as well as the sensitivity analysis of different parameter settings.\n\n2. Although the experimental results are impressive, a theoretical analysis can be conducted on why NAT to AT knowledge flow is more effective than two-way.\n\n3. The proposed method seems to be a universal architecture. Although models have advantages in solving de novo peptide sequencing, they should also be compared to similar general architectures.\n\n4. If the writing of the paper is corrected again, it will bring a good reading experience. For example, the first reference is duplicated with the second one.\n\n5. Although quantitative ablation experiments are conducted, qualitative results are more convincing.", "was_revised": true, "final_justification": "Some of my issues have not been resolved, although the author claims that policies do not allow it. Because it is not allowed to add visual results (although numerical results can also be provided, the author claims that it is not allowed). I have also read the feedback from other reviewers. I hope the authors will include the experimental results they promised to supplement in the revised version.", "rebuttal": "# **Reviewer Comment:**  \n> *The paper did not fully explore the impact of model size on performance, as well as the sensitivity analysis of different parameter settings.*\n\n> How sensitive are the parameters?\n\n**Author Rebuttal:**  \nWe appreciate the reviewer’s comment and fully agree that analyzing the sensitivity of model size and other parameters is valuable. However, in our case, the training cost is a significant constraint for performing parameter sweep on the training parameters.\n\nTraining de novo model is extremely computationally expensive, primarily due to:\n1. **Long input sequences** (hundreds of spectral peaks) processed by the Transformer.\n2. **The large-scale training dataset** (MassIVE-KB), which contains over 30 million spectrum-peptide pairs.\n\nEach full training run requires approximately **8–10 days on 8 A100 GPUs** (over **1600 GPU hours**)—assuming no interruptions. Importantly, this is consistent with the compute requirements for baseline models like **ContraNovo**, **Casanovo**, and **PrimeNovo**, which operate on the same data and task setup.\n\nAs a result, it is extremely challenging—not only for our model but also for baseline methods—to conduct extensive grid searches or sensitivity analyses on parameters like **model size** or **training hyperparameters** (e.g., importance annealing). Due to GPU constraints, we prioritized experiments that offer the most insight, focusing on **ablation studies** (Table 4) and **model tuning** to produce a competitive version of CrossNovo, while keeping the model size aligned with baselines for fair comparison.\n\n\nWe did conduct `parameter sensitivity analysis at inference time`, where computational demands are `significantly lower`. As shown in **Table 5**, we analyzed the effect of different beam sizes on both amino acid and peptide-level metrics, and reported those results in detail.\n\n\n# **Reviewer Comment:**  \n> * a theoretical analysis can be conducted on why NAT to AT knowledge flow is more effective than two-way.*\n\n**Author Response:**  \nWe thank the reviewer for this thoughtful suggestion. The decision to adopt a unidirectional knowledge transfer—from the Non-Autoregressive Transformer (NAT) decoder to the Autoregressive Transformer (AT) decoder— is grounded in both theoretical considerations and practical constraints.\n\nFormally, let **E_spec** represent the shared spectrum encoder output, **V_nat** the bidirectional contextual features learned by the NAT decoder, and **H_at** the hidden states produced by the AT decoder. During generation, the AT decoder conditions on previously generated tokens **a_t** and attends to both **E_spec** and **V_nat**, yielding representations of the form:\n\n$\nH\\_{at}^t = \\text{TransformerAT}(a_{<t}, E_{spec}, V_{nat})\n$\n\nThis one-way information flow reflects our architectural philosophy, in which the `NAT decoder serves as a global feature extractor, capable of leveraging full bidirectional context to model peptide structure from spectral inputs`. In contrast, the AT decoder ensures stable, autoregressive generation while benefiting from enriched contextual guidance provided by the NAT decoder.\n\nAllowing a two-way flow—where the NAT decoder also attends to AT decoder states—`introduces  theoretical information leaking risk` (as we briefly stated in the manuscript). In particular, `since the AT decoder is trained with teacher forcing, its intermediate states can directly encode ground-truth information`. Feeding these representations into the NAT decoder risks information leakage, violating the independence assumptions of the NAT objective (e.g., CTC loss) and compromising training fairness (as part of the ground truth can be seen during attention).\n\nAdditionally, our use of cross-decoder **gradient blocking** ensures that **V_nat** `remains fixed` and `uninfluenced by gradients from the AT loss`, preserving the semantic integrity of NAT-derived features and maintaining a clean modular separation of learning objectives.  Such a configuration introduces `a co-dependent optimization loop, where both objectives are defined as conditional on each other`: i.e., minimizing both **L(AT | NAT)** and **L(NAT | AT)** without a fixed reference. This mutual conditioning can lead to an **ill-posed learning problem**, as there is no stable optimization target. In practice, this may cause representational drift or collapse \n\nFrom both a theoretical and empirical standpoint, we have observed that this unidirectional NAT → AT information flow results in stronger performance and more stable training dynamics and we will include a clearer discussion to the revision!\n\n**Reviewer Comment:**  \n> *The proposed method seems to be a universal architecture. . .. they should also be compared to similar general architectures.  Can CROSSNOVO's hybrid architecture design principles be extended to other sequence modeling tasks*\n\n\n**Author Rebuttal:**  \nWe thank the reviewer for this insightful and forward-looking comment. We agree that our proposed architecture—particularly the principle of leveraging bidirectional representations to enhance unidirectional generation—has potential applicability beyond de novo peptide sequencing. However, we would like to clarify the current  limitations of our work.\n\nOur research team has primarily focused on the peptide sequencing problem space. Expanding to other domains such as DNA or RNA sequence modeling, although highly relevant, would require a significant investment in domain-specific study, replication of prior benchmarks, evaluation pipelines, and experimental design. This mirrors what is commonly seen in natural language processing (NLP) or computer vision (CV) research groups, where deep domain specialization is often necessary to make rigorous and impactful contributions. Each biological sequence modeling problem—be it genomic variant calling, RNA folding, or DNA language modeling—comes with its own datasets, preprocessing strategies, evaluation metrics, and biological assumptions. At this stage, we do not yet possess the necessary expertise or infrastructure to explore these other biological tasks with the level of rigor we aim to maintain in our work.\n\nWe appreciate the reviewer’s suggestion and are encouraged that our framework is recognized as potentially general. Indeed, the **core architectural principle** behind CrossNovo—**using bidirectional context (via NAT) to enhance autoregressive generation (via AT)**—may actually be **better suited to biological sequence modeling** than to NLP tasks. Unlike natural language, where autoregressive models such as GPT or LLaMA often perform well due to strong left-to-right dependency structures, **biological sequences (e.g., proteins, RNAs)** exhibit **functional and structural dependencies that are inherently bidirectional**. The biological role of a single residue or base often depends on both upstream and downstream context. \n\nWhile we have limited our evaluation to peptide sequencing due to our domain expertise, we strongly believe that this architectural framework—particularly the asymmetric NAT-to-AT design—has broader applicability to biological sequence generation tasks. We are excited by this direction and hope future work will extend the approach.\n\n# **Reviewer Comment:**  \n> *If the writing of the paper is corrected again, it will bring a good reading experience. For example, the first reference is duplicated with the second one.*\n\n**Author Rebuttal:**  \nWe sincerely thank the reviewer for pointing this out. We apologize for the duplicated reference entry. We will carefully revise the bibliography to eliminate redundancies  and to work on the clarity of the paper.\n\nAs there was some of a time crunch during submission, we will take this opportunity to conduct a thorough proofreading of the manuscript to improve clarity, consistency, and overall readability. \n\n**Author Rebuttal:**  \nAs also requested by Reviewer2Zcz, we conducted additional Qualitative analyses on the `cross-decoder attention maps` and feature vector similarities to support the effectiveness of our design. Specifically, we analyzed how the AT decoder attends to the NAT-derived representations during sequence generation.\n\nInterestingly, we observed a consistent and interpretable pattern: when the AT decoder generates a token at position *t*, its cross-attention weights are predominantly aligned with the region around the *t*-th token position in the NAT feature map. This alignment suggests that the AT decoder has learned to selectively attend to the NAT’s **bidirectional contextual embedding** at the corresponding position, effectively leveraging it as a complementary signal to guide its autoregressive prediction. These findings support the hypothesis that the NAT decoder serves as a powerful auxiliary encoder, enriching the generation process with global sequence context. `We were unable to include full qualitative examples in this response due to policy of images inclusion`. However, we will add these to our revisions and thanks again for the comment!\n\n\n**Reviewer Comment:**  \n*Can the author analyze the performance of CROSSNOVO on peptide sequences of different lengths?*\n\n**Author Rebuttal:**  \nThank you for the suggestion. We further analyzed peptide recall across sequence lengths from 5 to 45 on the benchmark test set. As we can not include the `line graphs` here due to policy, we summarize findings:\n\nFor `short peptides` (<12),  `CrossNovo (Hybrid)` and `PrimeNovo (NAT)` perform similarly and outperform `ContraNovo (AT)`.\n In the `mid-length range` (13–25), ContraNovo (AT) catches up to PrimeNovo (NAT), but `CrossNovo  maintains a consistent ~2% higher recall`. For `long peptides`(>30), PrimeNovo’s performance drops, while both `CrossNovo` and `ContraNovo` handle longer sequences better, with **CrossNovo performing best overall**. We will include this line graph in our final version!\n\n\n**Thank you again for your effort and let us know if there's further questions!**"}, {"review_id": "56vIWf2IC1", "reviewer": "Reviewer_7Vk1", "summary": "The authors introduce **CROSSNOVO**, a hybrid framework designed for de novo peptide sequencing from mass spectrometry data. Traditional autoregressive (AT) models capture strong sequential dependencies but miss important global bidirectional context, whereas non-autoregressive transformers (NATs) capture holistic context but face optimization challenges. CROSSNOVO combines these two paradigms by using a shared encoder and two decoders: an NAT decoder that generates rich, bidirectional context features, and an AT decoder that leverages these features via a novel cross-decoder attention module to improve sequential prediction. To stabilize training, the authors introduce a tailored multitask loss strategy incorporating importance annealing and gradient blocking. Empirical evaluations on comprehensive 9-species peptide sequencing benchmarks demonstrate significant improvements over both autoregressive and non-autoregressive baselines, validating CROSSNOVO's capability to integrate the strengths of both modeling paradigms.", "questions": "1. The cross-decoder attention mechanism (Eq. 6, page 5) is intriguing, but its formulation isn't super clear to me. Specifically, how exactly are the positional encodings for the concatenation of NAT-derived features \\( V^{(L')}_{p\\{1:T_{max}\\}} \\) and original spectral embeddings \\( E^{(b)}_{p\\{T_{max}+1:T_{max}+k\\}} \\) computed and differentiated mathematically? It would greatly strengthen the paper if you explicitly clarified the mathematical or algorithmic steps used to distinguish these positional encodings, perhaps providing explicit equations or pseudocode in the appendix.\n\n2. Your importance annealing schedule (Eq. 3, page 5) linearly increases the weighting coefficient \\( \\lambda_{AT}(i) = \\frac{i}{T_{total}} \\). I wonder why you specifically chose a linear schedule over alternatives like exponential decay, sigmoid, or stepwise annealing. Could you provide either an empirical sensitivity analysis comparing these alternative schedules or a theoretical justification explaining why a linear schedule is optimal or preferable for this training strategy?\n\n3. Your Precise Mass Control (PMC) decoding algorithm (detailed in Appendix A.3) is quite sophisticated. However, it's unclear exactly how PMC decoding integrates mathematically with the cross-decoder attention mechanism during inference. Could you provide a more rigorous mathematical integration or pseudocode explicitly detailing the interaction between PMC decoding and cross-decoder attention during sequence inference? \n\n5. Considering the computational complexity of your dual-decoder architecture and PMC decoding, explicit computational benchmarking is missing. Could you provide runtime benchmarks, inference memory usage, and scalability comparisons against baseline methods? \n\nAddressing these concerns will help me move my decision to a solid accept.", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n- The authors propose a sophisticated and mathematically rigorous framework, CROSSNOVO, that effectively integrates autoregressive (AT) and non-autoregressive (NAT) approaches. The mathematical formulations, especially the cross-decoder attention mechanism (Eq. 6, Section 3.4) and Connectionist Temporal Classification (CTC) loss derivations (Eqns. 5, 8–15 in Appendix), are detailed, clearly explained, and well-justified. I think this is a strong innovation.\n- The joint training strategy with importance annealing (Eq. 3, Section 3.3) and cross-decoder gradient blocking (Eq. 7, Section 3.4) is really cool and clearly justified. The authors thoroughly validate these choices with rigorous ablation experiments (Table 4 on page 9).\n- The empirical validation is super strong and extensive, evaluating on challenging, multi-species benchmarks (9-species-v1 and v2, Tables 1 and 2), and explicitly demonstrating superiority over well-established baselines (CasanovoV2, ContraNovo). The rigorous downstream evaluation on antibody sequencing tasks (Figures 5 and 7, Tables 6 and 7) further emphasizes practical significance.\n- The authors present detailed evaluations of hyperparameter sensitivity (beam size analysis in Table 3) and model generalization to biologically important cases (amino acids with similar masses, Figure 4). These analyses demonstrate thoughtful, thorough experimental design and validation.\n\n**Weaknesses:**\n- While the paper provides a detailed mathematical exposition, for me, some critical aspects of the cross-decoder attention mechanism (Eq. 6) could benefit from additional clarity/info. Specifically, I wonder how the distinct positional encodings are exactly defined or computed for combining NAT decoder representations and original spectral features. A more explicit mathematical formulation or intuitive explanation would significantly enhance understanding.\n- The choice and sensitivity of the importance annealing schedule (Eq. 3, λ_AT parameter) are only briefly discussed. It would greatly improve the clarity and robustness of the training strategy if the authors explicitly provided a sensitivity analysis or mathematical rationale for the chosen linear schedule. Why not alternative schedules such as exponential or piecewise-linear annealing?\n- The dynamic programming-based PMC decoding described extensively in the appendix (Section A.3) appears critical for practical performance. However, I was confused why the interplay between PMC decoding and cross-decoder attention is not clearly defined mathematically. Clarifying this interaction explicitly (perhaps via an integrated algorithm or explicit joint mathematical formulation?) would significantly improve methodological rigor and reproducibility.\n\nIn summary, while CROSSNOVO demonstrates strong methodological rigor, empirical validation, and clear practical significance, I would help to explicitly address the detailed mathematical clarifications, rigorous comparison with key peptide-generation baselines, and thorough computational efficiency analysis would greatly enhance the overall stre", "was_revised": true, "final_justification": "The authors did a great job explaining the formulations and responding to my comments. I thank them for explaining the scheduler choice, but I hope the authors can provide the runtime benchmarks, inference memory usage, and scalability comparisons against baseline methods. The same goes for more mathematical justifications for their formulations. I will maintain my score of 4 because I think the overall paper is strong, but the lack of these benchmarks and rigorous justifications prevents me from increasing my score.", "rebuttal": "# **Reviewer Comment:**  \n> * I wonder how the distinct positional encodings are exactly defined or computed for combining NAT decoder representations and original spectral features*\n\n**Author Response:**  \nWe thank the reviewer for the helpful observation and apologize for not providing sufficient detail in the original submission. We will revise the manuscript to make this point clearer.\n\nTo clarify, each component of our model—NAT decoder, AT decoder, and spectrum encoder—uses its **own independent sinusoidal positional encoding**, ensuring there is no ambiguity when representations are combined or attended together.\n\nFor a position index $p$ and dimension index $i$, the sinusoidal encoding is defined as:\n\n$\\text{PE}(p, 2i) = \\sin\\left(\\frac{p}{10000^{2i/d}}\\right), \\quad \\text{PE}(p, 2i+1) = \\cos\\left(\\frac{p}{10000^{2i/d}}\\right)$\n\nwhere $d$ is the hidden dimension. This encoding is applied separately to `each` of the following:\n\n- **NAT decoder**: Positions 1 to 40 are used, as we follow the same convention as PrimeNovo with a maximum length of 40.\n- **AT decoder**: Uses the same sinusoidal function, applied independently from position 1 to $n$, where $n$ is the peptide length at decoding time.\n- **Spectrum encoder**: To enable joint attention over both NAT decoder outputs and spectrum features (as described in Eq. 6), we assign **non-overlapping positional indices**. Specifically, we start positional indices for spectrum input from 41 onwards. That is, the first peak receives positional encoding for index 41, the second peak gets 42, and so on, up to $K$ (the number of peaks). This ensures that the positional encodings for spectrum do not conflict with those of NAT (1–40).\n\nDuring cross-decoder attention, We form the attention input by concatenating the NAT decoder features with positional encodings from indices $1$ to $40$, and the spectrum features with positional encodings from $41$ to $40 + K$, where $K$ is the number of spectrum peaks.\n\nIn summary, we design the positional encodings so that NAT, AT, and Encoder operate with distinct index spaces, preserving clear token identity and preventing overlap when their outputs are merged.\n\nAs a final note, we have begun exploring the use of **rotary positional embeddings (RoPE)** for the spectrum encoder, which are better suited for `capturing relative distance among peaks`. Preliminary results suggest RoPE improves modeling of local ion relationships, though this investigation is ongoing and `beyond the scope of the current work`.\n\nWe thank the reviewer again for highlighting this important detail, and we will revise the paper to include both mathematical clarification and an intuitive explanation.\n\n# **Reviewer Comment:**  \n> *The cross-decoder attention mechanism (Eq. 6, page 5) is intriguing, but its formulation isn't super clear to me. Specifically, how.... *\n\n**Author Rebuttal:**  \nWe thank the reviewer for raising this important question. The formulation in Eq. 6 involves the concatenation of two distinct feature sets—NAT-derived decoder outputs and the encoded spectrum peaks—which are fed into the cross-decoder attention mechanism. To avoid positional ambiguity between these sources, we assign **non-overlapping and distinct sinusoidal positional encodings** to each as stated above.\n\nTo clarify the design:\n\n1. The NAT decoder output consists of $T_{\\text{max}}$  (40 here) peptide positions. These are assigned sinusoidal positional encodings from index $1$ to $T_{\\text{max}}$ using the standard formulation:\n\n$\\text{PE}(p, 2i) = \\sin\\left(\\frac{p}{10000^{2i/d}}\\right), \\quad \\text{PE}(p, 2i+1) = \\cos\\left(\\frac{p}{10000^{2i/d}}\\right)$\nwhere $p$ is the position index (ranging from $1$ to $T_{\\text{max}}$), $i$ is the embedding dimension index, and $d$ is the hidden size.\n\n2. The spectrum features consist of $k$ peaks. To avoid overlap with NAT positions, we assign these peaks positional indices starting from $T_{\\text{max}} (40)+ 1$ to $T_{\\text{max}} + k$. Each peak is then encoded with:\n$\\text{PE}(p + T_{\\text{max}}, i) \\quad \\text{for} \\quad p = 1 \\dots k$\n\n\nWe appreciate the reviewer’s suggestion and will add these mathematical steps (along with optional pseudocode) to the appendix in the camera-ready version to improve clarity and reproducibility.\n\n# **Reviewer Comment:**  \n> * I wonder why you specifically chose a linear schedule over alternatives like exponential decay, sigmoid, or stepwise annealing.*\n\n**Author Rebuttal:**  \nWe thank the reviewer for this excellent question. Indeed, we initially considered a variety of annealing strategies, including exponential and stepwise schedules. However, conducting a comprehensive sensitivity analysis across all such variants would have required retraining the entire model multiple times—`each run involving 30 million spectra and taking over 1600 A100 GPU-hours (8A100 for 8-10 days)`. Given the computational cost, we prioritized resource allocation toward architectural ablation studies (Table 4), which we believed would yield more actionable insights for model design.\n\nWe chose the **linear annealing schedule** as a practical and empirically grounded option, drawing inspiration from prior work in multitask training for non-autoregressive auxiliary tasks (e.g., Hao et al., 2021, on NAT-augmented machine translation), where linear schedules have shown strong performance and stable behavior. The intuition is that it provides a **gradual and interpretable shift** in training focus from the NAT objective to the AT objective without sudden gradient shocks, which is particularly helpful in our hybrid training setup.\n\nThat said, we completely agree this is a promising area for deeper exploration. If time and computational resources permit, we would be very interested in performing a detailed comparison of alternative annealing strategies in future work.\n\n# **Reviewer Comment:**  \n> *The dynamic programming-based PMC decoding described extensively in the appendix (Section A.3) appears critical for practical performance. However, I was confused why the interplay between PMC decoding and cross-decoder attention is not clearly defined mathematically .*\n\n**Author Rebuttal:**  \nWe appreciate the reviewer’s careful reading and thoughtful suggestion. To clarify, the **PMC decoding** algorithm described in Appendix A.3 is included as a **reference decoding method for the NAT decoder only**, and `it does not interact directly` with the **cross-decoder attention mechanism** or the main AT decoding pathway described in the core of the paper.\n\nThe motivation for including PMC decoding is to offer a practical utility: after joint training, the `NAT decoder learns a strong bidirectional representation that can be decoded independently`. For users `who wish to leverage this NAT decoder directly` (without AT), the PMC decoding provides a viable alternative, especially in this task where exact precursor mass compliance is necessary.\n\nPMC decoding itself is based on a **dynamic programming solution to a variant of the knapsack problem**. Given a total mass constraint (precursor mass), `each amino acid token is treated as an item with a \"weight\" (its molecular mass) and a \"value\" (its predicted probability from the NAT output).`  The decoding algorithm aims to **maximize the total sequence likelihood while satisfying the exact mass constraint**, like picking items to fit in a knapsack to maximize the values of the bag. This method exploits the fact that NAT outputs a full distribution over all positions at once, allowing flexible sequence assembly under constraints—something that autoregressive (AT) models cannot easily accommodate due to their stepwise nature.\n\nBecause PMC decoding applies **only to the NAT branch**, and does **not feed back into or affect the cross-decoder attention module or AT generation**, we chose to place this utility in the appendix and `released codebase` for optional use. The **core contribution of our work** remains focused on how **cross-decoder attention enables the AT decoder to access NAT-derived bidirectional features**, rather than on decoding techniques themselves.\n\nWe acknowledge the reviewer’s suggestion and will make this dependency separation more explicit in the text, and clarify that PMC decoding is a standalone utility for users interested in NAT-only prediction.\n\n\n# **Reviewer Comment:**  \n> *Could you provide runtime benchmarks, inference memory usage, and scalability comparisons against baseline methods?*\n\n**Author Rebuttal:**  \nWe thank the reviewer for this important point. In terms of model size and memory usage, our architecture remains **comparable to baseline models**. Specifically, our AT decoder contains 33 million parameters, and with the NAT decoder included, the total model size is approximately 45 million parameters. This is similar in scale to PrimeNovo (33M), Casanovo V2 (38M), and ContraNovo (43M), and `we observe no significant difference in peak GPU memory usage during inference`.\n\nIn terms of **inference time**, we do observe a moderate overhead due to the **additional cross-decoder attention**. Empirically, **inference latency increases by approximately 23%–30%** compared to standard AT-only models, with variation depending on peptide length. `This overhead arises because each decoding step in the AT decoder now attends not only to its own past states but also to the full set of NAT-derived representations` and encoded spectrum features. Noted that PMC is only used if user wants results from NAT-decoder, which is not activated usually therefore does not incur additional time.\n\n\n\nWe will add these runtime statistics to the final version of the paper to improve transparency and reproducibility!\n\n\n# Reference \n\nHao et al. (2020) Multi-task learning with shared encoder for non-autoregressive machine translation.\n\n\n**Thank you again for the thoughtful suggestion and we are happy to answer any further questions!**"}]

Author rebuttal:

Meta-review: The paper presents CrossNovo, a hybrid framework for de novo peptide sequencing that integrates the strengths of autoregressive (AT) and non-autoregressive (NAT) models. The approach employs a shared spectrum encoder with dual decoders, where the NAT decoder captures bidirectional context and the AT decoder incorporates these features through cross-decoder attention, supported by importance annealing and gradient blocking. Reviewers consistently highlighted the strong empirical performance, the clarity of the architectural design, and the significance of the contributions. Concerns raised regarding missing baselines, prior work on hybrid models in NLP and speech, and computational cost were largely addressed during the rebuttal. Overall, this is a solid and impactful contribution, and I recommend acceptance.
Decision: Accept (poster)