Neural Graph Matching Improves Retrieval Augmented Generation in Molecular Machine Learning
Authors: Runzhong Wang
Rui-Xi Wang
Mrunali Manjrekar
Connor W. Coley
Venue: ICML 2025 poster (2025)
Keywords: Retrieval Augmented Generation
Graph Matching
Mass Spectrometry
AI for Science
Primary area: applications->chemistry_physics_and_earth_sciences
TL;DR: We develop a retrieval-augmented generation approach with neural graph matching for mass spectrum prediction, achieving a 45% relative improvement in top-1 retrieval accuracy.

Abstract: Molecular machine learning has gained popularity with the advancements of geometric deep learning. In parallel, retrieval-augmented generation has become a principled approach commonly used with language models. However, the optimal integration of retrieval augmentation into molecular machine learning remains unclear. Graph neural networks stand to benefit from clever matching to understand the structural alignment of retrieved molecules to a query molecule. Neural graph matching offers a compelling solution by explicitly modeling node and edge affinities between two structural graphs while employing a noise-robust, end-to-end neural network to learn affinity metrics. We apply this approach to mass spectrum simulation and introduce MARASON, a novel model that incorporates neural graph matching to enhance a fragmentation-based neural network. Experimental results highlight the effectiveness of our design, with MARASON achieving 27% top-1 accuracy, a substantial improvement over the non-retrieval state-of-the-art accuracy of 19%. Moreover, MARASON outperforms both naive retrieval-augmented generation methods and traditional graph matching approaches. Code is publicly available at https://github.com/coleygroup/ms-pred.

Neural Graph Matching Improves Retrieval Augmented Generation in Molecular Machine Learning
Runzhong Wang" !
Rui-Xi Wang"
Mrunali Manjrekar' Connor W. Coley'
Abstract
reliable models is prominent, retrieval-augmented genera- tion has achieved significant success. Recent progress in small-molecule research further highlights the potential of siriivallaugmentee generation, which is also the main focus of this paper.' This approach has been demonstrated to en- hance the accuracy and robustness of various molecular ma- chine learning applications, including gruucturebbaed drug design (Zhang et al., 2024; Huang et al., 2024), fragment- based drug discovery (Lee et al., 2024), and monomer de- sign for advanced materials (Buchler, 2024).
Molecular machine learning has gained popularity with the advancements of geometric deep learn- ing. In parallel, retrieval-augmeentdd generation has become a principled approach commonly used with language models. However, the optimal inte- gration of retrieval augmentation into molecular machine learning remains unclear. Graph neu- ral networks stand to benefit from clever match- ing to understand the structural alignment of re- trieved molecules to a query molecule. Neural graph matching offers a compelling solution by explicitly modeling node and edge affinities be- tween two structural graphs while employing a noise-robust, end-to-end neural network to learn affinity metrics. We apply this approach to mass spectrum simulation and introduce MARASON, a novel model that incorporates neural graph matching to enhance a fragmentation-bart based neu- ral network. Experimental results highlight the effectiveness of our design, with MARASON achieving 28% top-1 accuracy, a substantial im- provement over the non-retrieval state-of-the-art accuracy of 19%. Moreover, MARASON out- performs both naive retrieval-augmenter genera- tion methods and traditional graph matching ap- proaches. Code publicly available at ht tps / //github....////// com/ http///lll/l/sssppppeeeeeoee
We assume a database exists with pairs of molecular struc- tures and their properties of interest. For a new structure of interest, references in the database can be straightforwardly retrieved with cheminformatic tools such molecular fin- gerprints (Morgan, 1965) and Tanimoto similarity (Bajusz et al., 2015). Domain experts benefit from such features in databases by searching for pairs of structures to corre- late differences in structures with differences in properties, even leading to entire subfields like matched molecular pair analysis (Griffen al., 2011). Therefore, we expect that augmenting structure-property relationship model with structurally similar molecules and their associated proper- ties should improve the accuracy of predictions.
However, designing an effective neural network architecture for retrieval augmentation presents significant challenges. Simple approaches, such as direct concatenation of refer- ence information the input of the neural network, often yield minimal pothesize that this is due to models' inabilities to adequately address differences between the retrieved references and the target molecule. We propose that employing atom-level or fragment- matching mechanisms in an end-to-end network would allow for reference information to be used more effectively.
improvements (see Table 3). We hy-
1. Introduction
Enhancing neural networks with task-relevant factual knowl- edge has shown great promise in advancing knowledge- intensive applications, a technique widely recognized as strrival-auugeenter generation (RAG) (Lewis al., 2020). In scientific domains, where the demand for accurate and
Graph matching, which addresses node-level correspon- dences across multiple graphs, naturally emerges as suit- able approach to this challenge. It explicitly incorporates structural matching by formulating both node-wise and edge-wise graph affinity scores into the quadratic assign-
Equal contribution 'Massachusetts Institute of Technology, Cambridge, MA, United States. Correspondence to: Connor W. Coley <ccoley @ mit.edu'>
Proceedings of the ndnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
I"Small molecules" are defined in this work by a mass upper limit of 1,500 Da, enforced in all experiments.

---

Neural Graph Matching Improves Retrieval Augmented Generation in Molecular Machine Learning
matched fragments by traditional graph matching
"metabolite dark matter" (Bittremieux et al., 2022).
Our contributions are:
1) We present an effective design strategy for retrieval- augmented generation in molecular machine learning with neural graph matching. Graph matching techniques in- corporate both node-wise and edge-wise affinities explic- itly, providing a principled way of assigning node-to-node or fragment-to tooeenttt--rggmettt alignment between the reference structure and the target structure² that emulates how domain experts leverage retrieved references. By further introducing learning- basend affinity metrics and differentiable matching layers, we improve the overall efficacy of matching and the resulting accuracy in applications.
matched fragments by neural graph matching
Figure 1. Comparison of graph matching results between tradi- tional algorithm (Cho et al., 2010) and the neural graph matching module in this paper. Neural graph matching is more robust es- pecially when the matched structures are not identical. More visualization results can be found in Fig. 6, 7 in Appendix.
ment problem (Lawler, 1963). In molecular tasks, graph matching recapitulates what domain experts might do when comparing two molecules. In practice, traditional graph matching methods typically define the graph affinity us- ing pre-established metrics, such as Gaussian kernels based on Euclidean distance. Such predefined affinity metric has significant limitation: its limited expressivity makes traditional graph matching methods vulnerable to random noise and not robust to possible ambiguities in retrieved structures (Wang et al., 2022). To address this robustness challenge, new class of neural graph matching methods has emerged, which learns the affinity metric and the solver module in an end-to-end manner (Zanfir & Sminchisescu, 2018; Wang et al., 2020; Li et al., 2019). As result, neu- ral graph matching provides a compelling design choice for retrieval augmentation, as shown in Fig. 1. Chemists are able to draw analogies between molecules beyond sim- ple maximum common substructure analysis and consider, for example, the relationship between non non-equivalent but functionally similar (i.e., isosteric) functional groups. By integrating this eeeemistry-inspire view of graph matching with the expressivity and adaptability of neural networks, neural graph matching represents an effective framework for aligning target structures with, and thus learning from, their retrieved counterparts.
2) We present matching-aware getrieval augmented spectrometry oracle with neural networks (MARASON), an implementation of neural graph matching- retrieval- augmented generator for mass spectra, as shown in Fig. 2. MARASON is built upon the ICEBERG (Goldman et al., 2024) model and incorporates design principles of neural graph matching (Wang et al., 2022; Yu et al., 2020). In MARASON, we retrieve structures with known reference spectra from the training dataset (NIST, 2020). We ex- ploit ICEBERG's fragmentation model to annotate reference spectrum peaks with fragments and form fragmentation directed acyclic graph (DAG) with peak annotations for both the target and reference structures. We then match reference fragments to target fragments to inform the second-stage model's intensity prediction. We incorporate nested graph neural networks (GNNs) (Zhang & Li, 2021; Scarselli et al., 2008) to encode (a) structural information of each fragment and (b) hierarchical information of the fragmentation DAG into node embeddings, building a neural graph matching network for MS/MS simulation.
3) Our experimental evaluation on standard benchmarks demonstrates state-of-the-art accuracy on the mass spec- trum simulation task, outperforming both RAG and non- RAG baselines and thus validating the effectiveness of our design strategy. Specifically, on the NIST (2020) dataset, we improve the top-1 retrieval accuracy from 19% to 28%. Ablations demonstrate that MARASON outperforms a naive RAG baseline (by concatenating the retrieved spectrum to the model input) as well as other matching methods using traditional graph matching solver (Cho al., 2010) or a sim- ple linear matching solver (Kuhn, 1955). emphasizing the value of its differentiable, end-to-end neural graph matching module. Given the broad applicability of mass spectrom- etry in biological and chemical campaigns, the improved accuracy of our model lends great potential for accelerating molecular discovery by expanding standard mass spectrum
To this end, we implement and validate our design strat- egy in a prominent application of molecular machine learn- ing: the neural simulation of spectra from tandem mass spectrometry (MS/MS). MS/MS is an analytical chemistry technique that generates profiles from unknown molccules which are used as diagnostic signatures in structure elu- cidation workflows. This approach has wide-ranging ap- plications in chemistry and biology, including biomarker discovery (Dang et al., 2009), metabolomics (Quinn al., 2020), and environmental science (Tian et al., 2021), among others. An MS/MS simulator is designed to take molecu- lar structures as input and predict their mass spectra, i.c., a set of cheree (n/2) values and peak intensi- ties. Simulating these spectra can accelerate the structural elucidation pipeline, illuminating the spaces of so-called
II this paper, we refer to all structures and spectra retrieved from the database as "reference", and the structures and spectra of interest "target".
2

---

Neural Graph Matching Improves Retrieval Augmented Generation in Molecular Machine Learning
target fragment DAG...
for all target fragments
MARASON input
m//
inten ghit
+HH
target structure
Matching layer
target fragment
GNN
Transformer
K3H
standard library
GNN
MLP
Mi
fragmenttto-fragmenn matchhing
ligh M
retrieved closest
ref. fragment DAG
ref. fragment
structure
ref. Intensity: [M+
ref. structure
MARASON output
intensity assignment
merge
ref. spectrum
target spectrum
Figure 2. Overview of MARASON: retrieval-augmented mass spectrum simulator with neural graph matching. We retrieve reference structures and spectra from a reference library (more specifically, the training datasct) based on Tanimoto similarity (Bajusz et al., 2015) to the target structure. Both target and reference structures are fragmented, resulting in two similar fragmentation DAGs, as similar structures are expected to have similar fragmentation patterns in chemistry (Shahneh et al., 2024). The neural graph matching module further finds the node-level matching between fragmentation DAGs, whereby each node represents fragment. The aligned target and reference fragment, together with reference intensity, are further concatenated to predict the final spectrum. We use three identical GNN modules with separate weights learn embeddings: one for the target fragments, one for the reference fragments, and one shared by both target and reference fragments to capture matching information. After computing the embeddings with these GNN modules, we use the shared GNN's outputs as input to matching module, which produces a matching matrix. This matrix is then used to align reference fragments and spectral peaks with the target fragments. The resulting matched fragment pairs, along with their similarity scores, are fed into transformer module predict the final relative intensities of the spectral peaks corresponding to each target fragment.
libraries with simulated spectra for novel annotation, from a current size of ~27K unique compounds in NIST (2020) to all 11 1M compounds in PubChem and beyond.
torial optimization problem that matches the nodes of mul- tiple graphs by maximizing the edge-wise and node-wist affinities. Neural graph matching was developed tackle the computational challenges of the NP-hard quadratic as- signment problem (Lawler, 1963) and the vulnerability of predefined affinity metrics (Rolínek et al., 2020; Nowak et al., 2018; Guo et al., 2023). Among all design choices, a family of linear-matching methodologies (Wang et al., 2020; Yu et al., 2020; Sarlin et al., 2020) incorporates graph neural networks (Scarselli et al., 2008) to embed edge affinities into node embeddings that then used in differentiable node- matching layer using, e.g., Sinkhorn & Rangarajan (1964) simpler Softmax for assignment. This paper explores and validates the applicability of neural graph matching in the context of in molecular machine learning.
2. Related Work
Retrieval Augmented Generation in AI for Science. Re- trieval augmented generation (RAG) is a technique that inte- grates relevant information retrieved from external databases into a model's training and inference processes. While origi- nally developed aid Large Language Model (LLM)-basee AI agents, their applicability has readily extended to scien- tific tasks, spanning LLMs for material discovery (Buehler, 2024), drug design (Pal et al., 2023), and organic synthe- sis (M. Bran et al., 2024). Utilizing RAG in various forms has also shown ground-breaking advances in scientific dis- covery beyond LLM agents; one of the most prominent examples AlphaFold model (Jumper et al., 2021), whose sequence alignment module is effectively RAG module. RAG is also found helpful in molecular machine learning tasks such as molecular generation (Lee et al., 2024), drug design (Zhang et al., 2024). and protein function predic- tion (Shaw et al., 2024). Despite success certain molecu- lar learning tasks, there remains the opportunity for a princi- pled RAG design strategy tailored to small molecules that can perform robust and informative structural matching.
mass spectra generation
3. Methods
MARASON is mass spectrum simulator built the ICE- BERG model (Goldman et al., 2024) with reference retrieval augmentation and neural graph matching. In this section, unless otherwise specified, we use capitalized bold letters for matrices, lowercase bold letters for vectors, and non- bold letters for scalars. M,F,, 9 indicate a molecular graph, fragment graph, and fragmentation DAG (from ICEBERG- Generate), respectively. All reference-related variables
Neural Graph Matching. Graph matching is combina-
3

---

Neural Graph Matching Improves Retrieval Augmented Generation in Molecular Machine Learning
annotated with a superseript "r"
by Tanimoto similarity (Bajusz et al., 2015) operating on Morgan fingerprints (Morgan, 1965). We exclude the entries that have nonmatching adduct types or instrument types recorded to mitigate the confounding effects of different experimental conditions.
3.1. Preliminary: ICEBERG MS/MS Simulator
Mass spectrometry (MS) is a powerful analytical chemistry method for the discovery of unknown compounds and nat- ural products (Wang et al., 2016). Compounds analyzed with tandem MS undergo an ionization and fragmentation process, after which charged fragments are detected at their m/z values with an intensity proportional to their relative abundance and cross section. MARASON is built on the ICEBERG model which reported state-of-the-a accuracy. As described by Goldman et al. (2024). ICEBERG is a two-stage model:
We then identify up to three reference spectra with the most similar collision energy to the target (query) structure M. Collision energies influence the extent of fragmentation and shift the spectrum patterns towards lower m/z ranges with higher energies; there are often multiple spectra with differ- ent collision energies for unique molecule in NIST (2020), The result of retrieval is therefore a single reference struc- ture M (animoto(MM M"), and up to three reference spectra and their collision energy values, which are fed into MARASON. As the collision energy value is continuous, our aim of including three reference energies is to learn an appropriate spectrum embedding for the target collision energy value by interpolating from three closest energies.
ICEBERG-Generate learns to mimic collision-induced dis- sociation by predicting the most likely fragments through recursive removal of atoms and bonds from the molecular graph. It results in an autoregressive model to handle mul- tiple breakages, whose output is a directed acyclic graph (DAG) where the root node is the input molecule and frag- ments the children. The hierarchical information of fragments from ICEBERG-Generate is used in MARASON for graph matching.
3.2.2, FRAM-FTAGMENT ASSIGNMENT AND LEARNING
Reference spectra only provide m/2 values and peak inten- sities. This information is enriched by annotating peaks with ICEBERG-predicted fragments of the reference structure. For each fragment Fj. we assign all peaks that fall within the 20 ppm (parts-per-million
ICEBERG-Score takes the fragmentation DAG and outputs the intensities of peaks corresponding to each fragment. As shown in Fig. 2, the output layer also accounts for up to #6 hydrogen shifts in mass to support chemical rearrange- ments. ICEBERG-Scor Score is composed of 1) GNN module that takes in the molecular graphs and context parameters and outputs the corresponding hidden representation of frag- ments and root molecules; and 2) a set transformer (Lee et al., 2019) that predicts intensity values for each fragment. Predicted intensity values are merged into a mass spectrum, where the m/a values are straightforwardly calculated from the structures of fragments.
range of at most 13 mass values: { - 65, Fj - 56, oo., Fy.... ..... Fj + 68), where 8 is the mass of a hydrogen atom. Such a process leads to a 1 di-dimensional vector for each Fj indicating its intensi- ties in the reference spectrum. The assigned 13-dimension intensity is concatenated with values of the reference and target collision energies. All reference intensities at the same collision energy are processed by set transformer, followed by an average pooling layer that merges intensity embeddings per fragment from three collision energies. The obtained reference intensity embeddings, denoted as T", are defined such that the pooled embedding for fragment j is represented as ty.
MARASON aims to improve ICEBERG by integrating retrieval augmentation with neural graph matching into ICEBERG-Score.
3.3. Fragmentation DAG Graph Matching
3.2. Retrieval Augmentation Processing
We define a fragmentation DAG graph matching problem to recapitulate domain experts' practice of comparing simi- lar fragments from the reference and the target, alongside their hierarchy with respect to the original structures. In fragmentation DAGs from the (CEBERG-Geneeee fragments are viewed as nodes, and fragmentation paths are viewed as edges. Since each fragment is a molecu- lar graph, a fragmentation DAG is a graph of graphs, i.c., a meta-graph. Following training steps in Goldman et al. (2024), a pretrained ICEBERG-Generain model predicts two fragmentation DAGs: one for the reference molecular graph and one for the target. We then approach the graph matching task using fixed affinity metrics with traditional
With the chemical intuition that structurally similar molecules are expected to have similar spectra (Shahneh et al., 2024), we retrieve molecules with the highest struc- tural similarity to our target molecule. This subsection presents our implementation of MS/MS retrieval and pro- cessing of retrieved data for best deep-learning efficacy.
model,
3.2. 1. RETRIEVAL OF STRUCTURES AND SPECTRA
Let our retrieval database be any database with annotated cruuctrre-spectrun pairs; in our experiments, we use the training dataset to mitigate concerns about data leakage. We retrieve the most similar reference molecule as quantified
4

---

Neural Graph Matching Improves Retrieval Augmented Generation in Molecular Machine Learning
graph matching solvers as well as learnable affinity metrics with neural graph matching.
in the context of RAG for MS/MS simulation, we propose neural graph matching method that leverages message- passing networks on the fragmentation DAG. Specifically,, we develop nested GNN (Zhang & Li, 2021) that learns fragment-level embedding and DAG hierarchical embed- ding by two GNNs, followed by a differentiable matching layer.
3.3.1. TRADITIONAL SOLVERS WITH FIXED METRICS
Defining the affinity metric is the first step of matching fragments between two fragmentation DAGs. Since each fragment is a molecular graph, we resort to Tanimoto sim- ilarity (Bajusz et al., 2015) and construct the following pairwise affinity matrix M:
Fragment-level embedding learning. For each frag- ment F, from the target molecule M,, since both F, and .M are molecular graphs, we learn graph-level em- beddings with a shared GNN We build embeddings at the fragment level for F, using an MLP to project the concatenation of GNN (M), (MM), GNN (Fr), and (GNN - GNN (Fi)), together with an encoded number of broken bonds (from ICEBERG-Generate) en- coded chemical formula of Fi and the chemical formula difference from M to F, These differences in formula and GNN embeddings represent the chemical concept of "neutral losses". The same neural networks are applied to the target fragments and the reference fragments, yielding H and H' respectively as the fragment-level embeddings learned from GNNs.
m + (animoto(Fs..F))
(1)
where Fi is fragment i of the target structure and Jj is fragment j of the reference structure. With the Tanimoto-
based fragment-level affinity scores, we can formulate linear assignment problem:
max tr(M X), X
s.t. X E (0,1) .1,,' S 1n, X 1n S 1m";
where X is the matching matrix, n and n' are the number of fragments from the target DAG and the reference DAG,
respectively, I, means a column vector of n 1s. Ti.j means F, is matched to Fÿ and Tú, = otherwise. Eq. (2) can be solved by the Hungarian algorithm (Kuhn, 1955),
DAG hierarchical embedding learning. For each fragmen- tation DAG, we construct two graphs G and g-1, where G is the top-down fragmentation DAG in which eij E G if and only if fragment is a parent of fragment j. G-1 is the bottom-up DAG where all edges are reversed. With H (or Hn) extracted by aforementioned fragment-level GNNs, it is updated as
whereby it incorporates fragment-level graph affinities but ignores the hierarchical information in DAGs.
Graph matching further extends Eq. (2), by explicitly mod- eling edge affinities in the DAG, resulting in a quadratic assignment problem (Lawler, 1963):
H H + (H, GNN,ev (H,
(4)
max vxc(X) K vxc(X), X
(3)
so that the DAG structures are embedded into fragment-level embeddings. We apply the same GNNs for both target DAG and reference DAG.
where the constraints are the same as Eq. (2), K E gan""an" is the affinity matrix, and vec(-) means column- wise vectorization. The diagonal elements of K are taken to be M as the fragmeent-level affinities, while the off-diagonal elements are constructed by the inner product of edges of DAGs. Solving Eq. (3) is harhard, but there are approxi- mate solvers available (Cho et al., 2010; Leordeanu et al., 2009) through Python interface (Wang et al., 2024).
xnn"
Similarity and differentiable matching. We then calculate the similarity matrix M of given reference target pair as
mai + cosine(h,, hz),
where h, is the embedding of fragment i of the target, and hy is the embedding of fragment j of the reference. We can then calculate the matching matrix X as follows
After solving Eq. (2) or Eq (3), we establish the matching between fragments and retrieved intensities using the match- ing result X. As shown in Fig. I and later ablation studies (Table 3), these methodologies yield inferior results to neu- ral graph matching, indicating the importance of fiexible, learnable affinity metrics.
X + matching(M)..
(6)
In neural graph matching research, matching(-) could ei- ther be the Sinkhorn (Sinkhorn & Rangarajan, 1964) or Softmax algorithms; Sinkhom is preferred in smaller-sized graphs to enforce stronger matching constraints (Wang et al., 2020), whereas Sofumax is used in larger-sized graphs for its efficiency (Sarlin et al., 2020). We implement Softmax- based matching in this paper after a careful ablation study, as empirical experiments with 100 fragments generated
3.3.2. NEURAL MATCHING WITH LEARNED METRICS
The biggest drawback of traditional graph matching meth- ods is that their fixed affinity metrics do not incorporate the flexibility required to handle noisy real-world data ( (Wang et al., 2020; Cho et al., 2013). To overcome this challenge
5

---

Neural Graph Matching Improves Retrieval Augmented Generation in Molecular Machine Learning
by ICEBERG-Generate suggest this number does not pose serious concern of constraint violation. It is worth not- ing that X is continuous matrix so that the neural graph matching pipeline is differentiable, and that X is further integrated into the intensity prediction module so that it will receive gradients during training and learn the affinity metric end-to-end.
Random split
Scaffold split
0.8
spectrrum retrieval
0.7
Cosine Similerity
0.6
0.5
0.4
3.4. Intensity Prediction
0.3
3DM5iMs NEIMS (PPN)
NEBMS (GReN)
resssfomme
ICEBERG fer coll. MARA RASON (ours)
SCARF ICEBERG
20MeiMS
NEMS (PFN)
REIS (GWN)
Foodvocal
Massformer
ICEBERG com MARASON AARSSONN (ours)
SCARF iCEBERG
Given the target fragment embeddings H and the reference fragment embeddings H", together with peak intensities T" assigned to reference fragments from Sec. 3.2.2, as well as the matching matrix X bridging target and reference fragments from Sec. 3.3.2, we can then create triplets of (target fragments, reference fragments, reference intensities) to serve as input to the intensity prediction layers. To reiter- ate our motivation for this -augeeeee generation framework: similar structures tend to have similar fragmen- tation patterns in chemistry, and similar fragments tend to have similar response factors that relate their abundance to the observed intensity (Shahneh et al., 2024). Therefore, the intensity of reference fragments should offer hints for the intensity of the target fragments.
Figure 3. Spectral similarity between experimental spectra and pre- dictions from various MS/MS simulators on NIST (2020) dataset. Results of all baselines are from the benchmark developed by Goldman et al. (2024). Incorporating collision energies with ICE- BERG slightly improves the accuracy, while MARASON further improves the accuracy by a substantial margin. Error hars are re- ported for the random split with 3 random seeds. The seaffold split is more challenging and results in lower cosine similarity for all methods. It worth noting, from the dashed lines in the plots, that the retrieved spectra are informative, but the cosine similarities of the retrieved spectra are lower than the predicted spectra produced by all baseline models, demonstrating that RAG on MS/MS is non-trivial task.
When predicting the intensities, we also consider the match- ing score of target fragment as 8;: 8; <- Ej= T.. which is further aggregated with H. H", T", and the Tani- moto similarity of the original molecules for intensity pre- diction. The input the concatenation of
cess? We conduct experiments and answer these research questions quantitatively and qualitatively.
4.1. Experimental Setup
H XH", XTT, s, Manimoto(M,,
(7)
4.1.1. DATASET AND BASELINES
which then processed by a set transformer (Lee et al., 2019) where embeddings associated with each fragment are treated as one element in the set. Finally, the intensities are computed through an attention layer and an MLP.
We trained our models on the NIST (2020) dataset with 530,640 high-energy collision-induced dissociation (HCD) spectra and 25,541 unique molecular structures. The number of spectra is larger than unique structures because spectra are collected at different collision energies with different adducts. The dataset is split into structurally disjoint 80%- 10%-10% tain-valldatt-test subsets. Following Goldman et al. (2024), we evaluate on two different splits: (1) a ran- dom split that splits different InChl keys and (2) a Murcko scaffold split that clusters different molecular scaffolds that require more generalization to out-of distribbution structures. It is worth noting our dataset is preprocessed differently from Goldman et al. (2024) as we include collision energies as input that are crucial to retrieving the most informative spectra, whereas the original dataset pre-average spectra when multiple collision energies are used. We also include negative adduct types in training to exploit the full NIST (2020). Despite this difference, the testing split is kept consistent with the ICEBERG paper to maintain compar- ative evaluation; we also include an intermediate ablation (retrieval free ICEBERG with collision energy) to fairly position the improvement of RAG. Since collision energies
4. Experiments
We implement MARASON with PyTorch (Paszke et al., 2017), based on the official implementation of ICE- BERG (Goldman al., 2024) and the graph matching toolkit Pygmtools (Wang et al., 2024). All experiments are conducted on a workstation with AMD 3995WX CPU, 4XNVIDIA A5000 GPU, and 512GB RAM. Our exper- iments are designed to answer the following questions: (1) Does MARASON, a representative implementation of based RAG in molecular machine
neural graph matching basse learning, improve the performance of mass spectrum predic- tion compared to retrieval-fre ICEBERG and other state- of-the-art models? (2) How accurate is MARASON on authentic assessment of identifying unknown compounds to support its application to real-world chemical and biological campaigns? (3) Does MARASON generate a reasonable matching pattern that aligns with the fragmentation pro-
6

---

Neural Graph Matching Improves Retrieval Augmented Generation in Molecular Machine Learning
Table 1. Retrieval accuracy (mean 95% confidence interval on 3 random seeds) on NIST (2020) dataset with random (InChl key)
split on positive adduct types. MARASON surpasses all baselines in terms of retrieval accuracy, which is an authentic assessment of MS/MS simulators' performance in real real-world applications where they are used to distinguish the true structure from list of candidates.
Results of all baselines are from the benchmark developed by Goldman et al. (2024). We also outperform the recently developed method
FraGNNet (Young et al., 2024b), which is reported separately Table 6 in the Appendix as only allows prediction for [M+HI]+ adducts.
Accuracy @ Top-k
2
5
8
10
3DMoIMS (Hong et al., 2023)
0.055=0.003
0.105=0.000
0.146=0.005
0.185=0.007
0.225=0.009
0.332=0.005
0.394=0.008
FixedVocab (Murphy al., 2023)
0.172-0.004
0.304_0.004
0.399_00002
0.466_0.007
0.522-10012
0.63810.009
0.688_0.006
NEIMS (FFN) (Wei et al., 2019)
0.105=0.003
0.243=0.012
0.324=0.013
0.387=0.011
0.440=0.014
0.549=0.010
0.607-0.005
NEIMS (GNN) (Zhu et al., 2020)
0.175-0.005
0.305-0.003
0.398-0.002
0.462-0.004
0.515+0.005
0.632+0.007
0.687+0.005
MassFormer (Young et al., 2024a)
0.191=0.008
0.328=0.006
0.422=0.004
0.491=0.002
0.550=0.005
0.662=0.005
0.716=0.003
SCARF (Goldman et al., 2023)
0.187=0.008
0.321=0.006
0.417=0.007
0,486=0.008
0.541=0.009
0.652=0.008
0.708=0.009
ICEBERG (Goldman et al., 2024)
0.189_0.012
0,489-000007 0.375_0.005
0.56710.005
0.62310.004
0.77010.000 0.720
ICEBERG (w/ collion energy)
0.202=0.009
0.399=0.008
0.513=0.008
0.585=0.008
0.639=0.010
0.749=0.006
0.793=0.007
MARASON (ours)
0.278 0.002
0.455 0.004
0.562-0.009
0.636-0.000 0.006
0.685 0.004
0.827-0.004
are found crucial to RAG and the NPLIB1 dataset used in
Expa imeetal (47.2 uv)
Tacforncc 145.0 eil
Goldman et al. (2024) does not have collision energy la-
bels for most of the spectra, this dataset is not included for comparison.
We include all peer methods reported in the benchmark developed by Goldman et al. (2024). Traditionally, MS/MS simulators have relied on combinatorial enumeration of bond breakages, which mimics the physical process, but they are computationally demanding and often inaccurate (Allen et al., 2015; Ridder et al., 2014). Deep learning- based simulators significantly reduce the inference time with competitive accuracy, while the physical constraint is either fully lifted (Young al., 2024a; Wei et al., 2019) or relaxed to subformulae (Murphy et al., 2023; Goldman et al., 2023). ICEBERG combines both ideas by learning the fragmentation DAG. Besides them, we also consider a "spectrum retrieval" baseline by simply taking the retrieved spectrum as the prediction.
Predicted (MARASONI) Simmid 325
Fred ctud (MARASON, RAG) Cos 5im-0
1702
Figure 4. Qualitative comparison of experimental spectrum (i.e., bround-truth), reference spectrum, spectrum predicted by MARA- SON, and spectrum predicted by MARASON (no RAG). Cosine similarities between the experimental spectrum and predicted spec- tra are annotated. The retrieved reference structure, as shown in the plot, is structurally similar to the target structure, whereby the reference spectrum offers important intensity information. Our wet---mtttiinaaeeeee RAG strategy associates referenee peaks with the target, despite the peaks having different m/z values be- tween the reference and the target.
4.1.2. QUANTITATIVE EVALUATION METRICS
Spectrum similarity. We evaluate model performance via how similar the predicted mass spectra are to experimental spectra. Specifically, since we consider molecules under 1,500 Da we create a 15, 5.000-dimennionall vector valued be- tween 0 to 1 that encodes the spectrum, where each element means the peak intensity that falls within the mass bin of 0.1 Da, to balance mass resolution and the vector dimen- sion sparsity. All method outputs are transformed into this 15,000-dim vector so that it accommodates both methods that predict binned spectra (Wei et al., 2019; Young et al., 2024a) and methods that predict intensities with known ex- act mass (Murphy et al., 2023; Goldman et al., 2023; 2024), including ours. In this paper, we use cosine similarity to compare spectra, which is the de facto standard metric used in real-world case studies (Li et al., 2021).
candidate structures and compare them to the experimental spectrum. Candidate structures can then be ranked based on spectrum similarity to identify the most likely structure. This retrieval setting evaluates how this model might be deployed in a real-world structural elucidation campaign. Following Goldman et al. (2024), for each molecule from the testing dataset, we obtain up 49 isomeric decoys with the highest Tanimoto similarities to the true target structure, i.e., the decoys most likely to be mistaken under Tanimoto measurement. We run MARASON and all comparative methods on 50 structures per test spectrum (49 decoys +1 I true structure) and rank all of them by cosine similarity. We evaluate the retrieval accuracy at top-k:
Retrieval accuracy. In structural elucidation campaigns, MS/MS simulators can predict pseudo-spectrum labels for
7

---

Neural Graph Matching Improves Retrieval Augmented Generation in Molecular Machine Learning
Table 2. Retrieval accuracy (mean with 99.9% confidence intervals upon bootstrapping, 20,000 resamples) with known chemical formula on the MassSpecGym dataset (Bushuiev et al., 2024). We consider MassSpecGyy as more challenging setting than NIST (2020) as it incorporates fewer annotated spectra and emphasizes generalization to different molecular scaffolds. MARASON surpasses all baselines in terms of retrieval accuracy. Baseline performances are quoted from Bushuiev et al. (2024).
Accuracy @ Top-k
5
20
NEIMS (FFN) (Wei et al., 2019)
0.0762 (0.0677-0.0854)
0.2270 (0.2132-0.2412)
0.4412 (0,4251-0.4575)
NEIMS (GNN) (Zhu et al., 2020)
0.0363 (0.0305-0.0429)
0.1355 1246-0.6.. 1468)
0.3377 0..322--033577
FraGNNet (Young al., 2024b)
0.3193 (0.300000033300)
0.6320 (0.6164-0.6476)
0.8270 (0.8145-0.83933)
MARASON (ours)
0.3403 (0.3286-0.352))
0.6404 (0.6277-0.6519)
0.8539 (0.8448-0.86244)
upon ICEBERG from 18.7% to 27.8%, a marked increase in the state-of-the-art performance for this task by a relative margin of 48% over the next best method. As an abla- tion study, ICEBERG (with collision) has a top-1 accuracy of 20.2%, indicating that the majority of improvement is directly attributable to our use of RAG. MARASON con- sistently outperforms all baselines from top-1 to top- 10 retrieval accuracies. Table I only covers the random split; retrieval experiment results on scaffold split are in Table 5 in the Appendix.
Table 3. Ablation Study of MARASON design choices. RAG strategies include no RAG, concatenating one reference spec- trum to the model input, traditional matching methods dis- cussed in Sec. 3.3.1 including Hungarian (Kuhn, 1955) and RRWM (Cho et al., 2010), and neural graph matching (NGM) presented in Sec. 3.3.2 where the matching layer could be either Sinkhorn (Sinkhorn & Rangarajan, 1964) or Softmax. We compare cosine similarity on random split with seed = 1.
Base model
RAG strategy
Match layer
Cosine sim.
No RAG
0.739
Concat
0.737 (-0.3%
MARASON
Hungarian
0.746 (+0.9%)
4.2.3. RETRIEVAL FROM THE MASSSPECGYM DATASET
(shared GNN)
RRWM
0.742 (+0.4%)
We further retrain MARASON on the recently developed open-source dataset, MassSpecGym (Bushuiev et al., 2024), where we achieve state-of-the-arr retrieval accuracy, as shown in Table 2. Since not all spectra in MassSpecGym have collision energy label, but MARASON requires la- beled collision energies, we use a NIST-pretrained model to create pseudo labels for those unannotated spectra, i.e., transferring the collision energy knowledge from NIST to MassSpeeGym. The model is retrained from scratch on MassSpecGym data to ensure no data leakage from NIST. MARASON improves the top-1 retrieval accuracy upon the current state-of-the-art, FraGNNet, from 31.93% to 34.03%, a marked increase by a relative margin of 6% over the next best method. Random seed is fixed as 1 for MARASON.
NGM
Sinkhom
0.749 (+1.4%)
NGM
Softmax
0.753 (+1.99)
MARASON
NGM
Sinkhom
0.753 (+1.9%)
(not shared GNN)
NGM
Softmax
0.757 (+2.4%)
4.2. Results and Discussions
4.2.1. SPECTRAL SIMILARITY AND VISUALIZATION
We evaluate the prediction power of MARASON by compar- ing the spectral cosine similarity of the predicted spectrum of different baseline models on the NIST (2020) dataset. The results are summarized in Fig. 3. MARASON outper- forms all baselines in both types of splits, and the success on scaffold split excludes the possibility that MARASON needs highly similar reference structures for accurate pre- dictions. It improves the cosine similarity of the ICEBERG baseline by a relative 5.2% on random split and relative 3.7% on scaffold split. Since the original ICEBERG model does not consider collision energy, we also compare a colli- sion energy-awand version of ICEBERG. This study demon- strates that neural graph matching- based RAG improves the performance of MS/MS simulation, a representative task in molecular machine learning. An example of predicted spectra of MARASON is provided in Fig. 4.
4.2.4. ABLATION STUDY
We conduct an ablation study compare matching algo- rithms and GNN designs on the NIST (2020) dataset under random split, as shown in Table 3. MARASON includes modest engineering changes to ICEBERG for more efficient training and inference on GPUs, which accounts for the slight improvement over ICEBERG (with collision). The naive RAG strategy that concatenates the reference spectrum as an extra 15 15.000-dimensione input leads to a negative im- pact on the cosine similarity. Traditional graph matching methods discussed in Sec. 3. by comparison, yield a minor benefit of fragment-level matching in RAG. With the comparatively more expressive and end-to-end learnable neural graph matching described in See. 3.3.2, the cosine similarity further improved. A possible explanation for
4.2.2. RETRIEVAL. FROM PURCHEM CANDIDATES
The evaluation of the real-world applicability of MS/MS simulators is summarized in Table 1. In the retrieval bench- mark, MARASON improves the top-1 retrieval accuracy
8

---

Neural Graph Matching Improves Retrieval Augmented Generation in Molecular Machine Learning
Pair O.99
Pair 2. O.11
Pair Similarit=006822
Pair 4, Similarity D.569
5, Similarity 0.938
Pair Similarity
tv
Figure 5. Visualization of matched fragment pairs and their Softmax-similen scores discovered by the neural graph matching module in MARASON. Structures on the left are from the target fragmentation DAG and structures on the right are from the reference. Fragments are highlighted from the original structure. Our matching module learns match not only exact structures (pair 2, 3, 4, 5), but also correlated structures with slight modifications (pair 1, 6). Only pains from two fragmentation DAGs are shown due to space limits, the full list of matched pairs be found in Fig. 6 in the Appendix together with visualization of traditional graph matching in Fig. 7.
Table 4. The performance of MARASON with and without RAG on target and reference pairs grouped by Tanimoto similarity. We report cosine similarity on the random split with random seed 1. There is no testing sample with Tanimoto similarity hetween 0 and 0.I.
Tanimoto Similarity
(0, 0.1]
(0.1.022
(0.2, 0.3]
(0.3, 0.4)
(0.4, 0.5)
(0.5, 0.6]
(0.6, 0.7]
(0.7, 0.8]
(0.8, 0.9]
(0.9, 1.0]
MARASON
N/A
0.550
0.614
0.690
0.741
0.789
0.815
0.808
0.805
0.824
MARASON (No RAG)
N/A
0.566
0.611
0.682
0.727
0.768
0.791
0.780
0.759
0.784
the superiority of Softmax over Sinkhorn is that Softmax is sufficient for the many-to-one aggregation path in Eq, (7) and provides better gradients because it takes fewer itera- tions. It is also shown in Sarlin al. (2020) that Softmax outperforms the matching layer for larger-sized graphs. Finally, we discover that separating the GNN that learns embeddings for matching and the GNN that learns embed- dings for intensity prediction results in a higher spectrum similarity, compared to using GNNs with shared weights for both purposes. The reason could be that MARASON needs separate modules for learning intensity-related information (e.g., molecular fragment cross sections) and information about the fragmentation DAGs themselves.
based on their Tanimoto similarities and compare perfor- mances of MARASON with or without RAG (same model configurations as Table 3). As shown in Table 4, RAG starts bring a significant performance gain when the Tan- imoto similarity is larger than 0.3. When the best refer- ence structure has Tanimoto similarity between 0.1 and 0.3, RAG does not bring significant improvement, while MARASON still performs robustly with such irrelevant ref- erences. The slight performance drop between Tanimoto similarity (0. 1, 0.2] also suggests a simple trick to further improve MARASON's accuracy: use standard MARASON when the retrieved structure has a Tanimoto similarity > 0.2 and use the non-RAG version otherwise.
4.2.5. QUALITATIVE STUDY OF MATCHING PATTERNS
5. Conclusion
We visualize matching pairs of fragments by assigning each target fragment to the reference fragment with the highest Softmax score. An example of six matched fragment pairs shown in Fig. Fragment pairs also show similar frag- mentation patterns (missing the same functional group, C-N bond breakages, etc.). This example illustrates how MARA- SON has learned to match fragments that are generated through similar mechanisms in the fragmentation process in order to model the relationship between each matched peak and fragment pair the reference and target spectrum.
This paper presents a work for molecules by integrating neural graph matching into an existing end-to-end geometric deep learning frame- work. On the molecular machine learning task of mass spectrum simulation, we implement MARASON to match the fragmentation DAGs of the target and reference struc- tures, pair up peaks and fragments in the reference and target structures, and use that alignment information to generate improved spectrum predictions. MARASON establishes new state-of-the-an performance terms of the quality of simulated spectra and retrieval accuracy in downstream ap- plications. Future improvements and adaptations could see this neural graph matching strategy applied to broader range of eetrieval-augmente tasks across the field of molecular machine learning.
generation frame-
4.2.6. WHAT IF THERE NO REFERENCE STRUCTURE?
To understand the relationship between RAG's improvement and the availability of a similar-enough reference structure, for all testing data in NIST (2020), we group structural pairs
mewettur--rroperry prediction

---

Neural Graph Matching Improves Retrieval Augmented Generation in Molecular Machine Learning
Acknowledgements
Dang, L., White, D. W., Gross, S., Bennett, B. D., Bittinger, M. A., Driggers, E. M., Fantin, V. R., Jang, H. G., Jin, S., Keenan, M. C., et al. Cancer-associated idhl mutations produce 2-bydroxyglutarate. Nature, (62(7774):739--4444 2009.
This work was partly supported by DSO National Labora- tories in Singapore and the MIT Undergraduate Research Opportunities Program (UROP).
Impact Statement
Goldman, S., Bradshaw, J., Xin, J., and Coley, C. Prefix- tree decoding for predicting mass spectra from molecules. Advances in Neural Information Processing Systems, 36: 48548-48572, 2023.
This paper presents work that aims to advance both machine learning and mass spectroscopy. As a methodology- focusee paper, this paper itself does not pose any significant societal impact that is required be highlighted here. The authors also aware that mass spectroscopy could be used in human- research campaigns such as metabolomics, and it is important to ensure that the reference database is not biased towards ethnicity, sex, and age, or at least being acknowledged about the potential bias. On the other hand, efforts to develop better mass spectroscopy models, including this paper, will potentially lower the cost and expedite those discovery campaigns. If such models are used properly for the underrepresented cohort, it will finally mitigate the bias and unfairness in human-centric studies.
Goldman, S., Li, J., and Coley, C. W. Generating molecu- lar fragmentation graphs with autoregressive neural net- works. Analytical Chemistry, ((88))3419-4222 2024.
Griffen, E., Leach, A. G., Robb, G. R., and Warner, D. J. Matched molecular pairs as a medicinal chemistry tool: miniperspective. Journal of medicinal chemistry, 54(22): 7739-7750, 2011.
Guo, Z., Wang, R., Jiang, S., Yang. X., and Yan, J. Deep learning of partial graph matching via differentiable top-k. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pp. 6272-6281, 2023.
References
Allen, F., Greiner, R., and Wishart, D. Competitive fragmen- tation modeling of esi-ms/ms spectra for putative metabo- lite identification. Metabolomics, 11:981102 10, 2015.
Hong, Y., Li, Welch, J. Tichy, S., Ye, Y., and Tang, H. 3dmolms: prediction of tandem mass spectra from 3d molecular conformations. Bioinformatics, btaa 2023.
Bajusz, D., Rácz, A., and Héberger, K. Why is tanimoto in- dex an appropriate choice for fingerprint-basee based similarity calculations? Journal of cheminformatics, 7:1-13, 2015.
Huang, Z., Yang. L., Zhou, X.,, Qin, C., Yu, Y., Zheng, X., Zhou, Z., Zhang, W., Wang, Y., and Yang, W. Interaction- based retrieval- augmeeted diffusion models for protein- specific 3d molecule generation. In International Confer- encee on Machine Learning. 2024.
Bittremieux, W., Wang, M., and Dorrestein, P. C. The critical role that spectral libraries play in capturing the metabolomics community knowledge. Metabolomics, 18 (12):94, 2022.
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Zidek, A., Potapenko, A., et al. Highly accurate protein structure prediction with alphafold. Nature, 596(7873),533-589, 2021.
Buchler, M. J. Generative retrieval- augmented ontologic graph and multiagent strategies for interpretive large lan- guage model-based materials design. ACS Engineering Au, 4(2):241-271 2024. 2024.
Kuhn, H. W. The hungarian method for the assignment problem. Naval research logistics quarterly, 2(1-2):83- 97, 1955.
Bushuiev, R., Bushuiev, A., de Jonge, N. F., Young, A., Kretschmer, F., Samusevich. R., Heirman, J., Wang, F., Zhang, L., Dührkop, K., et al. Massspecgym: A bench- mark for the discovery and identification of molecules. In The Tiirty-eight Conference on Neural Information Pro- cessing Systems Datasets and Benchmarks Track, 2024.
Lawler, E. L. The quadratic assignment problem. Manage- ment Science, (4):586-559, 1963.
Lee, J., Lee, Y., Kim, J., Kosiorek, A., Choi, S., and Teh, Y. W. Set transformer: A framework for attention-based coamutation-invarian neural networks. In International conference on machine learning, pp. 3744-3753. PMLR, 2019.
Cho. M., Lee. J., and Lee, K. M. Reweighted random walks for graph matching. In European Conference on Computer Vision, pp. 492-505, 2010.
Cho, M., Alahari, K., and Ponce, Learning graphs to match. In International Conference on Computer Vision, pp. 25-32, 2013.
Lee, S., Kreis, K., Veccham, S. P., Liu, M., Reidenbach, D., Paliwal, S., Vahdat, A., and Nie, W. Molecule generation
10

---

Neural Graph Matching Improves Retrieval Augmented Generation in Molecular Machine Learning
with fragment retrieval augmentation. arXiv preprint arXiv:2411.12078, 2024.
Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E.,
DeVito, Z.,, Lin, Z.., Desmaison, A., Antiga, L.., and Lerer, A. Automatic differentiation in pytorch. 2017.
Leordeanu, M., Hebert, M., and Sukthankar, R. An integer projected fixed point method for graph matching and map inference. In Neural Information Processing Systems, PP- 1114-1122, 2009.
Quinn, R. A., Melnik, A. V., Vrbanac, A., Fu, T., Patras,
K. A., Christy, M. P., Bodai, Z., Belda-Ferre, P., Tripathi,
A., Chung, L. K., et al. Global chemical effects of the microbiome include new bile-acid conjugations. Nature, 579(7797) 2020.
Lewis, P., Perez, E.,, Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-t, Rocktäschel, generation for knowledge- intensive nlp tasks. Advances in Neural Information Pro- cessing Systems, 33:9459-9474, 2020.
Ridder, L., van der Hooft, J. J. J., and Verhoeven, S. Auto- matic compound annotation from mass spectrometry data using magma. Mass Spectrometry, 3(Special Issue_2 2); 2014. doi: 10.5702/masspectromeery.
T., et al. eetrieval-augmeent
Li, Y., Gu, C., Dullien, T., Vinyals, O., and Kohli, P. Graph matching networks for learning the similarity of graph structured objects. In International Conference Ma- chine Learning, pp. 3835-3845, 2019.
S0033.
Rolínek, M., Swoboda, P., Zietlow, D., Paulus, A., Musil, V., and Martius, G. Deep graph matching via blackbox differentiation of combinatorial solvers. In European Conference on Computer Vision, pp. 407-424, 2020.
Li, Y., Kind, T., Folz, J., Vaniya, A., Mehta, S. S., and Fichn, O. Spectral entropy outperforms ms/ms dot product simi- larity for
Sarlin, P.-E., DeToone, D., Malisiewicz, T., and Rabinovich, A. Superglue: Learning feature matching with graph neural networks. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, pp. 2020.
compound identification. Nature Methods, 18(12):552411 1531, 2021.
M. Bran, A., Cox, S., Schilter, O., Baldassari, C., White, A. D., and Schwaller, P. Augmenting large language models with chemistry tools. Nature Machine Intelli gence, 6(5):525-553) 535, May 2024. ISSN 2522-5839. doi: 10.1038//42255--24
Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M., and Monfardini, G. The graph neural network model. Trans- actions on Neural Networks, 20(1):61-80) 2008
URL pestts://dol org/10, 10.8 00w.00033422222222 0t1p110000000/22222--0---3333330 024-00332---00
Shahneh, M. R. Z., Strobel, M., Vitale, G. A., Geibel, C.. Abiead, Y. E., Garg, N., Wagner, B.,, Forchhammer, K., Aron, A., Phelan, V. V., et al. Modifinder: Tandem mass spectral alignment enables structural modification site localization. Journal of the American Society for Mass Spectrometry, 2024.
Morgan, H. L. The generation of unique machine de- scription for chemical structures-a technique developed at chemical abstracts service. Journal of chemical docu- mentation, 1(2):107 1965.
Murphy, M., Jegelka, S., Fraenkel, E., Kind, T., Healey, D., and Butler, T. Efficiently predicting high resolution mass spectra with graph neural networks. In International Con- ference on Machine Learning, pp. 25549-25562. PMLR, 2023.
Shaw, P., Gurram, B., Belanger, D., Gane, A., Bileschi, M. L.., Colwell, L. J., Toutanova, K., and Parikh, A. P. Protex: A aetrieeaa--uggmentell approach for protein func- tion prediction. bioRxir, pp. 2024-05, 2024.
Sinkhorn, R. and Rangarajan, A. A relationship between arbitrary positive matrices and doubly stochastic matrices. Ann. Math. Statistics, 35(2):876-879 1964.
NIST. NIST standard reference database. National Institute of Standards and Technology, 2020. URL https:// www. nist. gov/ /rd.
Tian, Z., Zhao, H., Peter, K. T., Gonzalez, M., Wetzel, J., Wu, C., Hu, X., Prat, J., Mudrock, E., Hettinger, R., et al. A ubiquitous tire rubber-derived chemical induces acute mortality in coho salmon. Science, (71655)) 2021.
Nowak, A., Villar, S., Bandeira, A., and Bruna, J. Revised note on learning quadratic assignment with graph neural networks. In Data Science Workshop, 2018
Pal, S., Bhattacharya, M., Islam, M. A., and Chakraborty, C. Chatgpt or Ilm in next-generation drug discovery and de- velopment: Pharmaceutical and biotechnology companies can make use of the artificial intelligence-bace device for a faster way of drug discovery and development, Dec 2023. URL ht /mc. nobi nlm, nih. nih. arti articles wtt.ccleee/cc110000000
Wang, M., Carver, J. J., Phelan, V. V., Sanchez, L. M., Garg, N., Peng. Y., Nguyen, D. D., Watrous, J., Kapono, C. A., Luzzatto-Knaan T., et al. Sharing and community curation of mass spectrometry data with global natural products social molecular networking. Nature biotech- nology, 34(8):828-88) 2016

---

Neural Graph Matching Improves Retrieval Augmented Generation in Molecular Machine Learning
Wang, R., Yan, J., and Yang, X. Combinatorial learning of robust deep graph matching: an embedding based approach. Transactions Pattern Analysis Machine Intelligence, 2020.
Wang, R., Yan, J., and Yang, X. Neural graph matching network: Learning lawler's quadratic assignment prob- lem with extension to hypergraph and multiple-graph matching. Transactions on Pattern Analysis Machine Intelligence, 44(9):5261-5279 2022.
Wang, R., Guo, Z., Pan, W., Ma, Zhang, Y., Yang, N., Liu, Q. Wei, L., Zhang, H., Liu, C., al. Pygmtools: A python graph matching toolkit. Journal of Machine Learning Research, 25:1-7, 2024.
Wei, J. N., Belanger, D., Adams, P., and Sculley, D. Rapid prediction of cecetonnionnizaion mass spectrom- etry using neural networks. ACS central science, 5(4): 700-708, 2019.
Young, A., Röst, H., and Wang, B. Tandem mass spectrum prediction for small molecules using graph transformers. Nature Machine Intelligence, 6(4):40444) 416, 2024a.
Young, A., Wang, F., Wishart, D., Wang, B., Röst, H., and Greiner, R. Fragnnet: deep probabilistic model for mass spectrum prediction. arXiv preprint arXiv:2404.02360, 2024b.
Yu, T., Wang, R., Yan, J., and Li, B. Learning deep graph matching with thannel-indepenee garian attention. In International Conference Learning Representations, 2020.
embedding and hun-
Zanfir, A. and Sminchisescu, C. matching. In Computer Vision and Pattern Recognition, pp. 1684-2693, 2018.
Deep learning of graph
Zhang, M. and Li, P. Nested graph neural networks. Ad- vances in Neural Information Processing Systems, 34: 15734-15747, 2021.
Zhang, P.-D.. Peng, X., Han, R., Chen, T., and Ma, J. Rag2mol: Structure- based drug design based on retrieval augmented generation. bioRxiv, pp. 2024-10, 2024.
Zhu, H., Liu, L., and Hassoun, S. Using graph neural net- works for mass spectrometry prediction. arXiv preprint arXiv:2010.04661, 2020.
12

---

Neural Graph Matching Improves Retrieval Augmented Generation in Molecular Machine Learning
A. Retrieval Accuracy on Scaffold Split
Following most peer methods, the retrieval accuracy evaluation in the main paper (Table 1) mainly focuses on random split and all positive adduct types. We also include the retrieval accuracy on scaffold split as follows. Scaffold split is considered more challenging than random split as separates structures with different molecular scaffolds into different training or testing sets, requiring the model with more generalization ability for out-of-distribution structures. MARASON maintains its performance superiority over the baselines on scaffold split. Since scaffold split is less studied in peer methods there are fewer baselines in Table 1. Multiple random restarts are not considered here because the standard deviation in random split is relatively small, indicating that most peer methods are somewhat stable against random seeds.
Table Retrieval Accuracy upon NIST20 Dataset with scaffold split on positive adduct types.
Accuracy @ Top-k
1
2
3
8
10
Graff-MS (Murphy et al., 2023)
0,142
0.265
0,36
0,446
0,508
0.636
0.703
MassFormer (Young et al., 2024a)
0.178
0.318
0.422
0.506
0.568
0.706
0.768
ICEBERG (Goldman et al., 2024)
0.206
0.396
0.519
0.604
0.658
0.769
0.815
MARASON (ours)
0.283
0.464
0.567
0.645
0.700
0.814
0.853
B. Retrieval Accuracy with [M+H]+ Only
One design choice in MS/MS simulator development how many adduct types are supported by the model. Our MARASON aims to cover most adduct types the NIST (2020) database, while some methods e.g. FraGNNet (Young et al., 2024b) only support the most common adduct Hype-[M+]]- type as [M+H]+, where the retrieval accuracy is shown as follows in Table The performance of FraGNNet is quoted from their paper (therefore missing accuracies at 2, 4, 8, also missing error bars), and other baselines are implemented with our benchmark. It is worth noting that although Young et al. (2024b) report better retrieval accuracy of FraGNNet than all peer methods, our reevaluation of all methods on the [M+H]+-only subset shows that ICEBERG (Goldman et al., 2024) still outperforms. Incorporating collision energy for ICEBERG also improves the retrieval accuracy on [M+H]++ and our MARASON reaches state-of-the-ee retrieval accuracy.
is still feasible to compare these methods by restricting testing adduct
Table Retrieval Accuracy (mean 1.96 standard deviation three random seeds) upon NIST (2020) dataset with random (InChl key) split on the [M+H]+ adduct type.
Accuracy @ Top-k
3
8
10
Grall-MS (Murphy al., 2023)
0.211:10.004
0.365+00.009
0.472-H0.015
0.551:10.013 0.617:10.007
0.608=0.005
0.723+10.008
0.775
MussFormer (Young al., 2024a)
0.252-10.001
0.422:10.022
0.539-10.005
0.675=0.004
0.794-10001
0.843
FraGNNet (Young ul., 2024b)
0.238
0.504
0.652
0.831
ICEBERG (Goldman al., 2024)
0.251++00016
0.454+00.004
0.576+-0.0000.7118
0.654++0.004
0.711 0.011
0.810++0.001
0.850+0.009
ICEBERG collision energy)
0.270-10.016
0.487:10.009
0.611-10.011
0.679-10.013
0.735=0.013
0.840:10.009
0.877+0.008
MARASON (ours)
0.331:10.002
0.520:10.005
0.633-10.010
0.706:100005
0.754_0.002
0.849:10.003
0.885 0.000
13

---

Neural Graph Matching Improves Retrieval Augmented Generation in Molecular Machine Learning
C. Cosine Similarity
We include the detailed cosine similarity numbers from Fig. 3 for better reproducibility future works.
Table Experimental results in line with Fig. 3 of cosine similarity. Random split has mean #1.9 standard deviation across 3 random seeds.
Models
Random
Scaffold
Retrieved Spec
0.359
0.334
3DMoIMS (Hong et al., 2023)
0.51+000001
0.466
NEIMS (FFN) (Wei et al., 2019)
0.617=0.001
0.546
NEIMS (GNN) (Zhu et al., 2020)
0.694-10.001
0.643
FixedVocate (Murphy et al., 2023)
0.704=1.001
0.658
MassFormer (Young et al., 2024a)
0.721
0.682
SCARF (Goldman al., 2023)
0.726
0.669
ICEBERG (Goldman et al., 2024)
0.727
0,699
ICEBERG (w/ colli.)
0.735+0.005
0.711
MARASON (ours)
0.765-1.002
0.725
D. Evaluation of Matching Patterns: Visualizing Failure Modes of Traditional Graph Matching
For the qualitative analysis of matching pairs identified by MARASON, Fig. 5 in the main text visualizes only six fragment pairs due to space constraints. Here, ICEBERG--eneratee predicts 78 fragments for the target molecule, and we provide a complete list of all 78 matched fragment pairs. As shown in Fig. 6, the original target structure (left) and the original reference structure (right) are included, with fragments highlighted by their respective atoms and bonds.
In Fig. 6, the top 50 matched pairs, which exhibit higher similarity scores, predominantly correspond exact structural matches. In contrast, the lower-scored pairs include more fragments that, while structurally distinct, follow similar fragmentation pathways. Notably, the learned matching function in MARASON effectively identifies informative and chemically interpretable fragment-matchie strategy. This ensures that the retrieved intensity information is accurately assigned to the correct target fragments, thereby enhancing MS/MS simulation accuracy.
For matching pairs identified by RRWM (Reweighted Random Walk Matching) (Cho et al., 2010), which relies on fixed wraph-matching affinity metrics, greater number of unreasonable matchings occur, particularly when the fragment structures not identical (see Fig. 7). This issue is especially pronounced for pairs 65-78, where the similarity score falls below 0.2. The limitation of traditional graph matching arises from its fixed affinity metric: for identical fragments, it functions effectively, producing accurate matches; however, for non-identical fragments, it assigns noisy similarity scores, often categorizing them as matching "outliers" (fragments with low affinity scores to all others). As result, these fragments are incorrectly matched with unrelated structures.
Additionally, within the tail distribution of challenging fragment pairs, the learned neural graph-matching method yields higher similarity scores, demonstrating its superior ability to capture meaningful fragment correspondences.
14

---

Neural Graph Matching Improves Retrieval Augmented Generation in Molecular Machine Learning
Pair 2, 5imilarity:0.997
Pair 2, Simllanity-0.955
Pair 3, Simil a01
Pair 4, 5im
Pair 5, Similarit::0.995
Pair 6, Simil Similarit::09955
7, Similarity:0.98
Pair Similarity-0.975
Pair 9, Simil 0.55
10, Similarity:0.3e6
Pair 12, Similarityco.96
Pair Similarity.0.991
13, 95
Pair 54m
Pair Similarity00.93
Pair Simi Similarity00.92
Onif
Pair 19, Simliarityo0.912
Pair 20, Simllarttn0.885
Pair 21, Smndl 888
Pair 22, Similarityro.883
Pair 23, Similartty00.8s
Pair 24, Similarity0.868
Pair 25, Similarity:0.8G.
Pair 26, Similarttyo.885
Pair 27, Siml hamilatttto...555
Pair 28, Similarity:0.85e
Pair 29, Similaritys0.55
Pair 30, Simi lariityy00:0.
Similerity:0.037
Simileritys0.82
Similarity:0.811
5imilerity:0.005
Similerity:0.72
Simi
Pair Similarity:0.772
Pair Similaritys.7228
Pair 39, simi 701
Pair 40, Similarity:0.875
Pair 42, Similarityo0.673
Pair Simi Similarity00.627
Pair 43, Simliarity00.613
Poir 44, Similarlty90.608
Pair 45, Simil Samitorit::0057
Pair 46, Similarity:0.57
Pair 47, Similarityo0.56
Pair 48, Simi larit
Similarity0..66
Stmilarttys0.55
51, Stmilarity:0.527
Simllartty:0.525
Similarttys0.45
Pair Simil baitto0ity
Pair Simllarityo0.45
Pair S5m
57, Similarity:0.449
58, Simllarity:0.46
Pair Similarttyo0.435
Pair Simi la1ltto0t1
Pair 61, Simllerity:0.807
Pair 62, Similaribys.4044
Pair 63, Simllariy:0.44
Pair 64, Similarity:0.384
Pair 45, Similerity:0.379
Pair d6, Simi larityy3333
Pair Similarity:0.373
68, similaritys9.33
69, hamilaaity:o.344
70, Similarity:0.32e
Pair 71, Similarityso.az
Pair similarity:o.30r
73, Similarltygo.25
066
Simllarity:0.245
Pair Simllarity00.22
Pair Simi Similarityyo.115
On A
Figure 6. full list fragment pairs generated by MARASON, sorted by the learned Softmax scores. Structures the left are from the target fragmentation DAG and structures on the right are from the reference.
15

---

Neural Graph Matching Improves Retrieval Augmented Generation in Molecular Machine Learning
Pair 2, Similerity:1.0
Pair 2, Simileriby11.0
Pair 3, Similartyy1..
Pair 4, Similarity11.o
Pair 5, Similarity11.0
Pair Similarit:1..0
Similarityc1.0
Pair Similarity11.0
Puir 9, Similarty:1..0
Pair 10, Similarity:1.0
Pair Similarity11.0
Pair Simil erity:1
Pair Similarity11.0
Similarit:110
Similarty:110
Similarity:1.0
Pair Similarit:1.00
Pair Similarit:110
Pair 19, Simliarty11.0
Pair 20, Simlirnyy11.0
Pair 21, Similartty110
Pair 22, Simitarty11.0
Pair 23, Similanity1100
Pair 24, Similartyy10
Pair 25, Simllartty:1.0
Pair 26, Similarity11.0
Pair 27, Similarit::1.0
Pair 28, Similarity:1.0
Pair 29, Similarity1..0
Pair 30, Simllarty:1.0
On A
similarity:1.0
Similerit::1.0
Similarity110
Similarity:1.0
Similarity:1.0
similrity:10
37, similarity:1.0
Pair Similarit::100
Pair 39, Similarity11o
Pair 40, Similarity:1.0
Pair 41, Similarity:100
Pair similarit:110
Pair 43, Simllarity:1.0
Pair 44, Simllarit:1.0
Pair 45, Similarfty:10
Pair 46, Similarity:1.0
Pair 47, Similarit:1.0
Poir 48, Similarity:1.
Similarity11.0
Similarity1100
51, Stmilarity00786
Simllarity0.766
53, Similartty00.37
Pair larit:0600
Pair Simllarityo0.55.
Pair Similarttss0.55
57, Similarity:0.526
58, Similarity:0.473
Similarttys..tt
Pair Simil 1a52tto0...
AN
Pair 61, Simllerity:0.047
Pair 62, Similarity:0.4
Pair 63, Simi la:2ty:0.257
Pair 64, Similarity:0.222
Pair 45, Similerity:0.195
Pair d6, Simi lemitt:0000
AN- Ar-
Pair Similarity:0.137
68, similarits99.121
69, 100
Pair 70, Similarity:0.00e
Pair 72, Similarityso.os
72, Similarityy..oo
Simllarityo.003
Sim1larity9.029
Similarity:00029
Similarity:0.055
Simllaritys0.0
Pair Similariyyio.o
A
Figure 7. full list fragment pairs generated by RRWM graph matching (Cho et al., 2010), sorted by fixed Tanimoto similarity scores. Structures on the left are from the target fragmentation DAG and structures on the right are from the reference.
16


Reviews:
[{"review_id": "DT2MF3B9Lk", "reviewer": "Reviewer_FNZQ", "summary": "Authors present MARASON, which augments a previously-developed framework ICEBERG by retrieving the most similar molecules in a database to a target structure based on Tanimoto similarity. Both target and reference structures are fragmented using ICEBERG; a GNN is used to construct a matching matrix to predict which reference fragments are matches; these are then used to help predict peak intensities for the original target molecule. Authors evaluate their framework on retrieval accuracy, where the closest spectrum to the generated spectrum is retrieved based on cosine similarity, and show that their method outperforms previous baselines that do not use retrieval-based augmentations.\n\n### update after rebuttal\n\nThank you to the authors for their response. I am glad the authors were able to evaluate their method on MassSpecGym and the performance gain is notable given that the benchmark has harder train/test splits.\n\n> We treat spectra with different collision energies as distinct spectra.\n\nBased on my understanding, baseline methods first combine spectra from different collision energies into a single spectrum and work with that -- I think it would be good for the authors to disentangle this effect, or at least discuss it if relevant.\n\nI am happy with the rebuttal and will increase my score.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for agreeing with the novelty and technical solidity of our paper. We provide more random seeds, new MassSpecGym results, and MassFormer baseline with collision energy as per your comments. Please find our reply to your questions as follows.\n\n> Why does the scaffold split experiment have only 1 random seed?\n\nIn the ICEBERG paper, there is only one random seed with scaffold split. Following your suggestion, we run MARASON on 3 random seeds on scaffold split with the following updated results:\n|cosine similarity|top-1 acc|top-5 acc|top-10 acc|\n|-|-|-|-|\n|0.727±0.002|0.284±0.001|0.705±0.006|0.856±0.004|\n\nFrom the results, MARASON’s variance to random seeds is less significant compared to the accuracy improvement (for handy reference, ICEBERG w/ collision energy has a cosine similarity of 0.711) and the trend is consistent with either random or scaffold split. We will update the results in future revisions.\n\n> Request to evaluate on MassSpecGym\n\nAlthough the rebuttal time frame was quite short, we retrained our model and were able to get the following retrieval accuracies on MassSpecGym. We demonstrate the superiority of MARASON on MassSpecGym for candidates with the same formula\n|Top-$k$ accuracy|1|5|20|\n|-|-|-|-|\n|FraGNNet|31.93|63.20|82.70|\n|MARASON|34.03|64.04|85.39|\n\nWe will perform more benchmarking and update results in future revisions.\n\n> Questions on model design choices and the \"neutral loss\" embedding.\n\nWe would like to clarify that the model architecture was adopted from ICEBERG and we hypothesized that the GNN architecture of ICEBERG learns the structural information essential for graph matching. To elaborate on encoding the “neutral loss”, since $\\mathcal{M}, \\mathcal{F}_i, \\mathcal{M} - \\mathcal{F}_i$ represent precursor, ionized fragment, and neutral loss, respectively, subtracting the embedding of $\\mathcal{F}_i$ from $\\mathcal{M}$ represents the embedding of the neutral loss. We will elaborate on these details in the revised manuscript.\n\n>  Why are both the forward and reverse graphs needed in DAG embedding?\n\nWhen embedding the fragmentation DAG, the forward graph handles parent-to-children message passing and the reverse graph handles children-to-parent. We believe the update functions of both directions should be distinct to avoid degeneration into an undirected graph. Empirically, this strategy was more powerful than using bi-directional GNNs during our model development.\n\n> How different collisions are incorporated?\n\nWe treat spectra with different collision energies as distinct spectra. The collision energy value becomes another input dimension that is concatenated to the GNN input. In NIST retrieval experiments, we compute spectra at all collision energies, compare each one with its corresponding real spectrum, and compute the average cosine similarity over all recorded collision energies. The preliminary MassSpecGym experiments follow the official benchmark setting, which we will elaborate upon in the revised version.\n\n> Are the data splits identical to FraGNNet?\n\nAs discussed in the FraGNNet paper (Appendix I), their NIST benchmark follows MassFormer and ICEBERG to ensure fair comparison, which should align with our benchmark.\n\n> It's also possible to improve other models with RAG.\n\nWe absolutely agree with that; we select ICEBERG as the base model because it is the current open-source state-of-the-art. Another conclusion is that neural graph matching is vital for the success of RAG—just concatenating the reference spectrum to the neural network even harms the test performance, as shown in Table 2. Ultimately, we have demonstrated a RAG approach that can be readily adapted to other modeling tasks in the molecular spectroscopy domain. \n\n> Is it possible to improve other baselines with collision energy?\n\nYes. We applied the same design to MassFormer and validated changes to retrieval accuracies on  NIST’20:\n|Top-$k$ accuracy|1|5|10|\n|-|-|-|-|\n|MassFormer|19.1|55.0|71.6|\n|MassFormer (w/ collision energy)|20.9|59.6|76.4|\n|MARASON|27.8|68.5|82.7|\n\nIt shows a marginal improvement but still does not outperform MARASON.\n\n> Why did the authors choose 0.1 Da? Is it possible to demonstrate with more fine-grained binning?\n\nSince we want to compare all methods under the same metric, every model’s output should be transformed into the same mass resolution. 0.1 Da is the result of balancing between the mass resolution of HCD/Orbitrap spectra and the feasibility of implementing binned-prediction baselines. Empirically, experimentalists also find rounding to 0.1 Da resolution is sufficient in practice. MARASON (and ICEBERG) are adaptable with any mass resolution and can be run with higher resolution; however, selecting a finer-grained resolution will make binned-prediction methods unfeasible as a 0.01 Da resolution with a maximum mass of 1500 Da will require a 150,000-dim output in their neural networks. \n\n> Typo on page 3\n\nThanks for pointing this out. It has been fixed.", "claims_and_evidence": "The claims presented in the paper are well-supported in my opinion. Authors use the ICEBERG framework and test whether or not adding a module that incorporates information from fragments that are predicted to be matching from other molecules in a known database. They show that on the same data splits from NIST, incorporating this module increases the performance on both spectral similarity and retrieval.", "methods_and_evaluation_criteria": "Authors evaluate their method on NIST, which is a standard benchmark in the field. Authors use two data splits for training/eval -- random and scaffold, which is good. Authors use 3 random seeds for the random split but it seems like only 1 seed for the scaffold split -- why is this the case? Authors could also evaluate their method on more recent such as MassSpecGym [1], which would show the utility of the RAG component in harder split settings. \n\n[1] https://arxiv.org/abs/2410.23326", "theoretical_claims": "Authors don't make any theoretical claims.", "experimental_designs_or_analyses": "In Section 3.3.2, authors describe how they build a fragment embedding for a fragment F_i from a molecule M, although they concatenate a lot of information and it's not clear why it's all needed. For instance, why did the authors decide to incorporate the embedding difference between M and F_i as well as the chemical formula difference between M and F_i?  Secondly, in the hierarchical embedding, why are both the forward and reverse graphs needed, as opposed to using a bi-directional GNN? \n\nOtherwise, I think authors did a good number of ablations for their RAG strategy and also made the direct baseline comparison more fair to the baseline. However, could authors clarify how they incorporated the collision energy information? Were the spectra for each collision point treated as separate entires or where the combined into a single spectrum? The authors improved the ICEBERG baseline, but based on my understanding, it's also possible to condition other models like NEIMS and MassFormer on this information.", "relation_to_broader_scientific_literature": "The problem of MS prediction is an important problem for improving the characterization of molecules, especially given the sparsity of experimental characterizations. The performance metrics indicate that the problem is challenging (the best top-1 performance is 27%). While the method cannot be used to predict the structure of an unknown molecule, it can be used for this indirectly by predicting spectacular for unannotated molecules and then using cosine similarity.", "essential_references_not_discussed": "I am not 100% familiar with SOTA models/works in this domain, but to my knowledge authors are not missing essential references.", "other_strengths_and_weaknesses": "Strengths:\n- Incorporating a RAG module into the ICEBERG framework had a positive effect on all metrics considered. \n- RAG for MS has not been previously explored to my knowledge (but it has been explored in other molecule settings)\n- Multiple data splits done (random/scaffold)\n- The paper is well-written and easy to follow for the most part; figure 2 is helpful for clarifying the method\n- The paper was submitted to the applications track which I think is appropriate; I think it's an interesting application of existing methods and empirically shows a nice improvement.\n\nWeaknesses:\n- Evaluation on 1 dataset\n- Unclear explanation of how collision energies were incorporated\n- Only 1 seed for scaffold and RAG ablations", "other_comments_or_suggestions": "- page 3: \"non-bond\" → should it be \"non-bold\"?\n- Authors write that this represents \"neutral losses\" but don't explain what that is; it would be helpful to define this concept. \n- why did the authors choose 0.1 Da? Is it possible to demonstrate the method on a more fine-grained binning?", "questions_for_authors": "[1] How did you make your design choices for the fragment-level hierarchical embeddings of what information to include? It seems like a lot of information is concatenated but not clear why (see Experimental Designs Or Analyses for other notes)\n[2] Why is random splitting done on 3 seeds but scaffold/RAG ablations only on 1?\n[3] Is it possible to compare on a second benchmark like MassSpecGym? (it's not necessary to do this for me to change my evaluation of the paper, but I think it would strengthen the paper a lot)\n[4] Can authors clarify how they incorporated spectra of different collisions? Also, authors improved the ICEBERG baseline by incorporating multiple collisions, is it also possible to improve other baselines this way? \n\nIf authors can clarify their experimental protocol/choices I will be happy to revisit my evaluation.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EcInECT3Zo", "reviewer": "Reviewer_yTFf", "summary": "This study introduces MARASON, an advanced computational framework that enhances RAG in mass spectrum prediction through neural graph matching. It evolves from the ICEBERG framework through a synergistic integration of graph-based neural architectures and spectral alignment mechanisms.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for recognizing our state-of-the-art performance and our technical soundness. We update with experiments on the recently developed MassSpecGym benchmark, evaluation with non-similar reference structures, and elaboration on the ablation study to address your concerns. We are more than happy to clarify any further questions.\n\n> The model's training and evaluation were exclusively conducted using the NIST dataset, with no validation performed on alternative publicly accessible mass spectral libraries (e.g., MoNA).\n\nWe truly appreciate your suggestion. We have run our method on the recently developed MassSpecGym benchmark [1], which is a publicly accessible library with spectra collected from MoNA, MassBank, and GNPS. We share an initial result of its performance here:\n| Top-$k$ accuracy | 1 | 5 | 20 |\n| - | - | - | - |\n| FraGNNet | 31.93 | 63.20 | 82.70 |\n| MARASON | 34.03 | 64.04 | 85.39 |\n\nMARASON outperforms FraGNNet, the state-of-the-art on MassSpecGym, in terms of retrieval accuracy. We get the aforementioned result within the tight rebuttal time frame, and we will keep working on this benchmark with a comprehensive study in future revisions.\n\nA bit more clarification on why we focused on NIST: as discussed under “software and data” on page 9, NIST is the only database where all spectra have collision energy annotations. In most open-source MS/MS libraries, collision energy labels are at least partially missing, making it challenging to develop a RAG model with them. It is still a reasonable assumption to have access to collision energy values for prospective use because it is an experimental variable set on MS/MS instruments.\n\n[1] Bushuiev et al. MassSpecGym: A benchmark for the discovery and identification of molecules. NeurIPS 2024\n\n> Under circumstances where the mass spectral database lacks structural analogs, could the RAG strategy potentially lose efficacy due to failed similarity-based retrieval?\n\nThank you for bringing up this important point. MARASON benefits from RAG with Tanimoto similarity > 0.3 (90.7% of the testing set), under low Tanimoto similarity (< 0.3) regimes, the performance of MARASON is still close to the non-RAG baseline. \n\nWe group test instances based on the Tanimoto similarity between the retrieved structure and the target structure. We report cosine similarities (higher is better) as follows\n| Tanimoto similarity | (0, 0.1] | (0.1, 0.2] | (0.2, 0.3] | (0.3, 0.4] | (0.4, 0.5] | (0.5, 0.6] | (0.6, 0.7] | (0.7, 0.8] | (0.8, 0.9] | (0.9, 1] |\n| - | - | - | - | - | - | - | - | - | - | - |\n| MARASON | N/A | 0.550 | 0.614 | 0.690 | 0.741 | 0.789 | 0.815 | 0.808 | 0.805 | 0.824 |\n| MARASON (non-RAG) |  N/A | 0.566 | 0.611 | 0.682 | 0.727 | 0.768 | 0.791 | 0.780 | 0.759 | 0.784 |\n\nAll results are from a random split on NIST20. MARASON (non-RAG) is from the first entry in Table 2. This study also highlights a strategy for further performance improvements: use standard MARASON when the retrieved structure has Tanimoto similarity > 0.3 and use the non-RAG version otherwise. There is a trend of decreased performance for non-RAG MARASON when the Tanimoto similarity drops, because a lower Tanimoto similarity means there are fewer similar structures in the training set, making it a more challenging out-of-distribution instance. We will incorporate new results and discussions in future revisions.\n\n> Does the observed enhancement primarily stem from additional information introduced by spectral retrieval or from the structural alignment capability enabled by neural graph matching?\n\nThank you for the thought-provoking question. We are happy to elaborate based on our ablation study presented in Table 2.\n\nSimply retrieving the reference spectrum and concatenating it with the input to the neural network—a naive RAG-style approach—does not improve performance for MS/MS generation. In fact, it slightly degrades it: the resulting cosine similarity is 0.737, representing a 0.3% decrease compared to the non-RAG baseline.\n\nOur findings suggest that effective fragment-level matching is crucial for realizing the benefits of retrieval. For instance, applying a simple Hungarian matching algorithm raises the cosine similarity to 0.746, which is a 0.9% improvement over the baseline. Further, introducing a learnable neural graph matching module with carefully designed architecture yields a cosine similarity of 0.757, amounting to a 2.4% relative improvement over the non-RAG model.\n\nWhile gains in retrieval accuracy are even more substantial—partly because the cosine similarity metric is more saturated—the top-1 retrieval accuracies indicate that there remains considerable room for improvement. \n\nIn summary, the structural alignment capability enabled by the neural graph matching module is the primary driver of MARASON's performance improvement.", "claims_and_evidence": "The claims are clearly supported.", "methods_and_evaluation_criteria": "The method is technically sound.", "theoretical_claims": "The paper focuses on application of AI method, and no proof is needed.", "experimental_designs_or_analyses": "The experimental designs are sound.", "relation_to_broader_scientific_literature": "The contributions of this paper are novel and origin.", "essential_references_not_discussed": "References are sufficient.", "other_strengths_and_weaknesses": "Strengths:\n-  The integration of RAG and neural graph matching under the ICEBERG framework synergistically enhances both prediction accuracy and generalization capability, establishing a new state-of-the-art performance benchmark.\n- The paper is clear and easy to follow.\n\nWeaknesses:\n- The model's training and evaluation were exclusively conducted using the NIST dataset, with no validation performed on alternative publicly accessible mass spectral libraries (e.g., MoNA) or laboratory-curated in-house datasets.", "other_comments_or_suggestions": "See questions.", "questions_for_authors": "1. Under circumstances where the mass spectral database lacks structural analogs, could the RAG strategy potentially lose efficacy due to failed similarity-based retrieval, thereby reverting to a purely de novo generation mode dependent on deep learning architectures?\n2. The MARASON framework demonstrates significant performance enhancement over the baseline (ICEBERG) in mass spectrometric analysis. I wonder whether the authors could elucidate the respective contributions of its dual modules. Specifically, does the observed enhancement primarily stem from additional information introduced by spectral retrieval or from the structural alignment capability enabled by neural graph matching?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "N/A"}}, {"review_id": "GfJkSoGmUl", "reviewer": "Reviewer_VQAR", "summary": "This paper proposes a modification of a method for generating mass spectra from molecular structures. Inspired by recent successes in retrieval augmented generation (RAG), the authors decided to apply this technique for querying similar molecules from the training set and to use them as references for generating spectra with a model that extends the previously proposed ICEBERG. The proposed model, named MARASON, retrieves the nearest reference molecule along with three of its spectra to construct a representation vector used for predicting peak intensities. A neural graph matching algorithm is introduced to align the fragment graphs in the fragmentation DAG of both the reference and target molecules. MARASON surpasses other methods on both random and scaffold splits in predicting mass spectra. Additionally, it demonstrates excellent performance in compound retrieval using mass spectra.\n\n## update after rebuttal\n\nThe Authors addressed all my comments. I decided to maintain my positive score.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We truly appreciate your recognition of our contribution to the mass spectrometry field and our technical novelty of introducing neural graph matching. We conduct statistical tests and perform preliminary MassSpecGym experiments following your suggestions. We will work actively to complete the new results in future revisions.\n\n> Conducting statistical tests for the results described in Section 4.2.1 will make claims stronger. \n\nThank you for the suggestion. We first generate results with 3 random seeds for the scaffold split and perform a t-test on ICEBERG (w/ collision energy) and MARASON. The P-values are 0.005 and 0.002 for random split and scaffold split, respectively, both above the 95% confidence interval of statistical significance.\n\n> It would also be interesting to see examples of predicted spectra compared to the original spectra and the spectra of the reference compound. For reference, similar qualitative results were presented in earlier works. Such plots would demonstrate how similar the predicted and reference spectra are.\n\nThank you so much for the suggestion, unfortunately, we do not have the option to include visualization results with Openreview. We will add the visualization of reference spectra, predicted spectra, and ground-truth spectra in the appendix in future revisions.\n\n> To enhance this paper, an evaluation of a second dataset is recommended.\n\nWe truly appreciate your suggestion. We have run our method on the recently developed MassSpecGym benchmark [1], which is a publicly accessible library with spectra collected from MoNA, MassBank, and GNPS. We share an initial result of its performance here:\n| Top-$k$ accuracy | 1 | 5 | 20 |\n| - | - | - | - |\n| FraGNNet | 31.93 | 63.20 | 82.70 |\n| MARASON | 34.03 | 64.04 | 85.39 |\n\nMARASON outperforms FraGNNet, the state-of-the-art on MassSpecGym, in terms of retrieval accuracy. We get the aforementioned result within the tight rebuttal time frame, and we will keep working on this benchmark with a comprehensive study in future revisions.\n\n[1] Bushuiev et al. MassSpecGym: A benchmark for the discovery and identification of molecules. NeurIPS 2024\n\n> Section 3.2.2 says: \"All reference intensities at the same collision energy are processed by a set transformer, followed by an average pooling layer that merges intensity embeddings per fragment from three collision energies.\" I understand that there are three different energies for the reference compound, yet here processing intensities at the same collision energy are described. Could you elaborate on this process in more detail? What vectors serve as input to the set transformer?\n\nYou are right about how we process reference spectra at each collision energy. Our purpose of using three reference spectra is that the target collision energy may not have a close match energy in the reference database and multiple reference spectra could be considered together to interpolate. We first concatenate spectral peaks with their corresponding collision energies and the target collision energy. After that, we feed the concatenated spectral vectors into a set transformer and eventually a linear layer, whereby our design of the linear layer is to “move” the peak intensities to the target energy level. Finally, we use average pooling to collect information from the same peaks at different energy levels to generate a reference spectrum embedding that is a learned interpolation from three energies.\n\n> How is this model trained? Are there any new components in the loss function? To train neural graph matching, do you use any additional training steps, or is it trained alongside intensity prediction as part of the vector in Equation 7?\n\nThe first-stage model that generates fragments is trained in the same way as ICEBERG-Generate. The second-stage model that predicts intensities is trained end-to-end, where the cosine loss between predicted spectra and ground truth spectra is the only supervision. We are able to make such a design choice because being fully differentiable is one of the major advantages of neural graph matching.", "claims_and_evidence": "The claims made in the paper are supported by experimental evidence. The method can generate mass spectra more accurately, even in the scaffold-based split setup (Figure 3). Moreover, Table 1 shows that retrieval accuracy is also improved over earlier methods. The ablation study confirms the effectiveness of both RAG and neural graph matching.", "methods_and_evaluation_criteria": "The methods and evaluation criteria are adequate to solve the described problem. However, some parts of the method description could be more detailed, especially given that the code is not yet available. I highlighted these parts in the Questions for Authors.", "theoretical_claims": "There are no theoretical claims that need formal proofs.", "experimental_designs_or_analyses": "The experiments answer the posed research questions. What would make the claims in the paper stronger, would be conducting statistical tests for the results described in Section 4.2.1. It would also be interesting to see examples of predicted spectra compared to the original spectra and the spectra of the reference compound. For reference, similar qualitative results were presented in earlier works. Such plots would demonstrate how similar the predicted and reference spectra are.", "relation_to_broader_scientific_literature": "This paper not only presents a significant contribution to mass spectra prediction, surpassing earlier models, but also demonstrates that RAG can be effectively utilized in the molecular domain. Additionally, neural graph matching is shown to be more effective than classical matching methods for processing fragmentation DAGs.", "essential_references_not_discussed": "All the key references have been discussed.", "other_strengths_and_weaknesses": "Most of the comments have been addressed in the other sections. Furthermore, I appreciate that Figure 2 offers a clear overview of the proposed method. To enhance this paper, an evaluation on a second dataset, such as NPLIB1, is recommended.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "1. Section 3.2.2 says: \"All reference intensities at the same collision energy are processed by a set transformer, followed by an average pooling layer that merges intensity embeddings per fragment from three collision energies.\" I understand that there are three different energies for the reference compound, yet here processing intensities at the same collision energy are described. Could you elaborate on this process in more detail? What vectors serve as input to the set transformer?\n2. How is this model trained? Are there any new components in the loss function? To train neural graph matching, do you use any additional training steps, or is it trained alongside intensity prediction as part of the vector in Equation 7?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hsNMWTrLLV", "reviewer": "Reviewer_HsCb", "summary": "This paper introduces a method, MARASON, for predicting a mass spectrum from molecular graphs. MARASON extends an existing deep learning framework (ICEBERG) by integrating retrieval with neural graph matching. The model retrieves reference molecules with known spectra, then aligns fragments between the target and reference molecules to guide its prediction of the target’s spectrum. Experimental comparisons show that MARASON outperforms current state-of-the-art methods in terms of accuracy and retrieval performance.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for recognizing the state-of-the-art accuracy of MARASON and the organization of this paper. We believe there are several misunderstandings and we clarify them as follows.\n\n> The paper does not describe a generation process and does not utilize RAG.\n\nThe problem MARASON aims to tackle is _generating_ MS/MS from a given molecular structure. On the technical side, our first-stage model is an _auto-regressive generator_ that predicts one bond-breaking event at each step.\n\n> The paper does not clarify which specific type of mass spectrometry it supports. \n\nWe will emphasize that MARASON is developed with ESI-MS/MS in the main text. As discussed in section 4.4.1, “We trained our models on the NIST 2020 dataset with 530,640 high-energy collision-induced dissociation (HCD) spectra and 25,541 unique molecular structures.” We believe we made it clear that our experiment setting contains MS/MS spectra (more specifically, Orbitrap spectra) with an instrument type label “HCD” in the NIST database. \n\n> Please report separately the results on the FT-HCD dataset and the FT-CID dataset.\n\nTo clarify, as discussed in section 4.4.1, we only use spectra labeled as “HCD” instrument type in NIST. We do not find any entries with instrument type labels ``CID`` or ``FT-CID`` in NIST. We believe we are working with an open-source training and testing framework that supports most baselines and all methods are trained and tested on the same dataset, ensuring fair comparison.\n\n> Questions on model architecture details and GNN layers.\n\nWe would like to clarify that MARASON adopts the same backbone architecture as ICEBERG, including both the GNN and transformer layers.\n\nRegarding the GNN blocks shown in Figure 2: the first and third GNNs operate on molecular graphs and are structurally identical to their counterparts in ICEBERG. The second GNN, discussed in Section 3.3.2 under “DAG Hierarchical Embedding Learning,” serves as the neural graph matching module. While all GNN blocks share the same architecture, our strategy involves using separate weights depending on the module's function—i.e., whether it is used for intensity prediction or for graph matching.\n\nWe will include more detailed discussions and highlight such differences in Figure 2 in future revisions. \n\n> I recommend citing additional references for the correlation between spectrum and structural similarities.\n\nThank you for the suggestion, we plan to include the following references on molecular networking. Please also let us know if there are any specific references in your mind.\n* Aron AT et al. Nature protocols. 2020\n* Wang M et al. Nature biotechnology. 2016\n\n> MoMS-Net and 2DMolMS are missing.\n\nWe do not find any open-source implementation for MoMS-Net and the dataset statistics in Table A1 are different from ours, making it challenging to include MoMS-Net for comparison. We do not find any references to 2DMolMS—if you meant 3DMolMS, it is the first entry in Figure 3 and Table 1.\n\n> How to prevent data leakage is unclear.\n\nThanks for the suggestion. One possible concern of running RAG on MS/MS simulation is that the retrieved spectra might be considered as a source of data leakage that simplifies the problem. Therefore, in numerical evaluations, we restrict retrieving spectra only from training data; in real-world cases, there is no doubt about using any reference spectra available for MARASON. We will elaborate in future revisions.\n\n> Why retrieve three reference spectra with different collision energies?\n\nSince the reference spectra may not have the exact collision energy of interest, MARASON tries to interpolate from three collision energies. We did not try including more references because the more reference spectra we have, the more computational overhead there will be.\n\n> Does this framework explicitly account for experimental variables?\n\nOur framework handles ionization mode by different adduct types. We incorporate collision energy as an important instrument variable. For others, we assume they are the same across Orbitrap instruments. If any important factors are missing, please let us know and we would be more than glad to address them in future work.\n \n> Testing on NIST23 for results with a larger dataset\n\nWe purchased the NIST23 license, but the raw data file (.SDF) we used for NIST20 is not available at least in the NIST23 distribution we purchased. We will certainly explore NIST23 after figuring out how to extract training data from it. \n\nWithin the timeframe of rebuttal, we try to address this concern from another direction by showing that MARASON achieves state-of-the-art retrieval accuracy on MassSpecGym (please refer to the table in response to reviewer VQAR).\n\n> Recommend to elaborate a bit more on Supplementary Material.\n\nAs we cannot edit the supplementary materials at this time, we will try to include more details in future revisions. If there are any specific points that feel unclear to you, please let us know.", "claims_and_evidence": "Most of the paper’s claims are backed by evidence. However, the reference to retrieval augmented generation (RAG) is misleading because the paper does not actually describe a “G (generation)” process. It is unclear whether this approach truly utilize RAG.", "methods_and_evaluation_criteria": "Please report separately the results on the FT-HCD dataset and the FT-CID dataset.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "Mostly", "relation_to_broader_scientific_literature": "N/A", "essential_references_not_discussed": "The paper is motivated by the assumptions that “similar structures tend to have similar fragmentation patterns in chemistry, and similar fragments tend to have similar response factors that relate their abundance to the observed intensity”, citing only (Shahneh et al., 2024). However, there is a lot of literature in computational mass spectrometry and cheminformatics that specifically discusses the correlations between structural similarity and fragmentation outcomes. I recommend citing additional references to support this idea.\n\nMoMS-Net and 2DMolMS are missing.", "other_strengths_and_weaknesses": "Strengths:\n• Clear discussion of background/previous work/motivation.\n• Clear explanation of relative concept and description of modeling approach.\n• The tables presenting results are well-constructed and straightforward. The ablation studies are comprehensive.\n• Manuscript is clearly written and easy to follow\n\nWeaknesses:\n1. The paper presents a framework for mass spectrum simulation but does not clarify which specific type of mass spectrometry it supports. While ICEBERG is trained on NIST’s tandem mass spectral (MS/MS) data, the reference “NIST20” in this paper does not explicitly stating whether it is electron ionization (EI) data, tandem MS, or another type of mass spectral library. It would be helpful if the authors clarified the applicability of their method to different mass spectrometry modes—such as MS/MS, EI, or GC-MS and provided evidence or discussion on its performance across these various platforms.\n2. There is lack of discussion about the architecture of the GNN and other layers used in the framework. It is unclear whether they are identical to the GNN block in ICEBERG. Do all the GNN blocks share the same structure in FIG. 2? Section 4.2.3 suggests that separating these two GNNs leads to better spectrum similarity, implying that they may have distinct architectures. To address this weakness, I recommend including a more detailed description of the GNN modules, MLP blocks and the matching layers blocks in the Supplementary Information and updating FIG. 2 accordingly.", "other_comments_or_suggestions": "None", "questions_for_authors": "1. In lines 209–211, you mention “in our experiments, we use the training dataset to mitigate concerns about data leakage.”, which is unclear how the data leakage is prevented. I suggest rewriting this sentence to elaborate how this procedure prevents leakage.\n2. In lines 217–219, you mention retrieving up to three reference spectra with collision energies similar to the target. How was the number three selected, and have you investigated how performance or computational efficiency might change if this limit were increased?\n3. In section 3.4 , when predicting the intensities, you rely on learned embeddings (H, Hr), reference intensities (Tr), and the matching matrix (X̄), along with a Tanimoto similarity measure. However, in real-world mass spectrometry, many experimental factors (e.g., ionization mode, instrument settings) can affect fragment intensities. Does this framework explicitly account for these variables, or do you assume consistent conditions across the NIST dataset?\n4. NIST released an updated tandem mass spectral library (NIST23), which includes around 60% more compounds than NIST20. I understand that the NIST database is not freely available, but, if possible, would you consider re-running the experiments on NIST23 to see how the results generalize with a larger dataset?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The rebuttal clarified a few concerns of the reviewers and provided further experimental results. After considering these, all reviewers recommend to accept the paper and particularly appreciate the comprehensive ablation study.
Decision: Accept (poster)