Prefix-Tree Decoding for Predicting Mass Spectra from Molecules
Authors: Samuel Goldman
John Bradshaw
Jiayi Xin
Connor W. Coley
Venue: NeurIPS 2023 spotlight (2023)
Keywords: molecules
prefix tree
mass spectra
mass spectrum prediction
metabolomics
GNNs
chemistry
biology
TL;DR: Predicting mass spectra from molecules by first predicting molecular formulae (factorized as an autoregressive prefix tree generation task) and secondly estimating intensities at each formula peak (set2set transformer).

Abstract: Computational predictions of mass spectra from molecules have enabled the discovery of clinically relevant metabolites. However, such predictive tools are still limited as they occupy one of two extremes, either operating  (a) by fragmenting molecules combinatorially with overly rigid constraints on potential rearrangements and poor time complexity or (b) by decoding lossy and nonphysical discretized spectra vectors. In this work, we use a new intermediate strategy for predicting mass spectra from molecules by treating mass spectra as sets of molecular formulae, which are themselves multisets of atoms. After first encoding an input molecular graph, we decode a set of molecular subformulae, each of which specify a predicted peak in the mass spectrum, the intensities of which are predicted by a second model. Our key insight is to overcome the combinatorial possibilities for molecular subformulae by decoding the formula set using a prefix tree structure, atom-type by atom-type, representing a general method for ordered multiset decoding. We show promising empirical results on mass spectra prediction tasks.

Prefix-Tree Decoding for Predicting Mass Spectra from Molecules
Samuel Goldman
John Bradshaw Chemical Engineering MIT
Computational and Systems Biology MIT
Cambridge, MA 02139 samlg@mit. edu
Cambridge, MA 02139 jbrad@mit. edu
Jiayi Xin
Connor Coley
Statistics and Actuarial Science The University of Hong Kong Pokfulam, Hong Kong xinjiayi@connect hku hk
Chemical Engineering
Electrical Engineering and Computer Science
MIT Cambridge, MA 02139 ccoleyêmit.. edu
Abstract
Computational predictions of mass spectra from molecules have enabled the dis- covery of clinically relevant metabolites. However, such predictive tools are still limited they occupy one of two extremes, either operating (a) by fragmenting molecules combinatorially with overly rigid constraints on potential rearrangements and poor time complexity (b) by decoding lossy and nonphysical discretized spectra vectors. In this work, we use a new intermediate strategy for predicting mass spectra from molecules by treating mass spectra sets of molecular formulae, which are themselves multisets of atoms. After first encoding an input molecular graph, decode dicted peak in the mass spectrum, the intensities which are predicted by second model. Our key insight is to overcome the combinatorial possibilities for molecular subformulae by decoding the formula set using a prefix tree structure, atom-type by atom-type, representing a general method for ordered multiset decoding. We show promising empirical results on mass spectra prediction tasks.
of molecular subformulae, each of which specify pre-
Introduction
Input mixture
As the primary tool discover unknown small molecule structures from biologi- cal samples, tandem mass spectrometry (MS/MS) experiments have enabled the iden- tification of numerous important molecules implicated in health and disease 4 38 55].
Collision cell
Mass spectrum
Co... 00
Intensity
COM
5,0,H
COM
Tandem mass spectrometers are capable of isolating, fragmenting, and measuring the re- sulting fragment masses small molecules from a sample, producing signature (a mass spectrum) for each detected molecule (Figure I top).
M/Z
Learned simulator SCARF
M Precursor molocule
Tandem mass spectrometers measure frag- mentation patterns of molecules, resulting in character- istic peaks that are indicative of their structure. SCARF simulates these fragmentation patterns in silico.
Figure 1:
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Computationally predicting mass spectra from molecules in silico (Figure bottom) is thus longstanding and important challenge. Not only does this assist practitioners in better understanding the fragmentation process, but it also enables the identification of molecules from newly observed spectra by comparing an observed spectrum to virtual spectra generated from a datahase of candidate molecules. While a large library of empirical mass spectra could theoretically serve the same purpose, the size of such libraries limited by the slow and expensive process of acquiring pure chemical standards and measuring their spectra, motivating computational prediction.
We argue that there are three core, interrelated desiderata for a forward calecule-to-spectnie simulation model, or "spectrum predictor". An ideal spectrum predictor should be (i) accurate, being able to predict the exact set of fragment masses and intensities with a precision comparable to experimental measurements; (ii) physically inspired. to avoid making physically nonsensical ("invalid")) suggestions and to provide interpretations of the chemical species responsible for each peak for the benefit of human expert chemists; and (iii) fast, such that it is computationally inexpensive to predict spectra for many (e.g., millions) hypothetical molecules.
Unfortunately, many existing spectrum predictors do not meet these criteria. Methods to date have tended follow one of two approaches: (a) physically motivated fragmentation approaches or (b) cooeeuue-to-veector (or "binned") approaches (Figure 2A-B). Fragmentation approaches ge.g.. Figure _А) take an input molecule and suggest bonds that may break, creating fragments that are scored by ML algorithms or curated rulesets. While interpretable, these methods are often slow and restrictive; certain mass spectrum peaks are generated by complex chemical rearrangements within the collision cell that cannot be approximated by bond breaking alone. That is, the bonds in observed fragments are not a subset of those in the original molecule 1 [_. On the other hand, binned prediction approaches (e.g., [500 55 53 Figure 3) are less physically grounded, using neural networks to directly learn a mapping from molecules to vectors representing discretized versions of the spectra. These methods, while fast, lack interpretability and due to discretization have a mass precision lower than that of most modern spectrometers, limiting their accuracy.
B0
We propose to address the shortcomings of previous work by predicting mass spectra from molecules the level of molecular formulae (e.g. C. and introduce new method, Subformulae Classification for Autoregressively Reconstructing Fragmentations (SCARF) to do so. Because the molecular formula for each input molecule known, each subformula in the predicted set of peaks constrained to contain subset of the atoms in the original formula. Our primary contributions are:
posing mass spectrum prediction as two step process: first generating the set of molecular formulae for the fragments, then associating these formulae with intensities;
overcoming the combinatorial subformula option space by learning generate formula prefix trees;
demonstrating the empirical benefit SCARF in predicting experimental mass spectra quickly and accurately using two separate datasets, providing a benchmark for future work.
2 Background
We provide short introduction of tandem mass spectrometry suitable for a general machine learning audience, detail previous approaches to modeling this process as they relate our proposed approach SCARF, and explain how such tools can utilized discover molecules from new spectra. We refer interested readers to 2 for further details on the physical process of mass spectrometry.
2.1 Tandem mass spectrometry
Tandem mass spectrometers (MS/MS) measure fragmentation patterns of molecules in multi- process. The input to the process a solution containing a precursor molecule, M E X, associated with molecular formula, FF, defining the counts of each element present; for instance F = H,H2, for the precursor molecule shown in Figure [ The precursor molecule is first ionized (i.e., made charged), often by bonding associating with an adduci (e.g., proton, Ht) present in the solution. The charged product is then measured by a mass analyzer (MS1), where its mass-to-charge ratio (m/ 2) is measured.

---

Fragmentation prediction
B
Binned prediction
C Formula prediction
Step SCARF-BR Threeu
intemsitt outpett
[OO, CON Cop, C.O.M. cop
Intensity
Intensitty
Step 2: SCA RF- Woree
CG CaN
50 5.0%
Intersity
M/Z
M/Z
C,O,H,
Figure 2: Overview of various approaches to spectrum prediction. A. Fragmentation prediction approaches use heuristics and scoring rules to break down the molecule into fragments and their associated intensities. B. Binned prediction approaches discretize the possible mass-to-charge values and predict intensities for each possible bin. C. Formula prediction approaches predict spectra as sets of molecular formulae and intensities. Our model SCARF utilizes two stage approach, first by predicting the product formulae present (constrained by the precursor formula), which defines the x-axis locations of the peaks, before secondly assigning intensities these formulae (defining the peaks' y-axis values).
This precursor ion is then filtered into a collision cell. Here, through interactions with an inert gas, the precursor ion is broken down into a set of one or more product ions, each which is associated with new chemical formula; for example, one might be f C,OH, for the process shown in Figure I Finally, this set of product ions is measured by second mass analyzer (MS2), along with the set of their intensities, y R (i.e., their relative frequencies over several repetitions of this process), creating for each ion what referred as a peak. The collection all peaks makes up a molecule's mass spectrum, and is commonly represented as a plot of intensities versus m/z (Figure right).
2.2 Predicting mass spectra from molecules (spectrum predictors)
Fragmentation prediction. A complex but physically grounded strategy is to model the bond breakage processes occurring in the collision cell (Figure A Examples include MetFrag [52). MAGMa 40, and CFM-ID (2). which recursively fragment molecules (either bond or atom removals) generate fragment predictions. These methods combine expert rules and local scoring methods to enumerate molecular fragmentation trees to predict spectra. CFM-ID [2] learns subsequent fragmentation transition probabilities between fragments with an expectation maximization algorithm to determine intensities at each fragment. Rule- bale-based methods and full tree enumeration reduce the flexibility of these approaches, and along with the inherent ambiguity the fragmentation process, limit this strategy's overall accuracy and speed.
Binned prediction. An increasingly popular and straightforward approach to spectra prediction to map molecules discretized 1D mass spectra from either molecular fingerprint 500 or graph inputs [355 58) (Figure B). Specifically, these methods divide the m/2 axis into fixed-widis width "bins" and predict an aggregate statistic of the peaks found in each bin (such their maximum summed intensity). While more flexible and end-to-end than fragmentation-bune based approaches, these methods do not impose the same physical constraints shared information across fragments, making them less interpretable and susceptible to making invalid predictions. Further, discretizing the input spectrum inherently restricts the precision of such models compared to exact-mass predictions.
Formula prediction. We introduce the strategy of predicting spectra at the level of molecular
formulae, an intermediate between binned and fragmentation prediction (Figure 2C). Simultaneous
to our work. two groups have separately explored formula prediction strategies 62! [59 However, to
generate plausible subfornulae candidates, they either generate fixed vocabulary formulae [34]
or restrict their model to molecules under 48 atoms for exhaustive enumeration [59] which is smaller than many compounds of interest. We overcome the combinatorial problem of formula generation using prefix trees, allowing method to scale and eliminating the need for large, fixed vocabularies.
3

---

2.3 Mass spectrum libraries
One important use of spectrum predictors is in building large in silico libraries of molecule spectra to augment the small size of existing, experimentally derived databases (on the order of 10 which are expensive to curate. These spectra libraries are then leveraged downstream in different ways, for example for training molecular property predictors directly from mass spectra 46]. Another common application of spectra libraries to infer an unknown molecule's structure from a newly observed spectra particularly hard problem, with only 13% of spectra measured from clinical samples identifiable using current elucidation tools [6. In this problem, spectra libraries are used as part of process called retrieval: The newly observed spectra compared with the existing spectra in the library using a fixed or learned spectral distance function, such as cosine distance 24] and the molecules associated with the closest spectra are returned possible matches. In practice, the retrieval process is constrained to choosing among isomers (i.e., molecules with the same molecular formula, and therefore molecular weight, but with different bond configurations) due the high resolution of modern mass spectrometers (i.e., absolute errors on the order of 10 to 10 m/2 for MSI measurements)
53
Given the varied use cases of spectra libraries, we focus on evaluating spectrum predictors in terms of both (a) their prediction accuracies ($4.2), using metries such "cosine similarity", and (b) their use in generating virtual spectral libraries to assist with retrieval ($ 44.3).
3 Model
Here, we describe model, SCARF, for predicting mass spectra from precursor molecules via first predicting subformulae of the precursor molecule, referred to as product formulae. Building upon the notation introduced the previous section, we continue denote precursor molecules as M X, and their associated formula vector as F E Ng. defining at each position, j E {l..... , the count of each possible chemical element present, FF (with zero indicating none of that chemical element is present). Likewise, we define the set of n. product formulae {f*) )== and associate with each an intensity, y'. Note that the mass corresponding of the peak on a mass spectrum) is determined deterministically from the counts of each elements present.
given formula (and, as such, the x-axis location
At a high level, SCARF generates mass spectra through the composition of two learned functions:
Neave
Thread
(M).M).
first mapping from the original molecule to a set of product formulae, 97 Thread . + {fr) and then mapping from this set of formulae (and the original molecule) the respective intensities, The particularities of both functions are descrihed in detail below. The specific architectures and hyperparameters used are deferred to the appendix; model code can be found at battps://githnn com/ s samgol @amgoldman97. dman97 /
go Neave [(f'))__,M, M) - {(f), r²)}"
3.1 SCARF- -Tread: Generating product formulae via generating prefix trees
SCARF-Thread is tasked to learn mapping to the set of produet formulae, { given the original molecule. Naively, one might try to define this model autoregressively. predicting the set formula by formula, chemical element by chemical element. However, such an approach soon runs into a number of problems as (i) the predictions are not invariant to set and ordering permutations; (ii) the time complexity of prediction would scale poorly, being proportional to both the number of elements and number of product formulae (i.e., O(e n)); and (iii) the predictions would likely contain duplicates.
'We model and discuss uncharged molecules and formulae, despite mass spectrometry measuring the masses of adduct ions. In practice, we reduce all molecules to uncharged candidates by simply shifting all the spectra weights by the m/2 22 of their respective adducts, which assume be equal to the (known) adduct of the parent molecule.
We assume singly charged adducts (as is common practice, 22.. such that masses and mass possstt-coe co-charge ratios are interchangeable.

---

A
B Next element count prodiction. given eurrent prefix and input molecule, M
Product formulae prefix tree
C,O,H,
F
C,
each possible continvation
C,O,H,
input context voctor; c² (comentenatedy
O, of the yeufee
is prodicted us binory lobel
Go. CO
O2
root.
C.H,
gnn(Ar)
c
C.H. C,O.H,
counts(C,...)
C,
O
countslyF C.,))
C,O.H, H
C,
O
ono-hat(O)
SCARF- -hrread
snoubd commis are masked
during inference each level the tvee (kere, the orygen cosntt) exponded pasuilel, multi-late binory classification task (right)
prediction path for f C,0,1,
NN
O
Figure 3: Illustration of the SCARF-Thread architecture. A. The formulae of the product fragments can be represented using prefix tree. SCARF-Thread predicts this tree for new molecules at test time. It does so by expanding each node at a given depth in parallel, treating the counts of subsequent elements dependent only on the counts of elements predicted so far (i.e., the prefix) and the original molecular structure. B. The SCARF-Threat predictive task at the C7 node from the prefix tree diagram shown in A. Here the network takes as input (i) an embedding of the overall molecule; (ii) vector representing the counts of each element in the prefix so far (counts yet to be predicted are represented using a special token), (iii) the difference of the counts predicted so far from the precursor molecule, and (iv) one-hot representation of the element for which the counts are currently being predicted. The network predicts which counts are valid next nodes in the prefix tree (where counts that are greater than those in the original precursor molecular formula are automatically masked out as invalid). See also Alg.
We therefore take a different approach using the insight that the set of all product formulae can be compactly represented as a prefix tree (Figure A). In this tree, edges at given depth represent valid counts of a particular chemical element, which are often identical across multiple product formulae (shown in the circles). By following each path from the root node to the different leaf nodes, we can reconstruct each product formula (as the orange dashed path does for a single product formula).
We thus propose SCARF- -hheead as an autoregressive generator to define probability distribution over such a prefix tree (Alg- I.II. We assume that each product formula is a subset of the precursor formula, meaning that the precursor formula sets an upper bound on the maximum number of each elemen At each node in the tree (corresponding of child nodes (corresponding to the set of valid counts of the subsequent element) as a multi-label binary classification problem (Figure B). Concretely, we use neural network module for this task, giving it as input a context vector representing the node being expanded:
prefix f22), we pose the prediction of the set
é' = [g),(M), counts f counts(FF - f22)) one-hot(j)],
where gnn(-) specifies a neural encoding of the molecular graph ($A.5.2 counts(-) specifies a count- based encoding of the associated prefix (A..5.). and one-hot(-) specifies one-hot encoding of the node's 's depth (or equivalently, which element the predicted count is for). In our experiments, we use a fixed ordering of the chemical elements ($ (A..2). but optimizing even learning the tree construction order could be carried out 45
Formulae differences. Following Wei et al. 50) we find it helpful to not only parameterize product formulae in terms of their element counts, but also in terms of the elements that they have lost,
'While it is possible for fragments to fuse together, potentially taking the count of a chemical element over the number in the original precursor formula, we postpone the extension to modeling such rare events to future work.
5

---

i.e., their difference from the precursor formula. On the input side. this is already covered by including in the context vector count-bert based embedding of the prefix formula minus the product formula (counts( (founts(Fff=3)) -2z)), However, on the output side this is achieved by combining the probabilities of "forward" and a "difference" network:
plf alffy. a.d (MLP"(c')).
- aa)a (MLP (c))
where MLPF()) (.) and (.) specify multi-layer perceptrons (MLPs) for predicting the probability of observing a count of a and a loss of F, atoms respectively; or is a variable (output from third, unshown network) deciding how to weight these predictions; and o (-) is the element-wise sigmoid function.
3.2
SCARF Weave: Predicting intensities given product formulae
Given the product formulae outputs from SCARF- Thread, SCARF- -eave predicts corre- sponding intensities at each formula. This is set-to-set problem, well suited for any equivari- ant set2set architecture [56] $3.1 In experi- ments, we use a Set Transformer 29 enables the model consider all the formulae present in the mass spectrum (and their possible interactions) when predicting final intensities.
inpul contert verlor c' (((ooccalenaled)
gnn(M)
counts(
counts F-
C,O.H. C,O.H
which
C,O.H
C,H,
SCARF- Weave" Trunsformer
We choose to represent formula the set similarly to the context vectors used in SCARF-Thread. put,
product formulae embeddiings
ponk intensities
For each in- concatenate vector embed- ding of the initial molecular graph with count-based embeddings of the prod- uct formula and its difference from the precursor formula (Figure 4. We again defer the particularities of the embedding functions to the Appendix (AA.)).
Figure The SCARF-Weavd network, which takes in the product formulae (e.g., predicted by and predicts their intensities. We use Set Transformer architecture (2), such that our model takes in the details the other product formulae present when predicting intensities.
3.3 Training and inference
Provided with a dataset molecules and formula-labeled mass spectra, we could train the two com- ponents of SCARF separately. However, in practice find it beneficial first train SCARF-Thread and then train SCARF- on its outputs so that the distribution the latter model sees is the same training and prediction time. SCARF-Weave Weave is trained using cosine loss (SA.5.5), as this most closely resembles the "retrieval" setting (&
ScreF-Thread is trained using the binary cross entropy losses associated with the multi-label classification tasks at each non-leaf node in the prefix tree. We use teacher forcing, i.c., we train on each level of the tree in parallel by conditioning our experiments, when generating the set of product formulae from this model we always pick the top 300. Empirically, based on a likelihood threshold.
the ground-truth set of prefixes at each stage. In
find that this provides better performance than picking variable number
Experiments
We evaluate SCARF spectra prediction ($ 222 and molecule identification in retrieval task ($ 43
4.1 Dataset
We train and validate SCARF on two libraries: gold standard commercial tandem mass spectrometry dataset, NIST20 [35], well as more heterogeneous public dataset, NPLIB1, extracted from the

---

Table I: Model coverage (higher better) of true peak formulae as determined by MAGMa at various max formula cutoffs for the NIST20 and NPLIB1 datasets. Best result for each column is in bold. Results are computed for single test set; all re-trained models (i.e., Autoregressive and SCARE variants) are averaged across three random seeds.
Dataset
NIST20
NPLIB1
Coverage
30
300
1000
10
30
300
1000
Random
0.009
0.026
0.232
0.532
0.004
0.014
0.126
0,336
Frequency
0.173
0.275
0.659
0.830
0.090
0.151
0.466
0.688
CFM-ID
0.197
0.282
0.170
0.267
Autoregressive
0.204
0.262
0.309
0.317
0.072
0.082
0.095
0.099
SCARF-Thread-D
0.248
0.425
0.839
0.941
0.158
0.284
0.681
0.856
0.249
0.476
0.855
0.943
0.155
0.306
0.708
0.859
SCARF-Thread 0.308
0.552
0.907
0.968
0.164
0.309
0.724
0.879
GNPS database 48 by Dührkop et al. [14] and subsequently processed by Goldman et al. 19. We prepare both datasets by extracting and preprocessing spectra, as well as filtering to compounds that (a) are under 1,500 Da (i.e., typically under 100 heavy atoms), (b) only contain predefined elements, and (c) are only charged with common positive-mode adduct types
Overall, NIST20 contains 35,129 total spectra with 24,403 unique structures, and 12,975 unique molecular formulae; NPLIB1 contains 10,709 spectra, 8,553 unique structures and 5,433 unique molecular formulae. Both datasets are evaluated using structure-de truuctureddisoinnt 90%/10% train/test split with 10% of training data held out for validation, such that all compounds in the test set are not seen in the train and validation sets.
Annotating spectra. We emphasize that SCARF can be trained with any product formula annotations, which can be labeled K55
or inferred with varied computational strategies 13) Herein, we utilize 40 In brief, for given molecule-spectrum pair the training dataset, the molecule is combinatorially fragmented at each atom up to depth of 3 breakages to create sub- fragments. This creates a bank of possible molecular formulae, and each peak in the spectrum is assigned its nearest possible formula within mass difference of 20 purts-per-million
the MAGMa algorithm
4.2 Spectra prediction
Predicting product formulae (SCARF-Thread) SCARF-Thread is trained and used to reconstruct prefix trees and evaluated by its ability to recover the ground truth product formula set. The set of generated product formulae is rank-ordered by the probability of each product formula and filtered to the top & predicted product formulae. The fraction of ground truth formulac (22.29 peaks average in NIST20) contained in the top k set is computed as coverage.
We compare coverage achieved by
to several baselines: (i) CFM-ID 22. fragmenta- tion based approach ($ A.3.T): (ii) a random baseline that samples product formulae from uniform distribution; (iii) frequency baseline, which ranks product formulae by the frequency the product formula candidate (or product formula difference) appears in the training set; (iv) an LSTM autore- gressive neural network baseline ($ A.3.2 that is trained to predict molecular formula vectors in sequence from highest to lowest intensity; and two model ablations, (v)
and (vi) SCARF-Thread-F: which only make uni-directional elements difference or forward predictions of element counts respectively (i.c., a in Equation B is fixed to for (v) and for (vi)).
In general, peaks, SCARF-Threal able to cover on average 91% and 72% of the true formulae in the ground truth test set for NIST20 and NPLIB1 respectively. Our difference- and forward-only directional prediction ablations demonstrate the benefits of modeling both the atom counts for each element and the differences in counts from the original molecule.
starkly outperforms all baselines tested (Table [T] By generating 300
Predicting mass spectra. We next evaluate the strength of SCARF -eave for intensity prediction on the same test dataset. We compare against five basclines: a fragmentation- based approach, CFM-ID
7

---

Table 2: Spectra prediction in terms of cosine similarity, coverage (proportion of ground-truth peaks that are covered by the top 100 non-zero predictions), validity (the fraction of predicted peaks for which chemically plausible explanation is possible), and time. Best value in each column is typeset in bold (higher is better for all metrics but time). Results are averaged across random seeds on single data split for all retrainable models (i.e.. not CFM-ID).
Dataset
NIST20
NPLIB1
Metric
Cosine sim.
Coverage
Valid
Cosine sim.
Coverage
Valid
Time (s)
CFM-ID
0.412
0.278
1.000
0.377
0.235
1.000
11 1114.7
3DMoIMS
0.510
0.734
0.945
0.394
0.507
0.919
3.5
Fixad
0.704
0.788
0.997
0.568
0.563
0,998
5.5
NEIMS (FFN)
0.617
0.746
0,948
0.491
0.524
0,949
3,9
NEIMS (GNN)
0.694
0.780
0.947
0.521
0.547
0.943
4.9
SCARF
0.726
0.807
1.000
0.536
0.552
1.000
21.1
using either feed forward network modules (FFNs), as in the original work, graph neural network modules (GNNs) as in SCARF-Weave and
2 two NEIMS binned prediction models [50]
A.3.3
described by Zhu et al. [58]: retrained variant of 3DMoIMS ((23); $ AA..4).. binned spectrum predictor that utilizes a point cloud neural network over a single molecular conformer input generated by RDKit 39 : and Fixed Vocab, formula prediction model that predicts intensities at fixed library
of formulae and formulae differences inspired by GRAFF-MS
A.3.5
To enable fair comparison across models, we predict test spectra at 15k bins (0.1 bin resolution between and 1500) with a maximum of 100 peaks for each predicted molecule. With the exception of CFM-ID, all models are hyperparameter optimized ($A.5.6). retrained completely, and conditioned on the same covariate inputs as SCARF: such steps lead large performance boosts the prior NEIMS method in particular. We evaluate the quality of our predictions based upon four core criteria reflecting our original desiderata of accuracy, physical-sensiblenees
and speed:
1. Cosine sim.: Cosine similarity between the ground truth and predicted spectra, indicating spectrum prediction accuracy.
2. Coverage: The fraction of ground truth spectrum peaks covered by the predicted spectrum.
3. Valid: The fraction of predicted peaks that can be explained by subformula (that obeys
basic ring-double bond equivalent heuristics (37]) of the predicted molecule.
4. Time (x): The wall time takes (using single CPU and no batched calculations) to load the model and predict spectra for 100 randomly selected molecules.
SCARF is more accurate than all other approaches on NIST20, improving cosine similarity over a GNN binned prediction approach by over 0.02 points in NIST20 and 0.01 in NPLIB1 (Table 2 Further, our method is more physically grounded insofar as all predicted peaks are guaranteed to be valid subformulae, unlike the unconstrained binned approaches, where nearly 5% of peak predictions cannot be explained by a valid molecular formula. Importantly, SCARF still operates orders of magnitude faster than CFM-ID (Table 2
The heterogeneity, reduced dataset size, and increased average molecular weight (Figure A3 of NPLIB1 leads to substantially worse absolute performance across all models. Interestingly. in this setting, the Fixed approach [K performs better, perhaps because the strict priors of formula constraints more helpful with fewer and more challenging training examples. We further stratify results by molecule size in Figure showing that all models are generally more accurate on smaller compounds. We additionally validate that cosine similarity is not merely measuring model's 's ability to predict the parent mass peak by computing modified cosine similarity with the original molecule's mass masked (Table A7
4.3 Retrieval
key application for forward spectrum prediction to use predicted spectra to determine the most plausible molecular structure assignment. We posit forward spectrum prediction models should be

---

A
B
10
Rasdon
2.5
CNOHA
1.5
CH.
44 CN
COO
Facklich
MERS (FTR)
Accuracy
0.0
NERSS (GNM)
SCARF
Intensity
0.5
Intensity
0.
0.
0.
0.5
0.5
SCARF
1.5 Trus
00
10
50
100
150
200
100
150
200
250
300
300
Top k
M/Z
M/Z
Figure 5: SCARF enables more accurate retrieval of ground truth molecules within the NIST20 dataset. A. Average retrieval accuracy of SCARF various top k thresholds. Retrieval is conducted on the same test split, and retrieval accuracy is averaged across models trained for three separate random seeds. B-C. Example spectrum predictions made by SCARF (top) compared to the ground truth spectrum (bottom). Up to 5 predicted peaks with the highest intensity are annotated with their molecular formula explanation predicted by SCARF. The full molecule is shown inset. Further examples are in the Appendix (Figure AT
particularly helpful in differentiating structurally similar molecules and design a retrieval task to showcase such potential. For each test set molecule, extract 49 potential "decoy" options based upon the most structurally similar isomers (i.c., compounds with the same precursor formula) within PubChem [27] as judged by Tanimoto similarity using Morgan fingerprints. While retrieval could be conducted on the entirety of PubChem or other similarly large molecular databases, we believe this subset retrieval setting is more practical and better mirrors a real-world setting (see $A.4 for justification). We predict the spectra for all molecules and rank them according to their similarity to the ground truth spectrum, computing the accuracy for retrieval. Herein, we specifically emphasize models and retrieval on the NIST20 dataset, as it is much larger and higher quality dataset.
SCARF reaches top 1 and top retrieval accuracy in this task of 18.7 1877% and 54.1 % respectively, representing an improvement over the methods with the second best top accuracy of 17.5% (NEIMS (GNN)) and top 5 accuracy of 52.2% (Fixed Vocab) (Figure 5AA.. We highlight two example predictions from SCARF (FigureBB-CC) with additional randomly sampled test set predictions shown in Figure A.I We repeat similar experiments within NPLIB1, but find that cosine similarity performance is uncorrelated with relative ranking performance; feed forward fingerprint based approaches are better retrieval, despite relatively weak cosine similarity (SA.I). Foced [34] performs especially well on NPLIB1, again likely due to the helpful biases imparted by constraining the formula vocabulary.
This result underscores previous observations regarding how database and model biases can skew retrieval results under certain settings classes of molecules, so the composition of these classes in the retrieval library may affect the retrieval accuracy accordingly. The observed discrepancy between cosine similarity and retrieval performance can further be explained by the dataset shift required for computing retrieval accuracy: cosine similarity is evaluated on "in-domain" data, whereas retrieval relies also accuracy on unlabeled data that may be "out-of-domain."
(22). That is, models may be more or less robust for certain
Related work
Forward vs backward models. Computational tools identify mass spectra are often divided into two categories: forward and backward models. Forward models, i.e., spectrum predictors, such as SCARF or the methods discussed in Section [ operate in the causal direction and try to predict the spectrum given the molecule. Backward models start from the spectrum and predict features or even full molecule structures. Early backward models used heuristics, expert rules, and even neural networks B I0 42 Such approaches have more recently been augmented with kernel methods and more modern, deep representation learning techniques complementary to spectrum predictors.
47
These models

---

Mass spectra for proteomics. Although this paper has focused on small molecules, similar trends of deep representation learning for mass spectra are also emerging in the adjacent field of proteomies 54
with Shouman et al. 41 recently proposing a benchmark challenge in this domain. While small molecule and protein spectra are similar, proteomics spectra tend to more easily predicted as fragments are often formed at peptide bonds. We believe adapting SCARF to this task would be an interesting direction for future work.
Neural set generation. Our work is also related to methods for modeling sets and multisets. SCARF- Thread generates a set as output, which has been studied elsewhere in the context of n n-gram generation 45 object detection +
57 and point cloud generation 28 The product formulac sets we generate, however, are different to those considered these other task; in our setting, each member of the set (i.e., individual product formula) represents multiset of atom types (i.e., multiple carbons, multiple hydrogens, etc.) and is constrained physically by the precursor formula.
6 Conclusion
In this paper we introduced SCARF, an approach utilizing prefix tree data structures to efficiently decode mass spectra from molecules. By first predicting product formulae and then assigning these formulae intensities, we are able to combine the advantages of previous neural and fragment based approaches, providing fast and physically grounded predictions. We show how these resulting predic- spectra and yield improvements
tions are both more accurate in predicting ssperimentally-obseree in identifying molecule's 's structure from its respective mass spectrum, tested on widely used dataset.
In term of limitations, our model is data dependent, as indicated by the relative performance across the NIST20 and NPLIB1 datasets. SCARF is also highly reliant upon the quality of product formula label assignment. The current commercial status of mass spectrometry training data poses barrier to entry, and identifying high quality public domain data is critical for future studies. A key contribution of this work is to retrain and optimize the hyperparameters of competing methods a publicly available dataset under equivalent conditions to allow for future extensions. Directly training on molking-bascdd loss learning model specific spectra distance function may be way to improve upon our model's performance in the retrieval setting, and we outline additional potential ideas more explicitly in A.6
Future directions will involve further developing SCARF for real world use cases such as unknown metabolite elucidation in clinical samples. Specific directions will include more carefully modeling covariates (e.g., collision energies and MS/MS instrument types), grounding product formulae in molecular graph substructures, and utilizing such models to augment inverse spectrum-to-moleeule annotation tools.
Acknowledgements
We thank members of the Coley Research Group, as well as Michael Murphy, for helpful discussions and comments. S.G. thanks the MaT-Takeda Program for financial support. J.B., J.X., and C.W.C. thank the Machine Learning for Pharmaceutical Discovery and Synthesis consortium for additional support.
References
[1] Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. Optuna: next-gemeration hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 2623 2631, 2019.
[2] Felicity Allen, Russ Greiner, and David Wishart. Competitive fragmentation modeling of ESI-MS/MS spectra for putative metabolite identification. Metabolomics, ((1):98-110, 2015.
[3] Peter Battaglia, Jessica B Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalez. Vinicius Zam- baldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner,
10

---

Caglar Gulcehre. Francis Song, Andrew Ballard, Justin Gilmer, George Dahl, Ashish Vaswani, Kelsey Allen, Charles Nash, Victoria Langston, Chris Dyer, Nicolas Heess, Daan Wierstra, Pushmeet Kohli, Matt Botvinick, Oriol Vinyals, Yujia Li, and Razvan Pascanu. Relational inductive biases, deep learning, and graph networks. arXiv preprint 1806v 2018
[4] Anelize Bauermeister, Helena Mannochio-Rusoo Letícia V Costa-Lotufis Alan K Jarmusch, and Pieter C Dorrestein. Mass spectrometry-based metabolomics microbiome investigations. Nature Reviews Microbiology, 20(3): 160-160, 2022.
[5] Wout Bittremieux, Robin Schmid, Florian Huber, Justin J J van der Hooft, Mingxun Wang. and Pieter C Dorrestein. Comparison of cosine, modified cosine, and neutral loss based spectrum alignment for discovery of structurally related molecules. Journal of the American Society for Mass Spectrometry, 33(9):1733-174) 2022.
[6] Wout Bittremieux, Mingxun Wang. and Pieter C Dorrestein. The critical role that spectral libraries play in capturing the metabolomics community knowledge. Metabolomics, 18(12):94, 2022.
[7] Michael M Bronstein, Joan Bruna, Taco Cohen, and Petar Veličković. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. arXiv preprint arXiv:2104. 13478, 2021.
[8] Bruce Buchanan, Georgia Sutherland, and Edward A. Feigenbaum. Heuristic Dendral: A program for generating explanatory hypotheses. Machine Intelligence, 4:209-254 254, 1969.
[9] Jing Chen, Yi Chen, Yang Jiang, Hua Fu, Bin Xin, and Yu-Fen Zhao. Rearrangement of P-N P-O bonds in mass spectra of Nadissoropyooxxyhooppooyy amino acids/alcohols. Rapid Communications in Mass Spectrometry, 15(20);193-199) 2001.
[10] Bo Curry and David Rumelhart. MSnet: A neural network which classifies mass spectra. Tetrahedron Computer Methodology,3CC ((3-4):213-231) 1990.
[11] Daniel P Demarque, Antonio EM Crotti, Ricardo Vessecchi, João LC Lopes, and Norberto Lopes. Fragmentation reactions using electrospray ionization mass spectrometry: an important tool for the structural elucidation and characterization of synthetic and natural products. Natural Product Reports, 33(3):432-445. 2016.
[12] Kai Dührkop, Huibin Shen, Marvin Meusel, Juho Rousu, and Sebastian Böcker. Searching molecular structure databases with tandem mass spectra using CSI:FingerID. Proceedings of the National Academy of Sciences, 12244): 2015.
[13] Kai Dührkop, Markus Fleischauer, Marcus Ludwig, Alexander A. Aksenov, Alexey V. Melnik, Marvin Meusel, Pieter C. Dorrestein, Juho Rousu, and Sebastian Böcker. SIRIUS 4: a rapid tool for turning tandem mass spectra into metabolite structure information. Nature Methods, 16
(4), 2019.
[14] Kai Dührkop, Louis-Félix Nothias, Markus Fleischauer, Raphael Reher, Marcus Ludwig, Martin A. Hoffmann, Daniel Petras, William H. Gerwick, Juho Rousu, and Pieter C. Dorrestein. Systematic classification of unknown metabolites using high-resolution fragmentation mass spectra. Nature Biotechnology, 3924):462-471, 2021.
[15] William Falcon and The PyTorch Lightning team. PyTorch Lightning, 3 2019. URL https //ithub.
[16) Ziling Fan, Amber Alley, Kian Ghaffari, and Habtom W. Ressom. MetFID: artificial neural network-based compound fingerprint prediction for metabolite annotation. Metabolomics, 16 (10):104, 2020. doi: 100/0/// 130-0000-1176---17
[17] J Gasteiger, W Hanebeck, and K P Schulz. Prediction of mass spectra from structural informa- tion. Journal of chemical information and computer sciences, 32(4):264-271, 1992.
[18] Samuel Goldman, Janet Li, and Connor W Coley. Generating molecular fragmentation graphs with autoregressive neural networks. arXiv preprint arXiv:2304.133. 2023.
11

---

[19] Samuel Goldman, Jeremy Wohlwend, Martin Stražar, Guy Haroush, Ramnik J Xavier, and Connor W Coley. Annotating metabolite mass spectra with fomain-inspired chemical formula transformers. Nature Machine Intelligence, pages 1-15, 2023.
[20] William L Hamilton. Graph representation learning. Synthesis Lectures Artificial Intelligence and Machine Leurning, 14()):1-19, 2020.
[21] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):
7735-1780, 1997.
[22] Martin A Hoffmann, Fleming Kretschmer, Marcus Ludwig, and Sebastian Böcker. Mad Hatter correctly annotates 98% of small molecule tandem mass spectra searching in PubChem. Metabolites, 13(3):114, 2023.
[23] Yuhui Hong, Sujun Li, Christopher Welch, Shane Tichy, Yuzhen Ye, and Haixu Tang. 3dmolms: Prediction of tandem mass spectra from three dimensional molecular conformations. bioRxiv, pages 2023-03, 2023.
[24] Florian Huber, Lars Ridder, Stefan Verhoeven, Jurriaan H Spaaks, Faruk Diblen, Simon Rogers, and Justin J van der Hooft. Spec2 Vec: Improved mass spectral similarity scoring through learning of structural relationships. PLoS computational biology, 17(2)xc1008724, 2021.
[25] Franziska Hufsky, Kerstin Scheubert, and Sebastian Böcker. Computational mass spectrometry for small-molecule fragmentation. TrAC Trends in Analytical Chemistry, 53:41-48, 2014.
[26] Hyun Woo Kim, Mingxun Wang, Christopher A Leber, Louis-Félix Nothias, Raphael Reher, Kyo Bin Kang, Justin JJ Van Der Hooft, Pieter Dorrestein, William Gerwick, and Garri- W Cottrell. Npclassifier: A deep neural Hewwrrk-basedd structural classification tool for natural products. Journal of natural products, 84(11):2795-298) 2807, 2021.
[27] Sunghwan Kim, Paul A. Thiessen, Evan E. Bolton, Jie Chen, Gang Fu, Asta Gindulyte, Lianyi Han, Jane He, Siqian He, and Benjamin A. Shoemaker. PubChem substance and compound databases. Nucleic Acids Research, 44(D1):D1022DD13
2016.
[28] Adam R Kosiorek, Hyunjik Kim, and Danilo J Rezende. Conditional set generation with transformers. In Workshop on Object-Oriented Learning ICML 2020, 2020.
[29] Juho Lee, Yoonho Lee, Jungtack Kim, Adam Kosiorek, Seungjin Choi, and Yee Whye Teh. Set transformer: A framework for attention- based permutation-invaaiie neural networks. In Proceedings of the 36th International Conference on Machine Learning, pages 3744-3753, 2019.
[30] Yujia Li, Daniel Tarlow, Mare Brockschmidt, and Richard Zemel. Gated graph sequence neural networks. International Conference on Learning Representations, 2016.
[31] Richard Liaw, Eric Liang, Robert Nishihara, Philipp Moritz, Joseph E Gonzalez, and Ion Stoica. Tune: A research platform for distributed model selection and training. ICML 2018 AutoML Workshop, 2018
[32] Francesco Locatello, Dirk Weissenborn, Thomas Unterthiner, Aravindh Mahendran, Georg Heigold, Jakob Uszkoreit, Alexey Dosovitskiy, and Thomas Kipf. Object-centric learning with slot attention. In Advances in Neural Information Processing Systems 33, 2020.
[33] Marcus Ludwig, Louis-Félix Nothias, Kai Dührkop, Irina Koester, Markus Fleischauer, Mar- tin A. Hoffmann, Daniel Petras, Fernando Vargas, Mustafa Morsy, and Lihini Aluwihare. Database-indeepedeen molecular formula annotation using Gibbs sampling through ZODIAC. Nature Machine Intelligence, 2(10):229641, 2020.
[34] Michael Murphy, Stefanie Jegelka, Ernest Fraenkel, Tobias Kind, David Healey, and Thomas Butler. Efficiently predicting high resolution mass spectra with graph neural networks. arXiv preprint arXiv:2301. 1419, 2023.
[35] NIST. Tandem Mass Spectral Library. NIST, 2020. URL httpss//www... /ov/ programs wwwpeeesstttanneeeeeeaaaae wwwpsasssssetaaa-l-bbree
12

---

[36] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang. 7ach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An imperative style, high-performance deep learning library. arXiv preprint arXiv: 191.0/703,, 2019.
[37] Ernö Pretsch, Philippe Bühlmann, Christian Affolter, Ernho Pretsch, Bhuhlmann, and Af- folter. Structure determination of organic compounds. Springer, 2000.
[38] Robert Quinn, Alexey V Melnik, Alison Vrbanac, Ting Fu, Kathryn A Patras, Mitchell Christy, Zsolt Bodai, Pedro Belda-Ferre, Anupriya Tripathi, Lawton K Chung, et al. Global chemical effects of the microbiome include new bile-acid conjugations. Nature, 579(7797): 123-129, 2020.
[39] RDKit Team. RDKit: Open-source cheminformatics, 2019. URL https::/wwwrkktttttot
[40] Lars Ridder, Justin JJ van der Hooft, and Stefan Verhoeven. Automatic compound annotation from mass spectrometry data using MAGMa. Mass Spectrometry, 3(Spec Iss 2):50033, 2014.
[41] Omar Shouman, Wassim Gabriel, Victor-Geeore Giurcoiu, Vitor Sternlicht, and Mathias Wil- helm. PROSPECT: Labeled tandem mass spectrometry dataset for machine learning in pro- teomics. In shitty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2022.
[42] Georgia Sutherland. Dendral-a computer program for generating and filtering chemical struc- tures. Technical report, Stanford University, Department of Computer Science, 1967.
[43] Matthew Tancik, Pratul P Srinivasan, Ben Mildenhall, Sara Fridovich- Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan T Barron, and Ren Ng. Fourier features let networks learn high frequency functions in low dimensional domains. In Advances in Neural Information Processing Systems 33, pages 7537-7547, 2020.
[44] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez. Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Informa- tion Processing Systems 30, pages 5998-6008, 2017.
[45] Oriol Vinyals, Samy Bengio, and Manjunath Kudlur. Order matters: Sequence to sequence for sets. In International Conference on Learning Representations, 2016.
[46] Gennady Voronov, Abe Frandsen, Brian Bargh, David Healey, Rose Lightheart, Tobias Kind, Picter Dorrestein, Viswa Colluru, and Thomas Butler. MS2Prop: A machine learn- ing model that directly predicts chemical properties from mass spectrometry data for novel compounds. bioRxiv, 2022. doi: 10.1 10009.511..9 org/ www.teet/eaally/02////////////// /ttp:///0000022220000991..11111.
[47] Gennady Voronov, Rose Lightheart, Joe Davison, Christoph A. Krettler, David Healey, and Thomas Butler. Multi-scale Sinusoidal Embeddings Enable Learning on High Resolution Mass Spectrometry Data. arXiv preprint arXiv:2207.02900 2022.
[48] Mingxun Wang, Jeremy J. Carver, Vanessa V. Phelan, Laura M. Sanchez, Neha Garg, Yao Peng, Don Duy Nguyen, Jeramie Watrous, Clifford A. Kapono, and Tal Krrzaatto-Knaan Sharing and community curation of mass spectrometry data with Global Natural Products Social Molecular Networking. Nature biotechnology, 34(8):828-88) 837, 2016.
[49] Minjie Wang, Da Zheng, Zihao Ye, Quan Gan, Mufei Li, Xiang Song, Jinjing Zhou, Chao Ma, Lingfan Yu, Yu Gai, Tianjun Xiao, Tong He, George Karypis, Jinyang Li, and Zheng Zhang. Deep graph library: A graph-centric. highly-performant package for graph neural networks. arXiv preprint 190.011009.01155, 2019.
[50] Jennifer N. Wei, David Belanger, Ryan P. Adams, and D. Sculley. Rapid prediction of elec- tron-ionization mass spectrometry using neural networks. ACS Central Science, 5(4):700-778 2019.
13

---

[51] David S. Wishart. Metabolomics for investigating physiological and pathophysiological pro- cesses. Physiological Reviews, 99(4):1819-1875 2019. doi: 10.1152/hhyxee.0000552220.
[52] Sebastian Wolf, Stephan Schmidt, Matthias Mülleer-Hanemann, and Steffen Neumann. In silico fragmentation for computer assisted identification of metabolite mass spectra. BMC Bioinformatics, (1(1):148; 2010. doi: 10.
[53] Shipei Xing, Sam Shen, Banghua Xu, and Tao Huan. Molecular formula discovery via bottom- up MS/MS interrogation. bioRxiv, 2022. doi: 00110012222.08.0355022222 URL https
www. biorxiv. www.bbioowv.ooooooooooo
502704
[54] Melih Yilmaz, William Fondrie, Wout Bittremieux, Sewoong Oh, and William S. Noble. De novo mass spectrometry peptide sequencing with a transformer model. In Proceedings of the 39th International Conference on Machine Learning, pages 25514-25522, 2022.
[55] Adamo Young, Bo Wang, and Hannes Röst. MassFormer: Tandem Mass Spectrum Prediction with Graph Transformers. arXiv preprint arXiv:2111.04822 2021.
[56] Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Ruslan Salakhutdinov, and Alexander Smola. Deep sets. In Advances in Neural Information Processing Systems 30, pages 3391-3401, 2017.
[57] Yan Zhang, Jonathon Hare, and Adam Prugel-Bennett. Deep set prediction networks. In Advances Neural Information Processing Systems 32, 2019.
[58] Hao Zhu, Liping Liu, and Soha Hassoun. Using Graph Neural Networks for Mass Spectrometry Prediction. In Machine Learning for Molecules Workshop at NeurlPS 2020, 2020.
[59] Richard Licheng Zhu and Eric Jonas. Rapid approximate subset-based spectra prediction for electron conization-mas spectrometry. Analytical Chemistry, 2023.
14

---

A Appendix
A.1 Extended results
We henchmark models in terms of retrieval accuracy as described ($ 4.3 for both the NIST20 and NPLIB1 datasets (Table AT A2 We recreate retrieval experiments using the full PubChem retrieval
library in Table A3
We reproduce main text results with standard error values included in Tables and A6 We showcase additional spectra predictions from our model trained NIST20 in
A4
A5
giure
Table A1: NIST20 spectra prediction retrieval top accuracy for different values of k. All values represent the mean across three separate random seeds the standard error of the mean for single test set.
10
Random
0.000
3DMMiMM
0.055
00.000
0.0022
0.185 0.004
0.225 0.004
0.332 00003
0.394
00.04
FisedNocab
0.172 1: 0.002
(13001 =: 0.002
0.339 :: 0.0001
0.466 1: 0.001
0.522 1: 0.006
0.638 :: 000005
0.688
0.003
NHIMS (FFN)
0.105 + 0.0022
0.243 = 0.006
0.324 :: 0.006
0.357 +: 0.005
0.640 +: D.00T
0.549 0.005
0.607
0.002
NHIMS (GNN)
0.175 0.003
0.001
0.632
SCARP
0.187 + 0.004
0.321 + 0.006
0.417 = 0.004
0,486 + 0.004
0.541 + 0.005
0.652 = 0.004 0.708 + 0.005
Table A2: NPLIB1 spectra prediction retrieval top accuracy for different values of k. All values represent the mean across three separate random seeds the standard error of the mean for a single test set.
top
Random
0.033 0.001
0.005
0.006
31MMiMMS
0.001
0.150 0..00
0:218
0.004
0:269 0.006
0.317 D.06
0.427 0.008
0.488 00.00
PixedVocah
0.193 0.000
0.314 +: 0.004
0,390 +: 0.003
0.448 + 0.005
0.492 +: 0.001
0.587 +: 0.005
0.635 0.006
NHIMS (FEN)
0.195 + 0.005
0.313 = 0.002
0.388 :: 0.003
0.477 +: 0.005
0.488 +: 0.002
0.586 0.007
0.624 0.010
NHIMS (GNN)
0.007
0.004
0.640
SCARP
0.135 2: 0.007
0,212 :2 0.001
0.520 1: 0.001
0.389 1: 0.001
0,411 0.002
0.569 1: 0.001
0.630 0.008
Table A3: Retrieval accuracy on NIST20 for single 500 molecule subset of the test set using library of 49 decoys or all decoys contained in PubChem ("None""). Results were repeated for random training seeds of the model and are shown + the standard error of the mean. The top value is shown in bold.
PubChem limit
50
None
Top k
2
3
2
3
Fixed Vocab
0.168 0.003
0.308 + 0.00 003
0.410 = 0.005
0.145 0.004
0.258 + 0.004
0.323 - 0.001
NEIMS (FFN)
0,102
0.002
0.237
: 0.003
0.315 1:200 0.004
0,087 0.003
0.183 1 0.008
0.236 = 0.007
NEIMS (GNN)
0.169 0.003
0.300 1 0.004
0.402
0.005
0.138 0.004
0.239 1 0.005
0.312 0.008
SCARF
0.204 0.009
0.326 + 0.005
0.432 + 0.009
0.167 0.003
0.258 + 0.001
0.336 + 0.003
A.2 Dataset preparation
is prepared by extracting all positive-mode experimental spectra collected in higher- energy collision-induced dissociation (HCD) mode (i.e., collected on Orbitrap mass spectrometers). Spectra are filtered, so that we keep only those for which the associated molecule (M) has (i) mass under 1,500 Da (ii) contains only elements from predefined set (i.c.. "C", "N", "P". "O", "S". "Si", "T", "H", "CT", "F", "Br", "B", "Se", "Fe", "Co", "As", "Na", "K"), and (iii) is charged with common adduct types (i.c., "[M+H]+", "IM+Na]+", "[M+K]+" "[M-H2++]+ "[M+NH3+H]+*" and "[M-2H20+H]+*). Because non-standard empirical spectra databases 48 often do not include the measured collision energies, pool all collision energies for each immpound-adduc pairing to create a single spectrum. We refer the reader to Young et al. 55 for detailed instructions for purchasing and extracting the NIST20 dataset.
NIST20 35
All spectrum intensities are square-root transformed to provide higher weighting to lower intensity peaks, normalized to maximum intensity of (i.c., through dividing by the maximum intensity).
15

---

1.0
Trac
10
Tus
Intendity
1.5
1.5
30
100
150
200
250
300
100
200
300
420
M/Z
M/Z
Intencity
0.0
0.0
M/Z /Z
/Z
3.5
1.5
SLAIB
SCARB
1.0
True
Intersity
3.0
1.0
1.5
320
M/Z
M /Z
1.0
0.5
0.5
Intencity
0.0
100
700
175
150
705
750
M/Z
M/Z
SCARF
SCARF
Intensity
Intensity
6.0
6.0
0:5
M/Z
M/Z
Figure A1: Example spectra predictions from the NIST20 dataset for 10 randomly selected test molecules. The ground truth spectra are shown underneath in black, with predictions above in teal. Molecules shown inset.
16

---

Table A4: Model coverage of true peak formulac as determined by MAGMa at various max formula cutoffs for the NPLIB1 dataset. Results are calculated for a single held out test split, shown + the standard error of the mean across three random seeds for all models that were retrained. The best value (i.e., highest) is typeset in bold for each column.
Coverage
10
30
300
1000
Random
0.004
0.014
0.126
0.336
Frequency
0.090
0.151
0.466
0.688
CFM-ID
0.170
0.267
Autoregressive
0.072 0.001
0.082 + 0.002
0.095 + 0.001
0.099 + 0.000
SCARF-D
0.158 + 0.001
0.284 = 0.003
0.681 + 0.002
0.856 0.002
SCARF-F
0.155 - 0.002
0.306 + 0.003
0.708 + 0.003
0.859 - 0.001
SCARF
0.164 + 0.009
0.309 + 0.014
0.724 + 0.013
0.879 + 0.004
Table A5: Model coverage of true peak formulae determined by MAGMa at various formula cutoffs for the NIST20 dataset. Results are calculated for a single held out test split, shown + the standard error of the mean across three random seeds for all models that were retrained. The best value (i.e., highest) is typeset in bold for each column.
Coverage @
10
30
300
1000
Random
0.009
0.026
0.232 0.659
0.532
Frequency
0.173
0.275
0.830
CFM-ID
0.197
0.282
Autoregressive
0.204 0.001
0.262 + 0.002
0.309 + 0.005
0.317 + 0.006
SCARF-D
0.248 + 0.001
0.425 + 0.002
0.839 + 0.002
0.941 + 0.001
SCARF-F
0.249 0.001
0.002 0.552 + 0.001
0.476
0.855 1 0.000
0.943 1 0.001
SCARF
0.308 + 0.002
0.907 + 0.002
0.968 + 0.001
filtered to exclude any noise peaks with normalized intensity under 0.003, and subsetted to only the top 50 highest intensity peaks. All peaks are mass-shifted by the weight of the parent adduct (i.c,. the spectrum is "[M+H]+", the weight of proton is subtracted from each child peak).
A.2.1 Product formulae assignments
Because the precursor ion and adduct species are known for the training dataset, we subtract the precursor adduct mass from every peak in the training set, and attempt to annotate each peak with plausible product formula (i.e., a subset of the true precursor formula).
We opt to constrain the training product formulae to be subsets of contiguous heavy atoms of the parent molecule as derived with the MAGMa algorithm 40)
We note two important limitations of these heuristics. First, by using molecular substructures to annotate product formulae, our model is less prone to correctly identifying complex rearrangements. Second, is also possible for adduct switching occur. Namely, if the precursor ion has sodium adduct ("[M+Na]+"), some of the product formulae may actually switch and acquire a hydrogen adduct instead. We assume adduct switching in our formulation, instead focusing the novelty of the prefix tree decoding approach, as these represent data labeling challenges, rather than modeling challenges.
In addition, any predictive models of product formulae distributions will more closely predict spectra that would be produced on instrumentation similar to the training sets utilized [9 [ Given this, we encourage users of such models to treat these predictions as putative, rather than experimentally valid.
17

---

Table A6: Spectra prediction in terms of cosine similarity, coverage (proportion of ground-truth peaks that are covered by the top 100 non-zero predictions), validity (the fraction of predicted peaks for which chemically plausible explanation is possible), and time. Best value in each column is typeset bold (higher is better for all metrics but time). Values are shown + the standard error of the mean computed across three random seeds on a single test set for all models that could be retrained (i.e., not CFM-ID).
XIST20
Cosine sim.
Coverage
Valid
Cosine sim.
Coverage
Valid
Time (6)
CFM-ID
0.412 + 0.000
0.278 + 0.000
1.00 + 0.000
0.377
0.000
0.233
0.000
1.00 0,000
1114.7
3DMolMS
0.001
0.394 0.002
FixodVocab
0.704 + 0.000
0.788 + 0.000
1.00
0.000
0.568
0.002
0.563 0.001
1.00 0.000
5.5
NETMS (FFN)
0.617 +00.000
0.746
0.001
0.95
0.001
0.491
+00.02
0.524 0.001
0.95 + +00000
3.9
NEIMS (GNN)
0.694 1: 0.000
0.780
0.000
0.95
0.001
0.002 0.521
0.547 0.003
0.94 0.001
4.9
SCARF
0.726 +: 0.001
0.807 + 0.000
1.00 + 0.000
0.536 0.007
0.562 0.008
1.00 + 0.000
21.1
Table A7: Spectra prediction accuracy comparing inclusion (Cosine sim.) and exclusion (Cosine sim. (no MS1)) of the precursor mass. For all compounds, the peak the mass of the input compound masked in the prediction and ground truth compute Cosine sim. (no MSI). All results represent an average on a single test set across three random seeds.
Dataset
NIST20
MPLIB1 Cosine sim.
Cosine sim.
Cosine sim. (no MS1)
Cosine sim. (no MS1)
CFM-ID
0.412
0.289
0.377
0.326
3DMoIMS
0.510
0,517
0.394
0.390
Fixed
0.704
0.637
0,568
0.505
NEIMS (FFN)
0.617
0,557
0.491
0,454
NEIMS (GNN)
0.694
0.620
0.521
0.477
SCARF
0.726
0.663
0.536
0.466
A.2.2
Dataset statisties
To probe the composition of two primary datasets, investigate both the molecular weight and chemical classes contained in the NPLIB1 and NIST20 datasets. We find that the average molecular weight higher for NPLIB1 (Figure A), consistent with the increased complexity of natural product molecules. We additionally compute chemical classes of the compounds using NPClassifier (26] to identify the types of compounds present in both datasets (Figure A33-CC. NPLIB1 enriched for steroids, coumarins, and various complex alkaloid natural products. On the other hand, NIST20 is enriched for small peptides, nicotinic acid alkaloids, and pseudoalkaloids, among others. While these descriptions are helpful to identify the dataset composition, chemical compound classification is itself learned classification and should be interpreted cautiously.
Table A8: NIST20 retrieval accuracy averaged across three random seeds of model training stratified by the weight of the target molecule.
Dataset
NIST20
Molecular weight
0 200
200
300
300
400
400
500
500
600
600
700
700
2000
Num. compounds
624
1358
900
387
129
56
89
Random
0.024
0.021
0.027
0.025
0,031
0.048
0.101
3DMoIMS
0,039
0,041
0.057
0.077
0.070
0.125
0.199
Fixed
0.143
0.184
0.168
0.172
0.196
0.220
0.161
NEIMS (FFN)
0,110
0.122
0.092
0.078
0.111
0.083
0.064
NEIMS (GNN)
0.155
0.192
0.164
0.182
0.147
0.214
0.161
SCARF
0.191
0.211
0.165
0.163
0.168
0.161
0.169
18

---

A
B
1.03
CRMDD
FreeWnnch NEMSS (PFN)
KFISS (6kM)
NPL/B1
3.00
Fanflichh NBHS (FRN)
NITMS (GVA)
MIST20
0.75
SDMMSS
SCARF
Cosine sim.
0.75
SCARF
Cosine
0.25
0.25
0.53
0:00
250
300
600
500
600
100
2000
1200
230
300
500
600
100
702
X00
Compound mass (Da)
Compound mass (Da)
D
1.00
CF1-1D
FoodVocb NBMS (FFN)
NEIMS (GAA)) NPLIBT
1.
CRIDD SDMMISS
Fouf/oca NEME (FFN)
NERS (CNN) SCARF
NIS T20
30McIMS
SCARF
Cosine sin.
0.75
Casine sim.
0.75
0.25
0.25
3.00
0.00
Storads
Counaees
Small enptil
Typroyiue akalnos
Frheppanil
Dospanaily
Binabiic aldiisa
Aushraniiic and dlatadd
Small pyyiiaa
Stroid
Flacessip
Typropii allanod
Bitatiii and ataliada
Sigultapenat
Counartr
panee alalni
Suppl
Compound superclass
Compound superclass
Figure A2: Cosine similarity of predicted spectra is stratified across molecular weight for both NPLIB1 (A) and NIST20 (B). We further stratify results across putative chemical classes of input molecules using NPClassifier (26] for both NPLIB1 (C) and NIST20 (D). The dotted line indicates the average predictive cosine similarity of SCARF across all examples and averaged over three random splits.
A
B NPLIB1
C NIST20
Tyrokie shakia
8:033
NISTPO
NILIB:
wile Ci)
add skaleds
0.022
Nostlicc dillits
D:022
Sterice
Small cepices
9.5%
Dorpaniis
Artioal add abalc
Pandul Jeal
Mal. weight
Crumait
Topcaha alaa del
Juiich
Figure A3: Spectra dataset molecule characterizations. A. Distribution of the molecular weight of compounds across NPLIB1 and NIST20. B-C. Chemical classes contained in NPLIB1 (B) and NIST20 (C) with the top 15 classes shown and all others grouped in 'Other'. Chemical classes are computed using NPClassifier [26].
A.3 Baselines
We further describe select baseline models.
A.3.1 CFM-ID baseline
CFM-ID ID 2 is long standing and important approach to fragmentation prediction. Because CFM-ID is fit using a time intensive EM training approach on an analogous dataset, we utilize the pretrained Docker implementation provided by the authors in line with [34] CFM-ID has two options for predicting molecules in either positive or negative adduct mode with "H" adducts (i.e., "[M+H]+" or "["- To directly compare to our method, we predict spectra in positive mode and remove hydrogens from all predicted peaks, as all training peaks are shifted by removing their adducts.
CFM-ID also produces predictions at three collision energies (i.e., low medium, or high fragmenta- tion). Because we opt not to include these, we merge these predictions and re-normalize the result to maximum of 1.
19

---

A.3.2 Autoregressive baseline
When considering the task of generating spectrum formulae candidates, we compare SCARF-Thread to an autoregressive recurrent neural network baseline, which based around a long short term memory (LSTM) module [21].
The LSTM generates formulae consecutively from single concatenated encoding of the input molecule and input full formula. At each step in the recurrent process, a one-hot encoding of the previous predicted element count is concatenated to a one-hot encoding of the element type being predicted in the current step. By embedding this information into the network, we can avoid predicting element types that do not appear in the parent molecule's 's molecular formula. the parent molecular formula has 5 element types, each autoregressively predicted formula requires generating only 5 element type counts; this eliminates the need for a stop condition between each formula. Formulae are generated autoregressively, from highest to lowest intensity. When predicting the counts of the next element type, we employ the same difference and forward count prediction strategy as used in SCARF for fair comparison. The model is trained with a cross entropy loss and full hyperparameters are listed in Table A10
A.3.3 NEIMS baseline
NEIMS 50 is a highly efficient binned spectrum prediction approach, originally developed for spectra (GC-MS). To enable fair comparison, we optimize its hyperpa- rameters on our dataset and add higher resolution bins. Furthermore, we also train a graph neural network-based version "NEIMS (GNN)" (in addition to the network that more closely matches Wei et al. [50) s original model and operates on the molecular fingerprint, "NEIMS (FFN)"). The adduct type is either concatenated to all atom features (for NEIMS (GNN)) or the fingerprint vector (for NEIMS (FFN)).
gas chromatography-mott
A.3.4 3DMoIMS baseline
is binned spectrum prediction approach developed simultaneously this work. Unlike the other binned NEIMS approach, 3DMoIMS utilizes a point-based deep neural network model operating on the point cloud of input molecule. To project 2D molecule SMILES string into 3D space, a single 3D conformer is first generated using RDKit 39 convolutions, the atom-wise representations are pooled, covariates corresponding to the settings of the machine and experiment are concatenated, and the result is projected into a fixed length binned spectrum.
3DMoIMS A
After several 3D
To enable a fair comparison, we copy the 3DMoIMS architecture into our modeling framework with minor tweaks to the network. Rather than use variable sized hidden layers, we fix hidden layer sizes to a single value across convolutions. In addition, we only use the covariate of the adduct type for consistency with model, excluding collision energy and instrument type. We hyperparameter optimize the model independently. We find that the performance of this model is substantially lower than the NEIMS baseline, likely due to the additional use of the "difference" prediction module in the NEIMS approach that allows the network to predict intensities at both fragments and neutral losses.
A.3.5 Vocab baseline
Concurrently to our work, Murphy et al.) introduue an alternative formula prediction strategy for mass spectrum prediction, a model they term GRAFF-MS. Unlike SCARF, GRAFF-MS utilizes a fixed vocabulary of molecular formulae and molecular formula differences, predicting intensities at each such value without learning to encode the formulac. These formulae and formula differences are selected in greedy fashion based upon their frequency in the training set.
Because no code was released for this approach the time this work was conducted, we reimplement a variant of their approach that emphasizes the use of fixed vocabulary of formulae and differences. For methodological consistency, we utilize equivalent formula annotations as used by SCARF (i.e.. one annotation per peak), do not model collision energies or instrument types, and utilize the same graph encoder as SCARF for encoding each molecule. We treat the number of fragment and difference formulae as tunable hyperparameter (which is optimized along with the rest of the hyperparameters - see (A.3.6p. We mask all invalid formulae and differences and utilize a cosine similarity loss with the original spectrum to train the model. To convert predicted formula and difference intensities into
20

---

binned spectrum, each formula-intes intensity pairing is projected into its respective binned position using a scatter max calculation. We note that because alternative adducts and isotopes are labeled in our preprocessing step, we do not predict isotopic or adduct variants for each fragment.
Given the differences between codebases, it is possible that the performance of reimplementation does not exactly match the original implementation, and we instead refer to it as "Fixed Vocab" rather than GRAFF-MS in table presentations. An earlier version of work understated the Foxed model's performance due to an implementation decision along these lines (specifically, not including the "0" neutral loss as a predicted vocabulary entry). This has since been rectified, increasing the accuracy of the FixedVocab model.
A.4 Retrieval subsets vs. PubChem
We restrict our retrieval experiments 4.3 to only the top 49 decoys per test case for two reasons. First, from a practical perspective, running the forward model every isomer match in PubChem (approx. thousands each, >1,000,000 for only 500 test cases) makes benchmarking across all considered models substantially more challenging both for this work and also future work. Second, we also believe that this top 50 challenge represents a more realistic setting. In practice, retrieval compound libraries will often be carefully crafted and designed to contain molecules similar the unknown molecule rather than all possible isomers (using either prior knowledge or "backward" models such as CSI:FingerID [I2] and MIST [9].. We conduct side-by-side analysis on small 500 molecule subset of the test set comparing the setting described above (with 49 decoys) to setting with limit on the number of decoys. The results shown Table A3 showing that SCARF still performs well in this setting.
A.5 Model details
Here, we describe details of our model's training setup, architecture, and hyperparameters that were omitted from the main text. Definitive details also be found in the code at https://gith:: com/ con/samgolan www.:omdmmn171/mmmmmmmm 97/ns///mmsss/ ht/s/mspas pred
A.5.1 Training
We train each of our models on single RTX A5000 NVIDIA GPU (CUDA Version 11.6), making of the Torch Lightning [[5] library to manage the training. SEARF-Thread and SCARF-Weatt take on the order of 1.5 and 2.5 hours of wall time to train respectively.
A.5.2 Molecule encoding
Within both SCARF-Threal and SCARF-Weave key component is an encoding of the molecular graph using message passing graph neural network, gnn(M). Such graph neural network models are now well described experiments, we use gated graph sequence neural networks 30 We made use of the implementation of this network the DGL. library 59] and use as atom features those shown Table A9 (which are computed using RDKit
20], so we will skip a detailed explanation of them here. our
39
DGL 49).
Table A9: Graph neural network (GNN) atom features.
Name
Description
Element type
one-hot encoding of the element type
Degree
one-hot encoding of number of bonds atom associated with one-hot encoding of the hybridization (SP, SP2, SP3, SP3D, SP3D2)
Hybridization type
Charge
one-hot encoding of atom's formal charge (from -2 to 3) binary flag indicating whether atom part of ring atom's 's mass as float
Ring-system
Atom mass
Chiral tag
atom's 'ss chiral tag as one-hot encoding one-hot encoding of the adduct ion
Adduct type
Random walk embed steps
positional encodings of the nodes computed using DGL
21

---

A.5.3 Molecular formulae representations
When forming representations of formulae (including formulae prefixes) we use count-based encoder, co) This encoder takes in as input the counts of all individual elements in the formula (which also can be "undefined" for counts of elements not yet specified indicated as 's' in Figure 3)) and returns a vector representation IRd, The encoder is based upon the Fourier feature mapping proposed by Tancik et al. [3]. but using only sin basis functions (to reduce the number of parameters required by our networks). Tancik et al. 43 has shown that such features perform better than encoding integers directly; furthermore, compared to learned representations, using Fourier features enables us (at least in principle) to deal with counts at test time that have not been seen during training.
To be precise, each possible count, v No. is encoded by our counts based encoder into the vector:
2TT Tj
2TU T2
2 T3
1)
abs
sin
.iin
sin
where the periods (Т1, T2, etc.) are set increasing powers of two that enable us to discriminate all possible element counts given in the input, and abs(.) is the absolute value function such that we get positive embeddings. For the "undefined" count we learn a separate encoding of the same dimensionality.
A.5.4 Further details of SCARF-Three
Pseudo-code for the SCARF-Thread model shown in Algorithm A. Note that the second for loop (on the line marked i) does not depend on previous iterations of the loop, so that in practice we perform this computation parallel. training time we use teacher forcing (33.). meaning the first for loop (marked f) is only run sequentially inference time.
The function scarf- ceaffthrredd-net() represents the network shown in Figure 33 and generates the set of subsequent valid element counts given a prefix (i.e., the child nodes of given prefix node). As discussed in the main text, we treat this as a multi-label binary classification task and predict the binary label for each possible count using forward and difference MLPs (Eq, B}. We fix a maximum possible element count (i.e., the number of possible classes in this classification problem), N 160. We do not allow product formulae to have more of given element than is present in the precursor formula, F. and we achieve this by setting the probability of these classes to zero.
Algorithm A.I: Pseudo-code for SCARF-Threat which generates prefix trees from a root node autoregressively, one level at a time.
Data: Input molecule, M, with corresponding input formula, F. Result: Set of product formulae, Pe = )
hm < gnn(MM)
Form embedding of precursor molecule.
Po < [()
Store the set of initial prefixes which just the undefined formuln,
for j E [1. [l. do
Loop over possible elements.
Pj - (
P Create the set of prefixes the next time around.
hj one-hot(j);
Encoding of which element predicting count
for J2z E Pj-1 do (in parallel)
61
b Loop over all current prefixes.
c' [hu counts( counts F J22g) hyl;
Create context vector, Eq
(f) 1 + scarf seaff-hheead-ne( F):
Predict the set of valid next element counts under this prefix.
Pj + Pj U create-new-prefiees
f 18/ Ir'i);;
Create new prefixes for element.
return
A.5.5 Further details of SCARF-Weave
As discussed in the main text, SCARF- -Wave is based off Lee et al. 229 's Set Transformer. After forming the input encoding using the molecule and count-based encoder ($A.5.2 $A.5.3), we further refine this embedding using an MLP (multi-layer perceptron) network. The output of this passed into a series of 13 Transformer 4 layers A.5.6 defines the exact number used in the experiments) with 8 attention heads cach.
22

---

We use a cosine distance loss to train the parameters of SCARF-Weards This loss is also used for the FFN and GNN baselines (Table 2). To ensure consistency with the baselines, we first project the output of
into a binned histogram representation ($A.5.6 defines the number of bins used); for each bin we take the max intensity across all applicable formulae. Given a predicted binned spectra, 8, and the ground-truth binned spectra, 8, the cosine distance is defined as the negative of the cosine similarity (computed using PyTorch's 's torch. wwwsinessmmiiiiieeeooeeeeeeiesee simieemimmiiiii function [3]):
8
cos-sim ($, 8) max (||||||||| c)
(A.1)
where x 10-8 8 is used to ensure numerical stability.
A.5.6 Hyperparameters
To enable fair comparison across models, hyperparameters were tuned for SCARF, the FFN binned prediction baseline, and the GNN binned prediction baseline. Parameters were tuned using RayTune 31 with Optuna (1] and an ASHAScheduler, Each model was allotted 50 different hyperoptimization trials for fitting. Models were hyperparameter optimized on a smaller 10, 000 spectra subset of NIST20. Parameters are detailed in Table AT0
A.6 Limitations and future work
We outline several potential directions for future work to address limitations of this work.
1. Improving the gold standard training annotation pre-processing. Because SCARF is flexible in that it can match distributions of formula assignments, a key step to improving and building upon this approach spectra. This includes adding complexity and removing potential assumptions, such as allowing annotations to account for rearrangement, elimination, or charge transfer. second goal is to identify potentially low quality training spectra, such as ones that emerge from mixtures, and remove these from the inputs. Another potential way to handle such cases would be to model each spectrum peak an ensemble of potential equivalent- which would be particularly helpful relating SCARF to inverse models such as MIST [9 in which the structure of the molecule and formula identity of each peak cannot be known priori.
develop more robust assignments of formula to training
formulae,
2. Incorporating other model covariates. Incorporating collision energy features explicitly into the model, as well as negative-ion mode inputs, will increase its usability. This could be enabled by aggregating public data containing these annotations.
3. Featurizing molecule imputs using different or more powerful molecular encoders. Recent and simultancous work to this used pretrained graph encoder as part of a binned spectrum prediction approach, MassFormer (55). It is possible include more powerful molecule or formula encoders into SCARF.
4. Consideration of interpretability by subgraph attribution and combination with ICEBERG. Following this initial work, developed second model, ICEBERG [8], that uses similar two step modeling approach, but instead encodes fragments, not formula. This increases accuracy and robustness, especially for retrieval, but substantially slows the model. In comparison to ICEBERG, SCARF still has several benefits including speed, the lack of required substructure labeling, and ability capture potential skeletal rearrangements of molecules (i.e., discontinuities in structure that may not be possible to model by only breaking bonds). An open question and exciting opportunity the future combine these two levels of abstraction and make formula-level predictions with graph-level attribution or featurization.
5. Retrieval-spectie -pecifffc loss functions to enhance retrieval performance. A significant finding of this work was the noise associated with the retrieval task and lack of correlation with spectrum prediction performance as measured by cosine similarity. Future work may consider how to more directly define loss functions that reflect the task of retrieval.
23

---

Table A10: Model and baseline hyperparameters.
Model
Parameter
Grid
Value
Autoregressive
learning rate learning rate scheduler
[le 4, lc 3]
0.0009
StepDecay (5,000)
lcarning rate decay
(0.7,
0.85
dropout
00..0.1.0.2.0000 0.3}
0.2
hidden size, d gnn layers
{128, 256, 512) [1,6]
512
rnn layers
[1.3]
batch size
(8, 32, 64)
64
weight decay
(0, le 6, le 7}
le - 6
differences (Eq 3
(True. False]
True
conv type
GatedGraphConn
random walk embed steps (Table A9
[0,20]
20
graph pooling
(mean, attention)
mean
NEIMS (FFN)
learning rate
[le 4, le - 3]
0.00087
learning rate scheduler
StepDecay (5,000)
learning rate decay
[0.7, 1.0]
0.722
dropout
(0.0,0.1,0.2,0.3)
0.0
hidden size, layers, 1 batch size weight decay
(64, 128, 512)
512
[1,2,3] (16. 128)
128
{0, le 6, le 7}
use differences (Eq
(True. False)
True
num bins ($5.55
15, 000
NEIMS (GNN)
lcarning rate learning rate scheduler
[le - 4, le 33
0.00052
StepDecay (5,000)
learning rate decay dropout
[0.7, 1.0
0.767
(0.0,0.1.0.2,0 0.3)
0.0
hidden size, layers, I batch size weight decay
(64, 128, 256, 512] [1.6]
512
(16, 64)
64
[0 6, le 7}
le -
use differences (Eq
(True, False)
True
num bins AA.5.
15, 000
conv type
GatedGraphCCnv
random walk embed steps (Table A9
[0,20] (mean, attention)
19
graph pooling
3DMolMS
learning rate
[1e 4, 1e - 3)
0.00074
learning rate scheduler
StepDecay (5,000)
Icarning rate decay dropout
[0.7, 1.0]
0.86
(0.0, 0.3)
0.3
hidden size, d layers, I top layers neighbors, & batch size weight decay num bins A.5.5
(64, 128, 512)
256
[1,6]
[1,3]
[3,6]
[16, 32, 64)
16
(0 6, 1e-e-7 7}
1e 6
15, 000
Focad
learning rate learning rate scheduler learning rate decay dropout hidden size, d layers, I batch size weight decay num bins A$..55
[le 4, le 3]
0.00018
StepDecay (5,000)
[0.7, L.0]
0.92
(0.0, 0.2, 0.3} (64, 128, 256, [1,6]
0.3
512
(16, 32, 64]
1. 6, 1 73
1c - 6
15, 000
conv type
GatedGraphConv
random walk embed steps (Table A9
[0,20]
11
graph pooling
(mean, attention)
mean
24

---

Table A10 continued from previous page
Model
Parameter
Grid
Value
formula library size
(1000, 5000, 10000, 25000,, 50000)
5000
SCARF-Threas
learning rate learning rate scheduler
[le 4, le -3)
0.000577
StepDecay (5,000)
learning rate decay dropout
[0.7. 1.0]
0.894
(..0,0.1.0.,,033)
0.3
hidden size, mlp layers, I gnn layers, la ($A.522 batch size weight decay
2128,256,512 512) [1.3]
512
[1,
16, 32, 64}
16
1e 6, 7}
1e 6
use differences (Eq)
(True. False)
True
conv type
GatedGraphConv
random walk embed steps (Table A9
[0,20] (mean, attention)
20
graph pooling
mean
SCARF-Weave
learning rate
[lc 4, 1c - 3]
0.00031
learning rate scheduler
StepDecay (5,000)
learning rate decay dropout
[0.7, 1.0]
0.962
(0.0, 0,2, 0.3)
0,2
hidden size, d
{128, 256, 512) 1,3 1,6
512
mlp layers, 1
($ A.5.5
gnn layers, A.3.2
transformer layers, 13 (8 A.5.5
0,3
batch size weight decay
84 16, 32, 64)
32
00, le 6, le 7}
num bins (A..55 conv type
15, 000
GatedGraphConv
random walk embed steps (Table
A9
[0,20] (mean, attention)
graph pooling
attention
25


Reviews:
[{"review_id": "aV9uROaz0F", "reviewer": "Reviewer_ZFnW", "summary": "The authors propose a model, SCARF, that predicts Tandem-MS spectra in a two-step process: 1) predicting atomic subsets, 2) predicting peaks given the subsets. The model's performance is evaluated on two datasets (NIST20, Natural Product Library) using on spectral similarity, coverage, validity, and retrieval, and compared with relevant baseline models.\n", "questions": "- It's great that the SCARF algorithm, and subset prediction generation in general, allows for easy interpretation of peaks. How does the peak annotation produced by SCARF compare to the assignments given by NIST's MS Interpreter tool?\n\n- In comparing predicted spectra with ground truth spectra such as in figures 5B&C and Figure A.1: Do you notice any classes of molecules or other patterns where SCARF is particularly good at making predictions? Or particularly bad at making predictions? What is your guess for why the SCARF model generates some extra peaks for some of the molecules in Figure A.1?\n\n- How does the performance of SCARF on the retrieval task depend on the size of the molecule, and therefore the number of plausible isomers.\n\n- In the retrieval task section, it is mentioned that 'database and model biases can skew retrieval results'. Could elaborate more on the biases that are referred to here?\n\n- It could be helpful to provide what you perceive the use case of the SCARF model in real world applications. Would you expect experimental chemists to predict all candidate structures for their target molecule using SCARF, and then perform the retrieval task to identify the right molecule?", "limitations": "I do not identify any negative broader societal impacts associated with this work.\n\nI agree with the author's statement that the model's performance will be data dependent. If you plan to release your model (which I would highly encourage), it would be helpful to give an overview of the types of molecules that you expect to be in or out of distribution for the model.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "+ Clear figures and descriptions of the modeling approach. \n+ Clear explanation of background/previous work. The main difference between the SCARF model and other models which use subset prediction is that the SCARF model uses prefix trees to predict subgraphs, so this model considers all relevant subformulae and does not have a limit on the number of valid substructures considered.\n+ Compares against relevant modeling baselines. Evaluates models on coverage and validity to judge the interpretability of the spectra.", "weaknesses": "- It would be helpful to provide some more context for the contents of the two datasets, NIST20 and NPLIB. For example, what classes of molecules are present in each dataset (Classyfire might be a helpful tool for this purpose: http://classyfire.wishartlab.com/). What are the molecular weights of compounds of molecules in this dataset? \n\n- Retrieval task: If I understand correctly, the retrieval task here is set up such that the 'library' or search space is 50 candidates total, 1 target and 49 other decoys, which are selected from the set of chemical isomers with the highest Tanimoto similarity. Each model is used to predict the spectra for all 50 candidates. \n\nWhy not perform the retrieval task on the entire NIST 20 library? Or if NIST20 is too large, why not use the set of all structures with the same chemical formula or precursor formula, rather than limiting the task to 50 isomers? I believe that by limiting the search library to a  set of 50 molecules that are most chemically similar to the target, the retrieval task might become easier than the real world version. In particularly, typically when one wants to identify a compound, you have few assumptions about what your target molecule is, other than the chemical formula, so you would not be able to filter by chemical similarity to the target. Retrieval tasks in previous works such as Wei et al 2019 were performed on the entire NIST17 dataset.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments! Point-by-point response below.\n\n**W1**: It would be helpful to provide some more context for the contents of the two datasets, NIST20 and NPLIB…  \n**A**:  We agree and will add a section in the Appendix with training dataset details by molecular weight and also molecular class as estimated by NPClassifier (we have found the API for Classyfire to be more time-intensive and difficult to utilize in the past); we refer the reviewer to Author Response Figure 1. \n\n**W2**: Retrieval task: … Each model is used to predict the spectra for all 50 candidates. Why not perform the retrieval task on the entire NIST 20 library? Or if NIST20 is too large, why not use the set of all structures with the same chemical formula or precursor formula, rather than limiting the task to 50 isomers?   \n**A**: We will make the retrieval process clearer in the text. We believe our retrieval library is a more challenging setting than using the entire NIST20 library. NIST20 contains roughly 24,000 unique compounds (i.e,. approximately 2 compounds per chemical formula), whereas our isomer library derived from PubChem contains 50 isomers per test case. \n\nWe posit that subsetting to a top 50 “hard” isomer retrieval library for each spectrum makes sense for two key reasons. First, from a practical perspective, running the forward model on every isomer match in PubChem (approx. thousands each, >1,000,000 for only 500 test cases) could make benchmarking across methods prohibitively expensive considering the full set of compared methods. Second, we believe that this top 50 challenge also represents a realistic setting. In practice, retrieval compound libraries will often be carefully crafted and designed to contain molecules similar to the unknown molecule rather than all possible isomers  (using either prior knowledge or “backward” models such as CSI:FingerID / MIST– which have demonstrated top 50 accuracy over 90%). \n\nTo demonstrate that this does not influence the relative ranking of methods, we have conducted a side-by-side retrieval analysis on the PubChem 50 isomer subset vs. all of PubChem isomers for a 500-spectrum subset of the NIST20 test set (see Author Response Table 3).\n\n**Q1**: It's great that the SCARF algorithm, and subset prediction generation in general, allows for easy interpretation of peaks. How does the peak annotation produced by SCARF compare to the assignments given by NIST's MS Interpreter tool?  \n**A**: Thank you for your kind words. SCARF is able to learn to approximate chemical formula labels provided for each peak in the training set (Main Text Table 1). For the trained models reported in this paper, they would therefore most closely match assignments from MAGMa; the model could be biased toward MS Interpreter by using that preprocessing pipeline instead. We have not conducted a comparison across different formula/peak explanation labeling methods, but this might be recommended if one’s primary use case for SCARF is peak annotation rather than spectral prediction. \n \n**Q2**: …. Do you notice any classes of molecules or other patterns where SCARF is particularly good at making predictions? Or particularly bad at making predictions? What is your guess for why the SCARF model generates some extra peaks for some of the molecules in Figure A.1?  \n**A**: We have conducted a new analysis comparing cosine similarity performance across molecular mass bins and also chemical classes (NIST20 shown in Author Response Figure 2). Chemical class stratifications are challenging to interpret, but it is clear that SCARF more accurately predicts smaller (<500Da) compounds. SCARF is trained on cosine similarity loss, which does not heavily penalize very low intensity peaks, potentially explaining extra peaks.\n\n**Q3**: How does the performance of SCARF on the retrieval task depend on the size of the molecule, and therefore the number of plausible isomers.   \n**A**: We add a table (Author Response Table 1) showing retrieval accuracy as a function of molecule size on the NIST20 library. As illustrated by cosine similarity, retrieval performance is best at lower masses, where there are also the most training examples.\n\n**Q4**: In the retrieval task section, it is mentioned that 'database and model biases can skew retrieval results'. Could elaborate more on the biases that are referred to here?   \n**A**: We mean to point out that certain models may be more or less robust for certain classes of molecules, so the composition of these classes in the retrieval library may affect the accuracy accordingly.\n\n**Q5**: It could be helpful to provide what you perceive the use case of the SCARF model in real world applications. Would you expect experimental chemists to predict all candidate structures for their target molecule using SCARF, and then perform the retrieval task to identify the right molecule?  \n**A**: We will update this section. We anticipate core applications of retrieval tasks as you mention, alongside data augmentation for training inverse spectrum-to-molecule models.\n\n**L** _\"... If you plan to release your model (which I would highly encourage), it would be helpful to give an overview of the types of molecules that you expect to be in or out of distribution for the model.\"_  \n**A**: We have provided model and baseline code in the Supplementary Material and will also provide a GitHub link on de-anonymization. We hope the new analysis we have provided in response to **W1** provides some insight into model performance across molecule classes.\n"}, {"review_id": "XhnvAKCG67", "reviewer": "Reviewer_sZMH", "summary": "The paper presents a new method for predicting mass spectra from molecules called SCARF, which stands for Subformulae Classification for Autoregressively Reconstructing Fragmentations. Mass spectra are sets of peaks that represent the masses and intensities of fragments of molecules after they are ionized and broken down in a mass spectrometer. Predicting mass spectra from molecules is useful for identifying unknown molecules from experimental data, as well as for understanding the fragmentation process and generating virtual spectra libraries.\n\nSCARF predicts mass spectra in two steps: first, it generates the set of chemical formulae for the fragments, which define the locations of the peaks on the mass-to-charge axis; second, it assigns intensities to these formulae, which define the heights of the peaks. The key innovation of SCARF is that it uses prefix trees to efficiently generate the set of formulae, overcoming the combinatorial challenge of enumerating all possible subformulae of a given molecule. SCARF also ensures that all predicted peaks are physically plausible, meaning that they correspond to valid subformulae of the original molecule.\n\nThe paper evaluates SCARF on two datasets of molecules and their experimentally measured spectra, NIST20 and NPLIB1. The paper shows that SCARF outperforms existing methods based on fragmentation rules or binned prediction in terms of accuracy, physical sensibility, and speed. The paper also demonstrates that SCARF can improve the retrieval of unknown molecules from new spectra by comparing them to predicted spectra from a database of candidate molecules. The paper concludes by discussing the limitations and future directions of SCARF.", "questions": "- How does SCARF handle molecules that contain elements that are not in the predefined element set? How does it handle molecules that have unknown or ambiguous formulae?\n- Can SCARF handle spectra that have multiple precursor ions or multiple charge states? How does it handle spectra that have different adduct types or ionization modes?\n- Will SCARF handle molecules that have multiple conformers or stereoisomers? How does it handle molecules with different fragmentation patterns depending on their conformation or configuration?\n- How sensitive is SCARF to the choice of hyperparameters, such as the number of predicted peaks, the prefix tree depth, or the neural network architecture? How did the authors tune these hyperparameters, and what are the trade-offs involved?\n- How generalizable is SCARF to other datasets or domains, such as proteomics, metabolomics, or natural products? How would the authors adapt SCARF to these domains or datasets?\n- How interpretable is SCARF in terms of explaining its predictions or providing chemical insights? How would the authors improve the interpretability or visualization of SCARF?", "limitations": "The authors have discussed some of the limitations of their work in Section 5, such as the data dependency, the quality of product formula annotation, and the assumptions and simplifications made by their model. However, they could also mention some of the other limitations I pointed out in my previous comments, such as the interference from other molecules or ions, the continuous and structural features of molecules, the correlated fragmentation pathways, and the low-quality or noisy spectra. They could also provide some empirical evidence or analysis to support their claims about the limitations and future directions of their work.", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "The paper makes several contributions. \n\nFirst, the paper introduces a new method for predicting mass spectra from molecules, which is based on a novel combination of subformulae classification and autoregressive reconstruction. The method overcomes the limitations of existing methods based on fragmentation rules or binned prediction, which are either too restrictive or too coarse-grained. It achieves state-of-the-art performance in terms of accuracy, physical sensibility, and speed, as demonstrated on two datasets of molecules and their experimentally measured spectra. The method is based on a simple yet powerful idea of using prefix trees to generate the set of formulae for the fragments efficiently. It has the potential to revolutionize the field of mass spectrometry by enabling more accurate and efficient identification of unknown molecules from experimental data.\n\nSecond, the paper provides a thorough evaluation of the proposed method on two datasets of molecules and their experimentally measured spectra, NIST20 and NPLIB1. It uses a new metric called physical sensibility, which measures how well the predicted spectra match the physical constraints of mass spectrometry. SCARF outperforms existing methods based on fragmentation rules or binned prediction in terms of accuracy, physical sensibility, and speed. The paper provides detailed descriptions of the datasets, metrics, baselines, and results. The evaluation is significant because it demonstrates the effectiveness and robustness of SCARF across different datasets and scenarios.\n\nThird, the paper discusses the limitations and future directions of SCARF. It identifies several open problems and challenges in mass spectrometry that SCARF or its variants can address and provides insightful analyses and suggestions for future research. \n\nOverall, the paper represents a significant advance in mass spectrometry by introducing a new method for predicting mass spectra from molecules.", "weaknesses": "I list some weaknesses below:\n\n1. The paper assumes that the fragmentation process is purely additive, meaning that each fragment is formed by adding one or more atoms to the previous fragment. This assumption may not hold for some molecules that undergo complex fragmentation pathways, such as rearrangement, elimination, or charge transfer. To address this weakness, future work could explore more general models of fragmentation that can capture these pathways, such as machine learning models or expert systems.\n\n2. It assumes that the mass spectra are measured under ideal conditions, meaning there is no interference from other molecules or ions. This assumption may not hold for some real-world scenarios, such as complex mixtures or dirty samples. To address this weakness, future work could investigate how to incorporate prior knowledge or external data sources to improve the accuracy and robustness of mass spectra prediction.\n\n3. The assumption is that the molecules are represented by their molecular formulae, which are discrete and symbolic. This representation may not capture the continuous and structural features of molecules that affect their fragmentation patterns and mass spectra. To address this weakness, future work could explore more expressive and flexible representations of molecules that can capture these features, such as molecular graphs or descriptors.\n\n4. The fragmentation patterns are assumed independent of each other, meaning that each fragment is formed independently of the others. This assumption may not hold for some molecules that undergo correlated fragmentation pathways, such as cleavage of adjacent bonds or the formation of cyclic structures. To address this weakness, future work could investigate how to model these correlations explicitly or implicitly in the prediction process.\n\n5. The paper assumes that the mass spectra are measured with high resolution and accuracy, meaning each peak is resolved and assigned a precise mass-to-charge ratio. This assumption may not hold for some low-quality or noisy spectra that have overlapping or shifted peaks. To address this weakness, future work could develop methods for denoising or deconvolving mass spectra before or after prediction.\n\nOverall, these weaknesses suggest several directions for future research in mass spectrometry and related fields.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments and positive reception of our work. We believe you have identified several key discussion points and highlighted limitations of the work that would be intriguing starting points for future work. We now plan to incorporate a new section into the Appendix that explicitly details these points and opportunities for improvement including:  \n- Expanding the gold standard training annotation preprocessing to annotate MS/MS peaks with formulae that result from rearrangement, elimination, or charge transfer.  \n- Detection of possible spectrum mixtures in the training dataset or strategies to deconvolve these at test time.  \n- Featurizing molecule inputs using different or more powerful molecular encoders.  \n- Ensembles of MS2 subformula possibilities in the case these cannot be known.  \n- Consideration of interpretability by subgraph attribution.   \n- Retrieval-specific loss functions to enhance performance on this task.   \n\n**Q1**: How does SCARF handle molecules that contain elements that are not in the predefined element set? How does it handle molecules that have unknown or ambiguous formulae?   \n**A**:   Currently, SCARF cannot handle such elements, but we suspect even if it could, these would be too out-of-domain for useful predictions. We extract the predefined element set from the training set; although we could use all possible elements as our vocabulary, the trained model would almost certainly predict zero counts for these other elements.    \n\nAmbiguous/unknown formulae is an interesting challenge (we note on lines 327-329 the importance of data quality)  and we will add our thoughts on this to our expanded section on possible directions for future work; for a molecule corresponding to ambiguous formulae, one idea is to use a weighted ensemble on different inputs. \n\n**Q2**: Can SCARF handle spectra that have multiple precursor ions or multiple charge states? How does it handle spectra that have different adduct types or ionization modes?  \n**A**: Ionization modes are accepted as a model input, so SCARF is able to predict several common adduct/ion types in its current form. Multiple charge states are not considered in the current framework, as we had access to substantially fewer training examples of such cases..  \n\n**Q3**: Will SCARF handle molecules that have multiple conformers or stereoisomers? How does it handle molecules with different fragmentation patterns depending on their conformation or configuration?\n**A**: Given the nature of data acquisition, spectra almost always represent an ensemble of accessible conformations; SCARF’s featurization decision reflects this, as it encodes molecules using a 2D GNN. This encoder is not sensitive to changes in stereochemistry, but an improved encoder could be easily integrated as our “molecule featurizer” if desired.   \n\n**Q4**: How sensitive is SCARF to the choice of hyperparameters, such as the number of predicted peaks, the prefix tree depth, or the neural network architecture? How did the authors tune these hyperparameters, and what are the trade-offs involved?   \n**A**: Great question! We carefully tuned these for both our model and the baselines with RayTune. The search space and final hyperparameters chosen are given in the Supplementary Material. Of note is that the model was quite sensitive to the choice of formula representation (see A.4.3). We believe this is because of the need to differentiate very small formula differences.  \n\n**Q5**: How generalizable is SCARF to other datasets or domains, such as proteomics, metabolomics, or natural products? How would the authors adapt SCARF to these domains or datasets?  \n**A**: SCARF is already well suited to metabolomics experiments of natural products, as one of the training sets NPLIB1 is derived from such experiments. Lipidomics or glycomics are potential areas of expansion but would require more investigation.  \nFor proteomics because fragmentation usually occurs at amide bonds, it might make more sense to use a formalism and vocabulary based on amino acids (rather than chemical elements).    \n**Q6**: How interpretable is SCARF in terms of explaining its predictions or providing chemical insights? How would the authors improve the interpretability or visualization of SCARF?  \n**A**: SCARF is able to ascribe a formula to every predicted sub-peak. An area of future inquiry and high interest would be to enforce the model to directly tie these formulae to molecular-substructures. We have added this to our Appendix list of future work.  \n"}, {"review_id": "ojI3sPDld0", "reviewer": "Reviewer_188s", "summary": "The manuscript presents a method called SCARF for the prediction of tandem mass spectra from molecular structures. This allows for building up large in-silico libraries that, together with measured spectra, can be used in spectral library search workflows for metabolite identification. From a general machine learning perspective, the method is an interesting approach for the problem of predicting sets of multisets (in this case: sets of molecular formulas). The author note that autoregression is not well suited for this task, as it is not ordering invariant. Instead, they suggest an approach that is quite similar to the way molecular formulas are decomposed from masses: they step-by-step build a prefix-tree where each node represents a subset of the molecular formula space with certain elements are fixed to specific abundancies. The child nodes are then possible abundancies for the next chemical element, restricting the formula space further until the the leafs of the tree describe only a single molecule formula. The resulting set of molecular formulas is then transformed into a spectrum, by utilizing a second transformer based network for intensity prediction.", "questions": "- does the cosine similarity calculation takes the precursor ion into account? For many spectra, the precursor can have high intensity (and, thus, heavily effect cosine similarity) while it does not provide any new information. If so, I would suggest to recalculate cosine similarity and leave out the precursor\n- I do not understand why the spectrum retrieval task is performed on a subset of 49 isomers (although the most similar ones) instead of just all isomers", "limitations": "The main limitation of the method is in my opinion its limited performance on the metabolite identification task. This and other limitations are discussed properly in the manuscript.", "rating": 7, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- it is the first paper that predicts high resolution spectra without involving combinatorial fragmentation\n\n- use of prefix tree for the prediction of molecular formulas (or sets of multisets) is very innovative\n\n- use of random Fourier features for molecular formula embedding is innovative and seem to work much better than the common approaches of encoding formulas as frequency vectors or one-hot vectors/dictionaries\n\n- evaluation on the spectrum prediction part is done very well. In particular, certain methods are retrained (or even re-implemented) and parameter-optimized to ensure fair comparison\n\n- manuscript is clearly written, the method is described in detail with all hyper parameters given in the supplement\n\n- source code is available ", "weaknesses": "- there is already a successor method (ICEBERG) by the same authors for the same task. This is not a problem in general, as both methods have very different strategies to solve the problem. I still think it would have strengthen the manuscript if pros and cons of both methods and their synergistic potential is discussed \n\n- the manuscripts list three applications for the predicted spectra: molecule identification, learning about fragmentation, and training machine learning methods on spectra. However, in contrast to ICEBERG the method presented here cannot give any insights about the fragmentation process (as it is not a fragmenter). I also doubt that synthetic data (predicted machine learning) is well suited as training data for other machine learning methods. This is some kind of self-training or self-supervised learning which is a difficult and challenging problem on its own. Thus, the only convincing application for the spectrum prediction task is the molecule identification. Beside spectrum prediction, there are other methods that allow for molecule identification (including methods from the same authors such as MIST). Although the evaluation on metabolite identification contains many methods for spectrum prediction, it does not contain any of these alternative approaches.\n\n- the observation that identification rate is decoupled from database retrieval rate is concerning, as this is the main application for spectrum prediction\n\n- although all methods perform worse on the NPLIB1 data, it is strange that NEIMS(FFN) and SCARF perform so very different on the  retrieval task. It would be interesting to find out why the fingerprint based method performs and the dictionary based method both perform so well here but not in NIST.\n\n- besides its application in metabolomics,  the presented approach might be interesting for other areas with sets of multisets as target variable. Unfortunately, the baseline method (autoregression) is not evaluated against the more sophisticated prefix tree decoding method ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your very fair comments and feedback. \n\n**W1**: Putting SCARF in context with other recent methods   \n**A:** Thanks for your suggestion. We will add a section to the Appendix explaining how SCARF has more flexibility as it does not require structural fragment labeling, higher speed, cosine similarity, but struggles on retrieval. We agree that SCARF and other models (including even older fragmentation-based approaches) have synergies and think developing ways to combine them (e.g., mixtures of experts)  would be an interesting future direction.    \n\n**W2.1**: Of the three stated applications of such a model (data augmentation, interpretation, and molecule identification), molecule identification is the only one that is relevant.   \n**A:** This is an interesting point; thank you for raising it. While we agree one could argue that molecule identification is currently probably the most promising aspect of these models, we want to push back a little here about it being the \"only convincing application\".   \n\nFor example, the MIST manuscript [14; p.44]  reported that data augmentation (via synthetic data) is surprisingly very beneficial, which follows a theme similar to how AlphaFold2 (Jumper et al., p.587) was able to generate increased performance with knowledge distillation methods. We strongly suspect that better / synthetic augmentation (perhaps generated from SCARF) will be critical to improving backwards models, which we hope future studies will prove.  \n\n**W2.2**: Molecule identification is not compared against other methods for molecule identifications such as MIST  \n**A** The comparison between other molecular identification methods such as fingerprint prediction or contrastive learning is an interesting, open question. For the purposes of this work, which we intend to contribute a new forward predictive method, we see that as somewhat out of scope (also absent in all existing work on spectrum prediction [36,44,24]). One interesting facet of spectra predictors compared to backwards models is that they work in the causal direction and so may perform differently when tested on out of distribution data (Kilbertus et al., 2018); we hope a future benchmark of all molecular identification methods also includes such tasks.   \n\n> Kilbertus, N., Parascandolo, G. and Schölkopf, B. (2018) ‘Generalization in anti-causal learning’,: http://arxiv.org/abs/1812.00524.\n\n**W3-4**: The decoupling between retrieval and cosine accuracy is perplexing   \n**A**: We agree with this and had attempted to highlight this in our paper. We believe this discrepancy results from cosine similarity accuracy being computed on labeled, “in-domain” data, whereas retrieval relies also on accuracy on unlabeled data. We will revisit this in the paper accordingly to make it more clear.   \n\nPart of this somewhat ties into your previous question (about application). If ultimately, one decides one wants to use a spectra predictor purely for retrieval it may make sense to consider directly training on a ranking-based loss or learning a model specific spectra distance function, but we leave such follow ups to future work.   \n\n**W5**: Autoregressive model baseline for formula generation.  \n**A**: We agree this could help clarify our point. We have implemented such an approach where an autoregressive language model successively predicts each formula in order of highest to lowest intensities. We find that, in preliminary experiments on NIST20, this model has top 10, 30, and 300 coverage of 0.214, 0.276, and 0.324 vs. SCARF-Thread’s respective coverage of 0.316, 0.559,  and 0.911.  The limitation of this baseline is likely due to the long range dependencies involved and tendency to stop generating unique formulae after a certain position (i.e., for a molecule of CHNOP elements, generating 300 potential product formulae would require 1,500 tokens). This is aside from the other issues we mention (lines 147-151).    \n\n**Q1**: Cosine similarity should be recomputed without precursor ions   \n**A**: We have conducted this additional analysis (see Author Response Table 2), masking out the mass of the full parent chemical formula in both the predicted and ground truth spectrum. Under these new conditions, SCARF is still the most performant on NIST20, but fails to outperform FixedVocab on NPLIB1 (consistent with poor retrieval on this dataset).    \n\nThrough this analysis, we uncovered that our implementation of FixedVocab prevented it from predicting the parent chemical formula and have updated it (included in the table). We will update all results for FixedVocab in the full manuscript accordingly; thank you for helping us to notice this. We note that we have also recomputed top 1 retrieval accuracy on NIST20 with this new method and it is 0.177 (vs. 0.165 in Appendix Table A1); SCARF is still superior with a Top 1 accuracy of 0.184.   \n\n**Q2**: Spectrum retrieval should be performed against the entirety of PubChem   \n**A**: To show this would not change the relative ordering of methods, we conduct a new experiment in which we repeat the retrieval evaluation using  the entirety of PubChem for a 500-spectrum subset of NIST20 and include it in Author Response Table 3.   \n\nIn addition, we point to our explanation to Reviewer ZFnW along a similar line of questioning regarding retrieval accuracy, where we explain why we believe this decision is both practical for benchmarking purposes and similar to real-world settings.   \n"}, {"review_id": "ZPGtpT6p40", "reviewer": "Reviewer_8EJR", "summary": "This paper proposes a novel method for predicting a tandem mass spectrum from a given molecular graph.  The key idea is to subdivide the problem into predicting fragmentation of the graph and then separately predicting intensities associated with each fragment.  Relative to other recent methods that have adopted this approach, this method uses a prefix tree to make the identification of subgraphs more efficient.  Experimental results show that the proposed method performs better than state-of-the-art methods.", "questions": "Is there a concrete sense in which binning methods are less interpretable? \n\nHow large are the sets handled by SCARF-Weave?\n\nWhy are MS-GRAFF and FormulaNet/SubsetNet not included in the empirical comparison?\n", "limitations": "N/A", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "The paper addresses an important problem.\n\nThe paper is extremely well written, with a nice intro to tandem mass spectrometry targeted to a NeurIPS audience.\n\nThe treatment of related work is well organized and clear.\n\nThe main idea is well supported by empirical results.\n", "weaknesses": "On lines 37-42, I wasn't super convinced that \"physically-inspired\" (which, incidentally, doesn't need to be hyphenated) belongs in this list of desiderata.  I think the main thing is that the method is fast and accurate.  It seems more like a hypothesis to say that in order to achieve speed and accuracy, a method should be physically inspired.  Perhaps the notion of \"accurate\" should be segregated into various dimensions: e.g., accurate in the sense that it doesn't just predict binned m/z values, or accurate in the sense that it doesn't predict extra peaks that can't be explained by the formula.  I would argue that if someone came along with a method that was very accurate and fast, there is no scenario in which an end user would prefer a slightly slower or slightly less accurate method just because it comes with a story about how it is physically plausible.\n\nFigure 1A is the key idea, and I think it needs to be expanded.  As drawn, it's not clear how, e.g., you decide that the root should have three children (C7, C12 and C14).  Why not C1, C2, etc.?  The answer presumably has to do with the molecular graph, which is the primary input but is not shown here.  I would also like to hear a bit more about this prefix tree step in the text.  E.g., does it matter what order the atom types are introduced?   If so, how is the order chosen?\n\nRelated to the first point above, the claim on lines 100-102 that binning methods are less interpretable was hard for me to understand.  The reduced accuracy due to the lack of shared information across fragments is plausible, but that seems like an empirical claim that requires support.  Maybe just say that you hypothesize that this may lead to reduced accuracy. I also found it strange that you didn't explicitly call out the most obvious source of inaccuracy, which is that the binning means that the peak m/z locations are inherently inaccurate.\n\nFor the SCARF-Weave predictor, please indicate how large the sets can be come, both in principle and in practice.  I wonder whether this is a potential problem for the method.\n\nWhen you present the various competing methods on lines 248-255, I think it would be helpful if you could introduce them in terms of the three groups from Section 2.2.\n\nThe closest competing methods are references [24] and [45].  In your experimental comparisons, why did you use a method based on MS-GRAFF ([24]) rather than just using MS-GRAFF itself?  And why didn't you use method [45] at all?\n\nOn a related note, if FixedVocab falls into the third category of methods, then why aren't its predictions 100% valid in Table 2?\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your very constructive comments and suggestions to help improve the work! \n\n**W1:** “Physically inspired” should not necessarily be a desiderata   \n**A**: We agree that the desiderata are somewhat interrelated and your point is well taken (and thanks — we have removed the hyphen!). We hypothesize that to achieve accuracy — particularly in low data or out of distribution domains — a method should be physically inspired (or at least pre-trained on a body of similar and physically relevant data), which may help to \"parameterize out'' certain false positive errors. We had originally used \"interpretable\" as a term (removed to disambiguate from interpretability about how the networks are working) and are open to suggestions.\n\n**W2** Fig 1A can be clarified   \n**A**: Assuming that you are referring to Figure 3A, we will make this Figure clearer to indicate that each element count number is predicted as a function of the input molecule and also reference pseudocode in the Supplementary Material (A.4.4). In answer to your specific questions:    \n- This figure shows the full prefix tree (from the product formulae set). At training time only the correct nodes would be expanded (i.e., the three children you mention, because they appear in the ground truth spectrum). At inference time, the model would predict (as a multi-label binary classification problem) which nodes to expand (as such it will sometimes make mistakes).     \n- The order of the atom types is fixed based on the order in which we define the vocabulary of elements, but it would be interesting to explore whether different orders would lead to better or worse performance.    \n \n**W3**: The \"most obvious source of inaccuracy\" is missing for binned spectra.   \n**A**: We attempted to point this out (line 54), and we will also add it to this later section! We will also clarify interpretability (see answer to Q1 below).   \n\n**W4**: Is the SCARF-Weave predictor limited in how large sets can be?   \n**A**: We were able to train models with up to 500 predicted formulae in reasonable time on a single GPU (<5 hours) given the modest size of the dataset (approx. 35k spectra) We anticipate this number could be pushed higher but would likely require switching the pairwise inter-formula attention to a sparse attention framework. Of course, this does increase the time for inference.   \n\n**W5** Other formula-based methods are not included as baselines.   \n**A**: We address this in our response to Q3 below.  \n\n**W6**: FixedVocab does not have 100% validity.   \n**A**: Our  “validity” metric checks if all peaks can be explained by a formula that passes the “ring double bond equivalent” heuristic for valence rules. Because FixedVocab also allows for the prediction of “loss” formulae, there are edge cases on which the parent molecule formula minus a “valid” loss prediction leads to an “invalid” fragment prediction. Such cases are very rare; any method that uses formula-based or structure-based representations should have roughly 100% validity.  \n\n**W7**: The closest competing methods are references [24] and [45]   \n**A**: See Q3 below.  \n\n**Q1**: Is there a concrete sense in which binning methods are less interpretable?   \n**A**: Binning methods provide no provenance for the peaks, or even if an individual bin is actually made up of many smaller peaks. SCARF (providing formulae) or fragmentation methods (providing substructures) provide further information, giving practitioners clues to interpret why a predicted spectrum looks the way it does (linking back to the original molecule's properties). These additional outputs can also be used to assess the reliability of the prediction.  \n\n**Q2**: How large are the sets handled by SCARF-Weave?   \n**A**: As described above (W4), SCARF-Weave sets were fixed at 300 candidate formulae, but we pushed them up to 500 during internal experiments. This could likely be increased further at the expense of speed but may require switching to sparse attention.   \n\n**Q3**: Why are MS-GRAFF and FormulaNet/SubsetNet not included in the empirical comparison?   \n**A**: We became aware of these methods (both published earlier this year), while finalizing our submission. In turn:\n\n_GRAFF-MS._ At the time of our submission, we were unable to obtain code or data for GRAFF-MS; the code has since been released approximately two weeks ago, at its presentation at the ICML conference (July 25, 2023). FixedVocab is our attempt at a reimplementation (from the details that existed in the GRAFF-MS preprint). To match our other models, we reimplemented it using 1) our own formula labeling scheme (rather than NIST’s internal tool that has isotopic peak labels); 2) the same GNN architecture as SCARF for consistency; and 3) the same cosine loss that we utilized to train SCARF.  We renamed it as “FixedVocab” to avoid misrepresenting the original work, but the core idea of the two models should be essentially the same.   \n\n_FormulaNet/SubsetNet._ We did not compare to FormulaNet/SubsetNet as it is trained using a maximum of 48 atoms (because it requires complete enumeration of formula candidates), a key limitation preventing its use on our tested datasets (NIST20 has molecules with up to 102 heavy atoms). It is trained for EI spectra acquired in combination with gas phase chromatography and more useful for more “volatile” molecule classes where this size limitation is less problematic.   \n"}]

Author rebuttal:
We thank all the reviewers for their thoughtful feedback and comments. Where relevant, we have addressed stated weaknesses (**W**), questions (**Q**) and limitations (**L**) with our answers (**A**). We also include an additional single page PDF containing additional referenced figures and tables.

Meta-review: The paper proposes a novel, inovative way of predicting tandem mass spectra of molecules, an important task encountered many fields including metabolomics, natural producst research, environmental sciences and biomedicine. The method is based on building a prefix tree that allows eficient exploration of the chemical subformula space to produce subformula candidates for the fragment peaks. The intensities for the predicted peaks are then preicted through a transformer network. The method overcomes limitations of previous approaches, based on combinatorial fragmentation or direct prediction from molecules to vector representations of mass spectra. The paper provides thorough and convincing empirical evaluation demonstrating a marked improvement in accuracy compared to state-of-the art. The reviewers identified minor weaknesses in the reviews which have been adequately answered by the authors in their rebuttai.
Decision: Accept (spotlight)