Title: Prefix-Tree Decoding for Predicting Mass Spectra from Molecules
Venue: NeurIPS 2023
Accepted: yes
Keywords: molecules

prefix tree

mass spectra

mass spectrum prediction

metabolomics

GNNs

chemistry

biology
Abstract: Computational predictions of mass spectra from molecules have enabled the discovery of clinically relevant metabolites. However, such predictive tools are still limited as they occupy one of two extremes, either operating  (a) by fragmenting molecules combinatorially with overly rigid constraints on potential rearrangements and poor time complexity or (b) by decoding lossy and nonphysical discretized spectra vectors. In this work, we use a new intermediate strategy for predicting mass spectra from molecules by treating mass spectra as sets of molecular formulae, which are themselves multisets of atoms. After first encoding an input molecular graph, we decode a set of molecular subformulae, each of which specify a predicted peak in the mass spectrum, the intensities of which are predicted by a second model. Our key insight is to overcome the combinatorial possibilities for molecular subformulae by decoding the formula set using a prefix tree structure, atom-type by atom-type, representing a general method for ordered multiset decoding. We show promising empirical results on mass spectra prediction tasks.

Prefix-Tree Decoding for Predicting Mass Spectra from Molecules

Abstract

Computational predictions of mass spectra from molecules have enabled the discovery of clinically relevant metabolites. However, such predictive tools are still limited as they occupy one of two extremes, either operating (a) by fragmenting molecules combinatorially with overly rigid constraints on potential rearrangements and poor time complexity or (b) by decoding lossy and nonphysical discretized spectra vectors. In this work, we use a new intermediate strategy for predicting mass spectra from molecules by treating mass spectra as sets of molecular formulae, which are themselves multisets of atoms. After first encoding an input molecular graph, we decode a set of molecular subformulae, each of which specify a predicted peak in the mass spectrum, the intensities of which are predicted by a second model. Our key insight is to overcome the combinatorial possibilities for molecular subformulae by decoding the formula set using a prefix tree structure, atom-type by atom-type, representing a general method for ordered multiset decoding. We show promising empirical results on mass spectra prediction tasks.

1 Introduction

As the primary tool to discover unknown small molecule structures from biological samples, tandem mass spectrometry (MS/MS) experiments have enabled the identification of numerous important molecules implicated in health and disease [4, 38, 51]. Tandem mass spectrometers are capable of isolating, fragmenting, and measuring the resulting fragment masses of small molecules from a sample, producing a signature (a mass spectrum) for each detected molecule (Figure 1, top).

Collision cell

Mass spectrum

Intensity

Input

mixture

Precursor molecule

Learned simulator

SCARF

M/Z

C 7

O 1

H 6

C 8

O 2

H 6

C 9

O 3

H 4

C 7

O 1

H 4

C 14

O 1

H 10

O

O

O

HO

Computationally predicting mass spectra from molecules in silico (Figure 1, bottom) is thus a longstanding and important challenge. Not only does this assist practitioners in better understanding the fragmentation process, but it also enables the identification of molecules from newly observed spectra by comparing an observed spectrum to virtual spectra generated from a database of candidate molecules. While a large library of empirical mass spectra could theoretically serve the same purpose, the size of such libraries is limited by the slow and expensive process of acquiring pure chemical standards and measuring their spectra, motivating computational prediction.

We argue that there are three core, interrelated desiderata for a forward molecule-to-spectrum simulation model, or 'spectrum predictor'. An ideal spectrum predictor should be (i) accurate , being able to predict the exact set of fragment masses and intensities with a precision comparable to experimental measurements; (ii) physically inspired , to avoid making physically nonsensical ('invalid') suggestions and to provide interpretations of the chemical species responsible for each peak for the benefit of human expert chemists; and (iii) fast , such that it is computationally inexpensive to predict spectra for many (e.g., millions) hypothetical molecules.

Unfortunately, many existing spectrum predictors do not meet these criteria. Methods to date have tended to follow one of two approaches: (a) physically motivated fragmentation approaches or (b) molecule-to-vector (or 'binned') approaches (Figure 2A-B). Fragmentation approaches (e.g., [2, 17, 40, 52]; Figure 2A) take an input molecule and suggest bonds that may break, creating fragments that are scored by ML algorithms or curated rulesets. While interpretable, these methods are often slow and restrictive; certain mass spectrum peaks are generated by complex chemical rearrangements within the collision cell that cannot be approximated by bond breaking alone. That is, the bonds in observed fragments are not a subset of those in the original molecule [9, 11]. On the other hand, binned prediction approaches (e.g., [50, 55, 58]; Figure 2B) are less physically grounded, using neural networks to directly learn a mapping from molecules to vectors representing discretized versions of the spectra. These methods, while fast, lack interpretability and due to discretization have a mass precision lower than that of most modern spectrometers, limiting their accuracy.

We propose to address the shortcomings of previous work by predicting mass spectra from molecules at the level of molecular formulae (e.g., C x N y O z H w ...) and introduce a new method, Subformulae Classification for Autoregressively Reconstructing Fragmentations ( SCARF ) to do so. Because the molecular formula for each input molecule is known, each subformula in the predicted set of peaks is constrained to contain a subset of the atoms in the original formula. Our primary contributions are:

posing mass spectrum prediction as a two step process: first generating the set of molecular formulae for the fragments, then associating these formulae with intensities;

overcoming the combinatorial subformula option space by learning to generate formula prefix trees;

demonstrating the empirical benefit of SCARF in predicting experimental mass spectra quickly and accurately using two separate datasets, providing a benchmark for future work.

2 Background

We provide a short introduction of tandem mass spectrometry suitable for a general machine learning audience, detail previous approaches to modeling this process as they relate to our proposed approach SCARF , and explain how such tools can be utilized to discover molecules from new spectra. We refer interested readers to [25] for further details on the physical process of mass spectrometry.

2.1 Tandem mass spectrometry

Tandem mass spectrometers (MS/MS) measure fragmentation patterns of molecules in a multi-stage process. The input to the process is a solution containing a precursor molecule, M ∈ X , associated with a molecular formula, F , defining the counts of each element present; for instance F = C16 O 4H12 for the precursor molecule shown in Figure 1. The precursor molecule is first ionized (i.e., made charged), often by bonding or associating with an adduct (e.g., a proton, H + ) present in the solution. The charged product is then measured by a mass analyzer (MS1), where its mass-to-charge ratio ( m/z ) is measured.

A

B

C

C 7

O 1

H 4

C 7

O 1

H 6

,

,

,

,

,

C 8

O 2

H 6

C 9

O 3

H 4

C 7

O 1

H 6

C 7

O 1

H 6

C 8

O 2

H 6

C 8

O 2

H 6

C 9

O 3

H 4

C 9

O 3

H 4

C 14

O 1

H 10

C 7

O 1

H 4

C 7

O 1

H 4

C 14

O 1

H 10

C 14

O 1

H 10

O

O

O

HO

O

O

O

HO

O

O

O

HO

O

HO

O

O

O

HO

O

O

O

O

O

O

O

HO

O

O

O

HO

Intensity

Binned prediction

Fragmentation prediction

{

{

M/Z

Intensity

M/Z

Step 1: SCARF-Thread

intensity vector

output

Step 2: SCARF-Weave

Intensity

M/Z

(i)

(ii)

(iii)

(iv)

(vi)

(vii)

(vii)

(ii)

(i)

This precursor ion is then filtered into a collision cell . Here, through interactions with an inert gas, the precursor ion is broken down into a set of one or more product ions , each of which is associated with a new chemical formula; for example, one might be f 1 = C7OH4 for the process shown in Figure 1. Finally, this set of product ions is measured by a second mass analyzer (MS2), along with the set of their intensities, y i ∈ R + (i.e., their relative frequencies over several repetitions of this process), creating for each ion what is referred to as a peak . The collection of all peaks makes up a molecule's mass spectrum , and is commonly represented as a plot of intensities versus m/z (Figure 1, right).

2.2 Predicting mass spectra from molecules (spectrum predictors)

Fragmentation prediction. A complex but physically grounded strategy is to model the bond breakage processes occurring in the collision cell (Figure 2A). Examples include MetFrag [52], MAGMa[40], and CFM-ID [2], which recursively fragment molecules (either bond or atom removals) to generate fragment predictions. These methods combine expert rules and local scoring methods to enumerate molecular fragmentation trees to predict spectra. CFM-ID [2] learns subsequent fragmentation transition probabilities between fragments with an expectation maximization algorithm to determine intensities at each fragment. Rule-based methods and full tree enumeration reduce the flexibility of these approaches, and along with the inherent ambiguity in the fragmentation process, limit this strategy's overall accuracy and speed.

Binned prediction. An increasingly popular and straightforward approach to spectra prediction is to map molecules to discretized 1D mass spectra from either molecular fingerprint [50] or graph inputs [55, 58] (Figure 2B). Specifically, these methods divide the m/z axis into fixed-width 'bins' and predict an aggregate statistic of the peaks found in each bin (such as their maximum or summed intensity). While more flexible and end-to-end than fragmentation-based approaches, these methods do not impose the same physical constraints or shared information across fragments, making them less interpretable and susceptible to making invalid predictions. Further, discretizing the input spectrum inherently restricts the precision of such models compared to exact-mass predictions.

Formula prediction. We introduce the strategy of predicting spectra at the level of molecular formulae, an intermediate between binned and fragmentation prediction (Figure 2C). Simultaneous to our work, two groups have separately explored formula prediction strategies [34, 59]. However, to generate plausible subformulae candidates, they either generate a fixed vocabulary of formulae [34] or restrict their model to molecules under 48 atoms for exhaustive enumeration [59], which is smaller than many compounds of interest. We overcome the combinatorial problem of formula generation using prefix trees, allowing our method to scale and eliminating the need for large, fixed vocabularies.

2.3 Mass spectrum libraries

One important use of spectrum predictors is in building large in silico libraries of molecule spectra to augment the small size of existing, experimentally derived databases (on the order of 10 4 ) which are expensive to curate. These spectra libraries are then leveraged downstream in different ways, for example for training molecular property predictors directly from mass spectra [46]. Another common application of spectra libraries is to infer an unknown molecule's structure from a newly observed spectra - a particularly hard problem, with only 13% of spectra measured from clinical samples identifiable using current elucidation tools [6]. In this problem, spectra libraries are used as part of a process called retrieval : The newly observed spectra is compared with the existing spectra in the library using a fixed or learned spectral distance function, such as cosine distance [5, 24], and the molecules associated with the closest spectra are returned as possible matches. In practice, the retrieval process is constrained to choosing among isomers (i.e., molecules with the same molecular formula, and therefore molecular weight, but with different bond configurations) due to the high resolution of modern mass spectrometers (i.e., absolute errors on the order of 10 -4 to 10 -3 m/z for MS1 measurements) [13, 33, 53].

Given the varied use cases of spectra libraries, we focus on evaluating spectrum predictors in terms of both (a) their prediction accuracies (§4.2), using metrics such as 'cosine similarity', and (b) their use in generating virtual spectral libraries to assist with retrieval (§4.3).

3 Model

Here, we describe our model, SCARF , for predicting mass spectra from precursor molecules via first predicting subformulae of the precursor molecule, referred to as product formulae . Building upon the notation introduced in the previous section, we continue to denote precursor molecules 1 as M ∈ X , and their associated formula vector as F ∈ N e 0 , defining at each position, j ∈ { 1 , . . . , e } , the count of each possible chemical element present, F j (with zero indicating none of that chemical element is present). Likewise, we define the set of n product formulae as { f i } n i =1 , and associate with each an intensity, y i . Note that the mass 2 corresponding to a given formula (and, as such, the x-axis location of the peak on a mass spectrum) is determined deterministically from the counts of each elements present.

At a high level, SCARF generates mass spectra through the composition of two learned functions:

first mapping from the original molecule to a set of product formulae, g Thread θ : M ↦→ { f i } n i =1 , and then mapping from this set of formulae (and the original molecule) to the respective intensities, g Weave θ : ( { f i } n i =1 , M ) ↦→ { ( f i , y i ) } n i =1 . The particularities of both functions are described in detail below. The specific architectures and hyperparameters used are deferred to the appendix; model code can be found at https://github.com/samgoldman97/ms-pred .

3.1 SCARF-Thread : Generating product formulae via generating prefix trees

SCARF-Thread is tasked to learn a mapping to the set of product formulae, { f i } n i =1 , given the original molecule. Naively, one might try to define this model autoregressively, predicting the set formula by formula, chemical element by chemical element. However, such an approach soon runs into a number of problems as (i) the predictions are not invariant to set and ordering permutations; (ii) the time complexity of prediction would scale poorly, being proportional to both the number of elements and number of product formulae (i.e., O ( e × n ) ); and (iii) the predictions would likely contain duplicates.

C 7

C 12

C 14

C 7

O 1

H 4

C 7

O 1

C 7

O 2

C 12

O 1

C 14

O 1

C 12

C 7

O 1

H 6

C 7

O 2

H 8

C 12

H 7

C 12

H 8

C 12

O 1

H 8

C 12

O 1

H 9

C 14

O 1

H 10

Next element count prediction,

given current prefix and input molecule,

SCARF-Thread

NN

input context vector,

c'

(concatenated)

...

O 0

O 1

O 2

O 3

O 4

O 5

O 6

{

{

{

{

A

B

=C 16

O 4

H 12

during inference each level of

the tree (here, the oxygen count)

is expanded in parallel, as a

multi-label binary classification

task (right)

each possible

continuation

of the prefix

is predicted as

a binary label

prediction path

for

f

3

=C 7

O 2

H 8

invalid counts

are masked

H 4

H 6

H 8

H 7

H 8

H 8

H 9

H 10

O 2

O 0

C 14

C 12

C 7

O 1

O 1

O 1

Product formulae prefix tree

root

one-hot(O)

counts(C

7 ** )

counts(   - C

7 ** )

gnn(   )

We therefore take a different approach using the insight that the set of all product formulae can be compactly represented as a prefix tree (Figure 3A). In this tree, edges at a given depth represent valid counts of a particular chemical element, which are often identical across multiple product formulae (shown in the circles). By following each path from the root node to the different leaf nodes, we can reconstruct each product formula (as the orange dashed path does for a single product formula).

We thus propose SCARF-Thread as an autoregressive generator to define a probability distribution over such a prefix tree (Alg. A.1). We assume that each product formula is a subset of the precursor formula, meaning that the precursor formula sets an upper bound on the maximum number of each element 3 . At each node in the tree (corresponding to a prefix f ′ <j ), we pose the prediction of the set of child nodes (corresponding to the set of valid counts of the subsequent element) as a multi-label binary classification problem (Figure 3B). Concretely, we use a neural network module for this task, giving it as input a context vector representing the node being expanded:

where gnn ( · ) specifies a neural encoding of the molecular graph (§A.5.2), counts ( · ) specifies a countbased encoding of the associated prefix (§A.5.3), and one-hot ( · ) specifies a one-hot encoding of the node's depth (or equivalently, which element the predicted count is for). In our experiments, we use a fixed ordering of the chemical elements (§A.2), but optimizing or even learning the tree construction order could be carried out [45].

Formulae as differences. Following Wei et al. [50], we find it helpful to not only parameterize product formulae in terms of their element counts, but also in terms of the elements that they have lost,

i.e., their difference from the precursor formula. On the input side, this is already covered by including in the context vector a count-based embedding of the prefix formula minus the product formula ( counts ( F -f ′ <j ) ). However, on the output side this is achieved by combining the probabilities of a 'forward' and a 'difference' network:

where MLP F ( · ) and MLP D ( · ) specify multi-layer perceptrons (MLPs) for predicting the probability of observing a count of a and a loss of F j -a atoms respectively; α is a variable (output from a third, unshown network) deciding how to weight these predictions; and σ ( · ) is the element-wise sigmoid function.

3.2 SCARF-Weave : Predicting intensities given product formulae

peak

intensities

product formulae

embeddings

input context vector

c'

(concatenated)

{

{

{

gnn(   )

counts(   )

counts(    -   )

...

...

SCARF-Weave

Transformer

C 7

O 1

H 4

C 7

O 1

H 6

C 7

O 2

H 8

C 12

H 7

3.3 Training and inference

Provided with a dataset of molecules and formula-labeled mass spectra, we could train the two components of SCARF separately. However, in practice we find it beneficial to first train SCARF-Thread and then train SCARF-Weave on its outputs so that the distribution the latter model sees is the same at training and prediction time. SCARF-Weave is trained using a cosine loss (§A.5.5), as this most closely resembles the 'retrieval' setting (§ 4.3).

SCARF-Thread is trained using the binary cross entropy losses associated with the multi-label classification tasks at each non-leaf node in the prefix tree. We use teacher forcing, i.e., we train on each level of the tree in parallel by conditioning on the ground-truth set of prefixes at each stage. In our experiments, when generating the set of product formulae from this model we always pick the top 300. Empirically, we find that this provides better performance than picking a variable number based on a likelihood threshold.

4 Experiments

We evaluate SCARF on spectra prediction (§ 4.2) and molecule identification in a retrieval task (§ 4.3).

4.1 Dataset

We train and validate SCARF on two libraries: a gold standard commercial tandem mass spectrometry dataset, NIST20 [35], as well as a more heterogeneous public dataset, NPLIB1 , extracted from the

Given the product formulae outputs from SCARF-Thread , SCARF-Weave predicts corresponding intensities at each formula. This is a set-to-set problem, well suited for any equivariant set2set architecture [56, §3.1]. In our experiments, we use a Set Transformer [29, 44], which enables the model to consider all the formulae present in the mass spectrum (and their possible interactions) when predicting final intensities.

We choose to represent formula in the set similarly to the context vectors used in SCARF-Thread . For each input, we concatenate a vector embedding of the initial molecular graph with count-based embeddings of the product formula and its difference from the precursor formula (Figure 4). We again defer the particularities of the embedding functions to the Appendix (§A.5).

GNPS database [48] by Dührkop et al. [14] and subsequently processed by Goldman et al. [19]. We prepare both datasets by extracting and preprocessing spectra, as well as filtering to compounds that (a) are under 1,500 Da (i.e., typically under 100 heavy atoms), (b) only contain predefined elements, and (c) are only charged with common positive-mode adduct types (§4.1).

Overall, NIST20 contains 35,129 total spectra with 24,403 unique structures, and 12,975 unique molecular formulae; NPLIB1 contains 10,709 spectra, 8,553 unique structures and 5,433 unique molecular formulae. Both datasets are evaluated using a structure-disjoint 90%/10% train/test split with 10% of training data held out for validation, such that all compounds in the test set are not seen in the train and validation sets.

Annotating spectra. Weemphasize that SCARF can be trained with any product formula annotations, which can be labeled [35] or inferred with varied computational strategies [13]. Herein, we utilize the MAGMa algorithm [40]. In brief, for a given molecule-spectrum pair in the training dataset, the molecule is combinatorially fragmented at each atom up to a depth of 3 breakages to create subfragments. This creates a bank of possible molecular formulae, and each peak in the spectrum is assigned to its nearest possible formula within a mass difference of 20 parts-per-million.

4.2 Spectra prediction

Predicting product formulae ( SCARF-Thread ). SCARF-Thread is trained and used to reconstruct prefix trees and evaluated by its ability to recover the ground truth product formula set. The set of generated product formulae is rank-ordered by the probability of each product formula and filtered to the top k predicted product formulae. The fraction of ground truth formulae (22.29 peaks on average in NIST20 ) contained in the top k set is computed as coverage .

We compare coverage achieved by SCARF-Thread to several baselines: (i) CFM-ID [2], a fragmentation based approach (§ A.3.1); (ii) a random baseline that samples product formulae from a uniform distribution; (iii) a frequency baseline, which ranks product formulae by the frequency the product formula candidate (or product formula difference) appears in the training set; (iv) an LSTM autoregressive neural network baseline (§ A.3.2) that is trained to predict molecular formula vectors in sequence from highest to lowest intensity; and two model ablations, (v) SCARF-Thread -D and (vi) SCARF-Thread -F, which only make uni-directional elements difference or forward predictions of element counts respectively (i.e., α in Equation 3 is fixed to 0 for (v) and 1 for (vi)).

In general, SCARF-Thread starkly outperforms all baselines tested (Table 1). By generating 300 peaks, SCARF-Thread is able to cover on average 91% and 72% of the true formulae in the ground truth test set for NIST20 and NPLIB1 respectively. Our difference- and forward-only directional prediction ablations demonstrate the benefits of modeling both the atom counts for each element and the differences in counts from the original molecule.

Predicting mass spectra. We next evaluate the strength of SCARF-Weave for intensity prediction on the same test dataset. We compare against five baselines: a fragmentation-based approach, CFM-ID

[2]; two NEIMS binned prediction models ([50]; §A.3.3), using either feed forward network modules (FFNs), as in the original work, or graph neural network modules (GNNs) as in SCARF-Weave and described by Zhu et al. [58]; a retrained variant of 3DMolMS ([23]; §A.3.4), a binned spectrum predictor that utilizes a point cloud neural network over a single molecular conformer input generated by RDKit [39]; and FixedVocab, a formula prediction model that predicts intensities at a fixed library of formulae and formulae differences inspired by GRAFF-MS ([34]; §A.3.5).

To enable fair comparison across models, we predict test spectra at 15k bins (0.1 bin resolution between 0 and 1500 ) with a maximum of 100 peaks for each predicted molecule. With the exception of CFM-ID , all models are hyperparameter optimized (§A.5.6), retrained completely, and conditioned on the same covariate inputs as SCARF ; such steps lead to large performance boosts to the prior NEIMS method in particular. We evaluate the quality of our predictions based upon four core criteria reflecting our original desiderata of accuracy, physical-sensibleness, and speed:

Cosine sim. : Cosine similarity between the ground truth and predicted spectra, indicating spectrum prediction accuracy.

Coverage : The fraction of ground truth spectrum peaks covered by the predicted spectrum.

Valid : The fraction of predicted peaks that can be explained by a subformula (that obeys basic ring-double bond equivalent heuristics [37]) of the predicted molecule.

Time (s) : The wall time it takes (using a single CPU and no batched calculations) to load the model and predict spectra for 100 randomly selected molecules.

SCARF is more accurate than all other approaches on NIST20 , improving cosine similarity over a GNN binned prediction approach by over 0.02 points in NIST20 and 0.01 in NPLIB1 (Table 2). Further, our method is more physically grounded insofar as all predicted peaks are guaranteed to be valid subformulae, unlike the unconstrained binned approaches, where nearly 5% of peak predictions cannot be explained by a valid molecular formula. Importantly, SCARF still operates 2 orders of magnitude faster than CFM-ID (Table 2).

The heterogeneity, reduced dataset size, and increased average molecular weight (Figure A3) of NPLIB1 leads to substantially worse absolute performance across all models. Interestingly, in this setting, the FixedVocab approach [34] performs better, perhaps because the strict priors of formula constraints are more helpful with fewer and more challenging training examples. We further stratify results by molecule size in Figure A2, showing that all models are generally more accurate on smaller compounds. We additionally validate that cosine similarity is not merely measuring a model's ability to predict the parent mass peak by computing a modified cosine similarity with the original molecule's mass masked (Table A7).

4.3 Retrieval

A key application for forward spectrum prediction is to use predicted spectra to determine the most plausible molecular structure assignment. We posit forward spectrum prediction models should be

A

B

C

0

5

10

Top k

0.0

0.2

0.4

0.6

0.8

1.0

Random

3DMolMS

FixedVocab

NEIMS (FFN)

NEIMS (GNN)

SCARF

C 24

NO 2

H 21

C 11

OH 8

C 10

H 6

C 8

H 8

C 6

H 4

50

100

150

200

250

300

350

M/Z

1.5

1.0

0.5

0.0

0.5

1.0

1.5

Intensity

SCARF

True

O

N

O

50

100

150

200

M/Z

1.5

1.0

0.5

0.0

0.5

1.0

1.5

Intensity

SCARF

True

C 11

NO 4

H 21

C 7

NO 4

H 13

C 6

NO 2

H 13

O

O

H

N

O

OH

particularly helpful in differentiating structurally similar molecules and design a retrieval task to showcase such potential. For each test set molecule, we extract 49 potential 'decoy" options based upon the most structurally similar isomers (i.e., compounds with the same precursor formula) within PubChem [27] as judged by Tanimoto similarity using Morgan fingerprints. While retrieval could be conducted on the entirety of PubChem or other similarly large molecular databases, we believe this subset retrieval setting is more practical and better mirrors a real-world setting (see §A.4 for justification). We predict the spectra for all molecules and rank them according to their similarity to the ground truth spectrum, computing the accuracy for retrieval. Herein, we specifically emphasize models and retrieval on the NIST20 dataset, as it is a much larger and higher quality dataset.

SCARF reaches a top 1 and top 5 retrieval accuracy in this task of 18.7% and 54.1% respectively, representing an improvement over the methods with the second best top 1 accuracy of 17.5% (NEIMS (GNN)) and top 5 accuracy of 52.2% (Fixed Vocab) (Figure 5A). We highlight two example predictions from SCARF (Figure 5B-C), with additional randomly sampled test set predictions shown in Figure A.1. We repeat similar experiments within NPLIB1 , but find that cosine similarity performance is uncorrelated with relative ranking performance; feed forward fingerprint based approaches are better at retrieval, despite relatively weak cosine similarity (§A.1). FixedVocab [34] performs especially well on NPLIB1 , again likely due to the helpful biases imparted by constraining the formula vocabulary.

This result underscores previous observations regarding how database and model biases can skew retrieval results under certain settings [22]. That is, models may be more or less robust for certain classes of molecules, so the composition of these classes in the retrieval library may affect the retrieval accuracy accordingly. The observed discrepancy between cosine similarity and retrieval performance can further be explained by the dataset shift required for computing retrieval accuracy; cosine similarity is evaluated on 'in-domain' data, whereas retrieval relies also on accuracy on unlabeled data that may be 'out-of-domain.'

5 Related work

Forward vs backward models. Computational tools to identify mass spectra are often divided into two categories: forward and backward models. Forward models, i.e., spectrum predictors, such as SCARF or the methods discussed in Section 2, operate in the causal direction and try to predict the spectrum given the molecule. Backward models start from the spectrum and predict features or even full molecule structures. Early backward models used heuristics, expert rules, and even neural networks [8, 10, 42]. Such approaches have more recently been augmented with kernel methods and more modern, deep representation learning techniques [12, 16, 19, 47]. These models are complementary to spectrum predictors.

Mass spectra for proteomics. Although this paper has focused on small molecules, similar trends of deep representation learning for mass spectra are also emerging in the adjacent field of proteomics [54], with Shouman et al. [41] recently proposing a benchmark challenge in this domain. While small molecule and protein spectra are similar, proteomics spectra tend to be more easily predicted as fragments are often formed at peptide bonds. We believe adapting SCARF to this task would be an interesting direction for future work.

Neural set generation. Our work is also related to methods for modeling sets and multisets. SCARF-Thread generates a set as output, which has been studied elsewhere in the context of n-gram generation [45], object detection [32, 57], and point cloud generation [28]. The product formulae sets we generate, however, are different to those considered in these other task; in our setting, each member of the set (i.e., individual product formula) represents a multiset of atom types (i.e., multiple carbons, multiple hydrogens, etc.) and is constrained physically by the precursor formula.

6 Conclusion

In this paper we introduced SCARF , an approach utilizing prefix tree data structures to efficiently decode mass spectra from molecules. By first predicting product formulae and then assigning these formulae intensities, we are able to combine the advantages of previous neural and fragment based approaches, providing fast and physically grounded predictions. We show how these resulting predictions are both more accurate in predicting experimentally-observed spectra and yield improvements in identifying a molecule's structure from its respective mass spectrum, as tested on a widely used dataset.

In term of limitations, our model is data dependent, as indicated by the relative performance across the NIST20 and NPLIB1 datasets. SCARF is also highly reliant upon the quality of product formula label assignment. The current commercial status of mass spectrometry training data poses a barrier to entry, and identifying high quality public domain data is critical for future studies. A key contribution of this work is to retrain and optimize the hyperparameters of competing methods on a publicly available dataset under equivalent conditions to allow for future extensions. Directly training on a ranking-based loss or learning a model specific spectra distance function may be one way to improve upon our model's performance in the retrieval setting, and we outline additional potential ideas more explicitly in §A.6.

Future directions will involve further developing SCARF for real world use cases such as unknown metabolite elucidation in clinical samples. Specific directions will include more carefully modeling covariates (e.g., collision energies and MS/MS instrument types), grounding product formulae in molecular graph substructures, and utilizing such models to augment inverse spectrum-to-molecule annotation tools.

References

Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining , pages 2623-2631, 2019.

Felicity Allen, Russ Greiner, and David Wishart. Competitive fragmentation modeling of ESI-MS/MS spectra for putative metabolite identification. Metabolomics , 11(1):98-110, 2015.

Peter W Battaglia, Jessica B Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalez, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner,

Caglar Gulcehre, Francis Song, Andrew Ballard, Justin Gilmer, George Dahl, Ashish Vaswani, Kelsey Allen, Charles Nash, Victoria Langston, Chris Dyer, Nicolas Heess, Daan Wierstra, Pushmeet Kohli, Matt Botvinick, Oriol Vinyals, Yujia Li, and Razvan Pascanu. Relational inductive biases, deep learning, and graph networks. arXiv preprint arXiv:1806.01261 , 2018.

Anelize Bauermeister, Helena Mannochio-Russo, Letícia V Costa-Lotufo, Alan K Jarmusch, and Pieter C Dorrestein. Mass spectrometry-based metabolomics in microbiome investigations. Nature Reviews Microbiology , 20(3):143-160, 2022.

Wout Bittremieux, Robin Schmid, Florian Huber, Justin J J van der Hooft, Mingxun Wang, and Pieter C Dorrestein. Comparison of cosine, modified cosine, and neutral loss based spectrum alignment for discovery of structurally related molecules. Journal of the American Society for Mass Spectrometry , 33(9):1733-1744, 2022.

Wout Bittremieux, Mingxun Wang, and Pieter C Dorrestein. The critical role that spectral libraries play in capturing the metabolomics community knowledge. Metabolomics , 18(12):94, 2022.

Michael M Bronstein, Joan Bruna, Taco Cohen, and Petar Veliˇ ckovi´ c. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. arXiv preprint arXiv:2104.13478 , 2021.

Bruce Buchanan, Georgia Sutherland, and Edward A. Feigenbaum. Heuristic Dendral: A program for generating explanatory hypotheses. Machine Intelligence , 4:209-254, 1969.

Jing Chen, Yi Chen, Yang Jiang, Hua Fu, Bin Xin, and Yu-Fen Zhao. Rearrangement of P-N to P-O bonds in mass spectra of N-diisopropyloxyphosphoryl amino acids/alcohols. Rapid Communications in Mass Spectrometry , 15(20):1936-1940, 2001.

Bo Curry and David E. Rumelhart. MSnet: A neural network which classifies mass spectra. Tetrahedron Computer Methodology , 3(3-4):213-237, 1990.

Daniel P Demarque, Antonio EM Crotti, Ricardo Vessecchi, João LC Lopes, and Norberto P Lopes. Fragmentation reactions using electrospray ionization mass spectrometry: an important tool for the structural elucidation and characterization of synthetic and natural products. Natural Product Reports , 33(3):432-455, 2016.

Kai Dührkop, Huibin Shen, Marvin Meusel, Juho Rousu, and Sebastian Böcker. Searching molecular structure databases with tandem mass spectra using CSI:FingerID. Proceedings of the National Academy of Sciences , 112(41):12580-12585, 2015.

Kai Dührkop, Markus Fleischauer, Marcus Ludwig, Alexander A. Aksenov, Alexey V. Melnik, Marvin Meusel, Pieter C. Dorrestein, Juho Rousu, and Sebastian Böcker. SIRIUS 4: a rapid tool for turning tandem mass spectra into metabolite structure information. Nature Methods , 16 (4), 2019.

Kai Dührkop, Louis-Félix Nothias, Markus Fleischauer, Raphael Reher, Marcus Ludwig, Martin A. Hoffmann, Daniel Petras, William H. Gerwick, Juho Rousu, and Pieter C. Dorrestein. Systematic classification of unknown metabolites using high-resolution fragmentation mass spectra. Nature Biotechnology , 39(4):462-471, 2021.

William Falcon and The PyTorch Lightning team. PyTorch Lightning, 3 2019. URL https: //github.com/Lightning-AI/lightning .

Ziling Fan, Amber Alley, Kian Ghaffari, and Habtom W. Ressom. MetFID: artificial neural network-based compound fingerprint prediction for metabolite annotation. Metabolomics , 16 (10):104, 2020. doi: 10.1007/s11306-020-01726-7.

J Gasteiger, W Hanebeck, and K P Schulz. Prediction of mass spectra from structural information. Journal of chemical information and computer sciences , 32(4):264-271, 1992.

Samuel Goldman, Janet Li, and Connor W Coley. Generating molecular fragmentation graphs with autoregressive neural networks. arXiv preprint arXiv:2304.13136 , 2023.

Samuel Goldman, Jeremy Wohlwend, Martin Stra ž ar, Guy Haroush, Ramnik J Xavier, and Connor W Coley. Annotating metabolite mass spectra with domain-inspired chemical formula transformers. Nature Machine Intelligence , pages 1-15, 2023.

William L Hamilton. Graph representation learning. Synthesis Lectures on Artificial Intelligence and Machine Learning , 14(3):1-159, 2020.

Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation , 9(8): 1735-1780, 1997.

Martin A Hoffmann, Fleming Kretschmer, Marcus Ludwig, and Sebastian Böcker. Mad Hatter correctly annotates 98% of small molecule tandem mass spectra searching in PubChem. Metabolites , 13(3):314, 2023.

Yuhui Hong, Sujun Li, Christopher J Welch, Shane Tichy, Yuzhen Ye, and Haixu Tang. 3dmolms: Prediction of tandem mass spectra from three dimensional molecular conformations. bioRxiv , pages 2023-03, 2023.

Florian Huber, Lars Ridder, Stefan Verhoeven, Jurriaan H Spaaks, Faruk Diblen, Simon Rogers, and Justin J J van der Hooft. Spec2Vec: Improved mass spectral similarity scoring through learning of structural relationships. PLoS computational biology , 17(2):e1008724, 2021.

Franziska Hufsky, Kerstin Scheubert, and Sebastian Böcker. Computational mass spectrometry for small-molecule fragmentation. TrAC Trends in Analytical Chemistry , 53:41-48, 2014.

Hyun Woo Kim, Mingxun Wang, Christopher A Leber, Louis-Félix Nothias, Raphael Reher, Kyo Bin Kang, Justin JJ Van Der Hooft, Pieter C Dorrestein, William H Gerwick, and Garrison W Cottrell. Npclassifier: A deep neural network-based structural classification tool for natural products. Journal of natural products , 84(11):2795-2807, 2021.

Sunghwan Kim, Paul A. Thiessen, Evan E. Bolton, Jie Chen, Gang Fu, Asta Gindulyte, Lianyi Han, Jane He, Siqian He, and Benjamin A. Shoemaker. PubChem substance and compound databases. Nucleic Acids Research , 44(D1):D1202-D1213, 2016.

Adam R Kosiorek, Hyunjik Kim, and Danilo J Rezende. Conditional set generation with transformers. In Workshop on Object-Oriented Learning at ICML 2020 , 2020.

Juho Lee, Yoonho Lee, Jungtaek Kim, Adam Kosiorek, Seungjin Choi, and Yee Whye Teh. Set transformer: A framework for attention-based permutation-invariant neural networks. In Proceedings of the 36th International Conference on Machine Learning , pages 3744-3753, 2019.

Yujia Li, Daniel Tarlow, Marc Brockschmidt, and Richard Zemel. Gated graph sequence neural networks. In International Conference on Learning Representations , 2016.

Richard Liaw, Eric Liang, Robert Nishihara, Philipp Moritz, Joseph E Gonzalez, and Ion Stoica. Tune: A research platform for distributed model selection and training. ICML 2018 AutoML Workshop , 2018.

Francesco Locatello, Dirk Weissenborn, Thomas Unterthiner, Aravindh Mahendran, Georg Heigold, Jakob Uszkoreit, Alexey Dosovitskiy, and Thomas Kipf. Object-centric learning with slot attention. In Advances in Neural Information Processing Systems 33 , 2020.

Marcus Ludwig, Louis-Félix Nothias, Kai Dührkop, Irina Koester, Markus Fleischauer, Martin A. Hoffmann, Daniel Petras, Fernando Vargas, Mustafa Morsy, and Lihini Aluwihare. Database-independent molecular formula annotation using Gibbs sampling through ZODIAC. Nature Machine Intelligence , 2(10):629-641, 2020.

Michael Murphy, Stefanie Jegelka, Ernest Fraenkel, Tobias Kind, David Healey, and Thomas Butler. Efficiently predicting high resolution mass spectra with graph neural networks. arXiv preprint arXiv:2301.11419 , 2023.

NIST. Tandem Mass Spectral Library. NIST , 2020. URL https://www.nist.gov/ programs-projects/tandem-mass-spectral-library .

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An imperative style, high-performance deep learning library. arXiv preprint arXiv:1912.01703 , 2019.

Ernö Pretsch, Philippe Bühlmann, Christian Affolter, Ernho Pretsch, P Bhuhlmann, and C Affolter. Structure determination of organic compounds . Springer, 2000.

Robert A Quinn, Alexey V Melnik, Alison Vrbanac, Ting Fu, Kathryn A Patras, Mitchell P Christy, Zsolt Bodai, Pedro Belda-Ferre, Anupriya Tripathi, Lawton K Chung, et al. Global chemical effects of the microbiome include new bile-acid conjugations. Nature , 579(7797): 123-129, 2020.

RDKit Team. RDKit: Open-source cheminformatics, 2019. URL https://www.rdkit.org/ .

Lars Ridder, Justin JJ van der Hooft, and Stefan Verhoeven. Automatic compound annotation from mass spectrometry data using MAGMa. Mass Spectrometry , 3(Spec Iss 2):S0033, 2014.

Omar Shouman, Wassim Gabriel, Victor-George Giurcoiu, Vitor Sternlicht, and Mathias Wilhelm. PROSPECT: Labeled tandem mass spectrometry dataset for machine learning in proteomics. In Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track , 2022.

Georgia Sutherland. Dendral-a computer program for generating and filtering chemical structures. Technical report, Stanford University, Department of Computer Science, 1967.

Matthew Tancik, Pratul P Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan T Barron, and Ren Ng. Fourier features let networks learn high frequency functions in low dimensional domains. In Advances in Neural Information Processing Systems 33 , pages 7537-7547, 2020.

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems 30 , pages 5998-6008, 2017.

Oriol Vinyals, Samy Bengio, and Manjunath Kudlur. Order matters: Sequence to sequence for sets. In International Conference on Learning Representations , 2016.

Gennady Voronov, Abe Frandsen, Brian Bargh, David Healey, Rose Lightheart, Tobias Kind, Pieter Dorrestein, Viswa Colluru, and Thomas Butler. MS2Prop: A machine learning model that directly predicts chemical properties from mass spectrometry data for novel compounds. bioRxiv , 2022. doi: 10.1101/2022.10.09.511482. URL https://www.biorxiv. org/content/early/2022/10/10/2022.10.09.511482 .

Gennady Voronov, Rose Lightheart, Joe Davison, Christoph A. Krettler, David Healey, and Thomas Butler. Multi-scale Sinusoidal Embeddings Enable Learning on High Resolution Mass Spectrometry Data. arXiv preprint arXiv:2207.02980 , 2022.

Mingxun Wang, Jeremy J. Carver, Vanessa V. Phelan, Laura M. Sanchez, Neha Garg, Yao Peng, Don Duy Nguyen, Jeramie Watrous, Clifford A. Kapono, and Tal Luzzatto-Knaan. Sharing and community curation of mass spectrometry data with Global Natural Products Social Molecular Networking. Nature biotechnology , 34(8):828-837, 2016.

Minjie Wang, Da Zheng, Zihao Ye, Quan Gan, Mufei Li, Xiang Song, Jinjing Zhou, Chao Ma, Lingfan Yu, Yu Gai, Tianjun Xiao, Tong He, George Karypis, Jinyang Li, and Zheng Zhang. Deep graph library: A graph-centric, highly-performant package for graph neural networks. arXiv preprint arXiv:1909.01315 , 2019.

Jennifer N. Wei, David Belanger, Ryan P. Adams, and D. Sculley. Rapid prediction of electron-ionization mass spectrometry using neural networks. ACS Central Science , 5(4):700-708, 2019.

David S. Wishart. Metabolomics for investigating physiological and pathophysiological processes. Physiological Reviews , 99(4):1819-1875, 2019. doi: 10.1152/physrev.00035.2018.

Sebastian Wolf, Stephan Schmidt, Matthias Müller-Hannemann, and Steffen Neumann. In silico fragmentation for computer assisted identification of metabolite mass spectra. BMC Bioinformatics , 11(1):148, 2010. doi: 10.1186/1471-2105-11-148.

Shipei Xing, Sam Shen, Banghua Xu, and Tao Huan. Molecular formula discovery via bottomup MS/MS interrogation. bioRxiv , 2022. doi: 10.1101/2022.08.03.502704. URL https: //www.biorxiv.org/content/early/2022/08/05/2022.08.03.502704 .

Melih Yilmaz, William Fondrie, Wout Bittremieux, Sewoong Oh, and William S. Noble. De novo mass spectrometry peptide sequencing with a transformer model. In Proceedings of the 39th International Conference on Machine Learning , pages 25514-25522, 2022.

Adamo Young, Bo Wang, and Hannes Röst. MassFormer: Tandem Mass Spectrum Prediction with Graph Transformers. arXiv preprint arXiv:2111.04824 , 2021.

Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Ruslan Salakhutdinov, and Alexander Smola. Deep sets. In Advances in Neural Information Processing Systems 30 , pages 3391-3401, 2017.

Yan Zhang, Jonathon Hare, and Adam Prugel-Bennett. Deep set prediction networks. In Advances in Neural Information Processing Systems 32 , 2019.

Hao Zhu, Liping Liu, and Soha Hassoun. Using Graph Neural Networks for Mass Spectrometry Prediction. In Machine Learning for Molecules Workshop at NeurIPS 2020 , 2020.

Richard Licheng Zhu and Eric Jonas. Rapid approximate subset-based spectra prediction for electron ionization-mass spectrometry. Analytical Chemistry , 2023.

A Appendix

A.1 Extended results

We benchmark models in terms of retrieval accuracy as described (§ 4.3) for both the NIST20 and NPLIB1 datasets (Table A1, A2). We recreate retrieval experiments using the full PubChem retrieval library in Table A3. We reproduce main text results with standard error values included in Tables A4, A5, and A6. We showcase additional spectra predictions from our model trained on NIST20 in Figure A1.

A.2 Dataset preparation

NIST20 [35] is prepared by extracting all positive-mode experimental spectra collected in higherenergy collision-induced dissociation (HCD) mode (i.e., collected on Orbitrap mass spectrometers). Spectra are filtered, so that we keep only those for which the associated molecule (M) has (i) a mass under 1,500 Da, (ii) contains only elements from a predefined set (i.e,. 'C', 'N', 'P', 'O', 'S', 'Si', 'I', 'H', 'Cl', 'F', 'Br', 'B', 'Se', 'Fe', 'Co', 'As', 'Na', 'K'), and (iii) is charged with common adduct types (i.e., '[M+H]+", '[M+Na]+", '[M+K]+", '[M-H2O+]+", "[M+NH3+H]+", and "[M-2H2O+H]+"). Because non-standard empirical spectra databases [48] often do not include the measured collision energies, we pool all collision energies for each compound-adduct pairing to create a single spectrum. We refer the reader to Young et al. [55] for detailed instructions for purchasing and extracting the NIST20 dataset.

All spectrum intensities are square-root transformed to provide higher weighting to lower intensity peaks, normalized to a maximum intensity of 1 (i.e., through dividing by the maximum intensity),

50

100

150

200

250

300

350

M/Z

1.5

1.0

0.5

0.0

0.5

1.0

1.5

Intensity

SCARF

True

100

200

300

400

M/Z

1.5

1.0

0.5

0.0

0.5

1.0

1.5

Intensity

SCARF

True

100

200

300

400

500

M/Z

1.5

1.0

0.5

0.0

0.5

1.0

1.5

Intensity

SCARF

True

100

200

300

400

500

600

M/Z

1.5

1.0

0.5

0.0

0.5

1.0

1.5

Intensity

SCARF

True

100

150

200

250

300

350

M/Z

1.5

1.0

0.5

0.0

0.5

1.0

1.5

Intensity

SCARF

True

25

50

75

100

125

150

175

M/Z

1.5

1.0

0.5

0.0

0.5

1.0

1.5

Intensity

SCARF

True

50

100

150

200

250

M/Z

1.5

1.0

0.5

0.0

0.5

1.0

1.5

Intensity

SCARF

True

75

100

125

150

175

200

225

250

M/Z

1.5

1.0

0.5

0.0

0.5

1.0

1.5

Intensity

SCARF

True

50

75

100

125

150

175

200

M/Z

1.5

1.0

0.5

0.0

0.5

1.0

1.5

Intensity

SCARF

True

0

100

200

300

400

M/Z

1.5

1.0

0.5

0.0

0.5

1.0

1.5

Intensity

SCARF

True

filtered to exclude any noise peaks with normalized intensity under 0 . 003 , and subsetted to only the top 50 highest intensity peaks. All peaks are mass-shifted by the weight of the parent adduct (i.e,. if the spectrum is '[M+H]+', the weight of a proton is subtracted from each child peak).

A.2.1 Product formulae assignments

Because the precursor ion and adduct species are known for the training dataset, we subtract the precursor adduct mass from every peak in the training set, and attempt to annotate each peak with a plausible product formula (i.e., a subset of the true precursor formula).

We opt to constrain the training product formulae to be subsets of contiguous heavy atoms of the parent molecule as derived with the MAGMa algorithm [40].

We note two important limitations of these heuristics. First, by using molecular substructures to annotate product formulae, our model is less prone to correctly identifying complex rearrangements. Second, it is also possible for adduct switching to occur. Namely, if the precursor ion has a sodium adduct ('[M+Na]+'), some of the product formulae may actually switch and acquire a hydrogen adduct instead. We assume no adduct switching in our formulation, instead focusing on the novelty of the prefix tree decoding approach, as these represent data labeling challenges, rather than modeling challenges.

In addition, any predictive models of product formulae distributions will more closely predict spectra that would be produced on instrumentation similar to the training sets utilized [9, 11]. Given this, we encourage users of such models to treat these predictions as putative, rather than experimentally valid.

±

±

±

±

±

±

A.2.2 Dataset statistics

To probe the composition of our two primary datasets, we investigate both the molecular weight and chemical classes contained in the NPLIB1 and NIST20 datasets. We find that the average molecular weight is higher for NPLIB1 (Figure A3A), consistent with the increased complexity of natural product molecules. We additionally compute chemical classes of the compounds using NPClassifier [26] to identify the types of compounds present in both datasets (Figure A3B-C). NPLIB1 is enriched for steroids, coumarins, and various complex alkaloid natural products. On the other hand, NIST20 is enriched for small peptides, nicotinic acid alkaloids, and pseudoalkaloids, among others. While these descriptions are helpful to identify the dataset composition, chemical compound classification is itself a learned classification and should be interpreted cautiously.

A

B

NPLIB1

C

D

NPLIB1

0 - 200

200 - 300

300 - 400

400 - 500

500 - 600

600 - 700

700 - 2000

Compound mass (Da)

0.00

0.25

0.50

0.75

1.00

Cosine sim.

CFM-ID

3DMolMS

FixedVocab

NEIMS (FFN)

NEIMS (GNN)

SCARF

0 - 200

200 - 300

300 - 400

400 - 500

500 - 600

600 - 700

700 - 2000

Compound mass (Da)

0.00

0.25

0.50

0.75

1.00

Cosine sim.

CFM-ID

3DMolMS

FixedVocab

NEIMS (FFN)

NEIMS (GNN)

SCARF

Small peptides

Anthranilic acid alkaloids

Steroids

Flavonoids

Tryptophan alkaloids

Nicotinic acid alkaloids

Pseudoalkaloids

Sesquiterpenoids

Coumarins

Lysine alkaloids

Compound superclass

0.00

0.25

0.50

0.75

1.00

Cosine sim.

CFM-ID

3DMolMS

FixedVocab

NEIMS (FFN)

NEIMS (GNN)

SCARF

Flavonoids

Steroids

Coumarins

Small peptides

Tryptophan alkaloids

Sesquiterpenoids

Triterpenoids

Diterpenoids

Monoterpenoids

Nicotinic acid alkaloids

Compound superclass

0.00

0.25

0.50

0.75

1.00

Cosine sim.

CFM-ID

3DMolMS

FixedVocab

NEIMS (FFN)

NEIMS (GNN)

SCARF

A

B

C

NPLIB1

500

1000

1500

Mol. weight

0.000

0.001

0.002

0.003

0.004

Frequency

NPLIB1

Other

29.6%

Steroids

11.8%

Flavonoids

8.4%

Coumarins

7.5%

Tryptophan alkaloids

7.0%

Small peptides

6.5%

Diterpenoids

3.9%

Sesquiterpenoids

3.7%

Triterpenoids

3.3%

Monoterpenoids

3.1%

Nicotinic acid alkaloids

2.9%

Anthranilic acid alkaloids

2.8%

Pseudoalkaloids

2.5%

Oligopeptides

2.4%

Tyrosine alkaloids

2.3%

Isoflavonoids

2.2%

Other

31.3%

Small peptides

9.3%

Anthranilic acid alkaloids

7.5%

Flavonoids

6.1%

Tryptophan alkaloids

6.0%

Nicotinic acid alkaloids

5.6%

Pseudoalkaloids

5.2%

Steroids

5.2%

Coumarins

3.4%

Lysine alkaloids

3.4%

Sesquiterpenoids

3.4%

Triterpenoids

3.1%

Diterpenoids

2.9%

Phenolic acids (C6-C1)

2.7%

Monoterpenoids

2.6%

Tyrosine alkaloids

2.3%

A.3 Baselines

We further describe select baseline models.

A.3.1 CFM-ID baseline

CFM-ID [2] is a long standing and important approach to fragmentation prediction. Because CFM-ID is fit using a time intensive EM training approach on an analogous dataset, we utilize the pretrained Docker implementation provided by the authors in line with [34]. CFM-ID has two options for predicting molecules in either positive or negative adduct mode with 'H' adducts (i.e., '[M+H]+' or '[M-H]-'). To directly compare to our method, we predict spectra in positive mode and remove hydrogens from all predicted peaks, as all training peaks are shifted by removing their adducts.

CFM-ID also produces predictions at three collision energies (i.e., low , medium, or high fragmentation). Because we opt not to include these, we merge these predictions and re-normalize the result to a maximum of 1 .

A.3.2 Autoregressive baseline

When considering the task of generating spectrum formulae candidates, we compare SCARF-Thread to an autoregressive recurrent neural network baseline, which is based around a long short term memory (LSTM) module [21].

The LSTM generates formulae consecutively from a single concatenated encoding of the input molecule and input full formula. At each step in the recurrent process, a one-hot encoding of the previous predicted element count is concatenated to a one-hot encoding of the element type being predicted in the current step. By embedding this information into the network, we can avoid predicting element types that do not appear in the parent molecule's molecular formula. If the parent molecular formula has 5 element types, each autoregressively predicted formula requires generating only 5 element type counts; this eliminates the need for a stop condition between each formula. Formulae are generated autoregressively, from highest to lowest intensity. When predicting the counts of the next element type, we employ the same difference and forward count prediction strategy as used in SCARF for fair comparison. The model is trained with a cross entropy loss and full hyperparameters are listed in Table A10.

A.3.3 NEIMS baseline

NEIMS [50] is a highly efficient binned spectrum prediction approach, originally developed for gas chromatography-mass spectra (GC-MS). To enable a fair comparison, we optimize its hyperparameters on our dataset and add higher resolution bins. Furthermore, we also train a graph neural network-based version 'NEIMS (GNN)' (in addition to the network that more closely matches Wei et al. [50]'s original model and operates on the molecular fingerprint, 'NEIMS (FFN)'). The adduct type is either concatenated to all atom features (for NEIMS (GNN)) or to the fingerprint vector (for NEIMS (FFN)).

A.3.4 3DMolMS baseline

3DMolMS [23] is a binned spectrum prediction approach developed simultaneously to this work. Unlike the other binned NEIMS approach, 3DMolMS utilizes a point-based deep neural network model operating on the point cloud of an input molecule. To project a 2D molecule or SMILES string into 3D space, a single 3D conformer is first generated using RDKit [39]. After several 3D convolutions, the atom-wise representations are pooled, covariates corresponding to the settings of the machine and experiment are concatenated, and the result is projected into a fixed length binned spectrum.

To enable a fair comparison, we copy the 3DMolMS architecture into our modeling framework with minor tweaks to the network. Rather than use variable sized hidden layers, we fix hidden layer sizes to a single value across convolutions. In addition, we only use the covariate of the adduct type for consistency with our model, excluding collision energy and instrument type. We hyperparameter optimize the model independently. We find that the performance of this model is substantially lower than the NEIMS baseline, likely due to the additional use of the 'difference' prediction module in the NEIMS approach that allows the network to predict intensities at both fragments and neutral losses.

A.3.5 FixedVocab baseline

Concurrently to our work, Murphy et al. introduce an alternative formula prediction strategy for mass spectrum prediction, a model they term GRAFF-MS. Unlike SCARF , GRAFF-MS utilizes a fixed vocabulary of molecular formulae and molecular formula differences, predicting intensities at each such value without learning to encode the formulae. These formulae and formula differences are selected in a greedy fashion based upon their frequency in the training set.

Because no code was released for this approach at the time this work was conducted, we reimplement a variant of their approach that emphasizes the use of a fixed vocabulary of formulae and differences. For methodological consistency, we utilize equivalent formula annotations as used by SCARF (i.e., one annotation per peak), do not model collision energies or instrument types, and utilize the same graph encoder as SCARF for encoding each molecule. We treat the number of fragment and difference formulae as a tunable hyperparameter (which is optimized along with the rest of the hyperparameters - see §A.5.6). We mask all invalid formulae and differences and utilize a cosine similarity loss with the original spectrum to train the model. To convert predicted formula and difference intensities into

a binned spectrum, each formula-intensity pairing is projected into its respective binned position using a scatter max calculation. We note that because alternative adducts and isotopes are not labeled in our preprocessing step, we do not predict isotopic or adduct variants for each fragment.

Given the differences between codebases, it is possible that the performance of our reimplementation does not exactly match the original implementation, and we instead refer to it as 'FixedVocab' rather than GRAFF-MS in table presentations. An earlier version of our work understated the FixedVocab model's performance due to an implementation decision along these lines (specifically, not including the '0' neutral loss as a predicted vocabulary entry). This has since been rectified, increasing the accuracy of the FixedVocab model.

A.4 Retrieval subsets vs. PubChem

We restrict our retrieval experiments in §4.3 to only the top 49 decoys per test case for two reasons. First, from a practical perspective, running the forward model on every isomer match in PubChem (approx. thousands each, >1,000,000 for only 500 test cases) makes benchmarking across all considered models substantially more challenging both for this work and also future work. Second, we also believe that this top 50 challenge represents a more realistic setting. In practice, retrieval compound libraries will often be carefully crafted and designed to contain molecules similar to the unknown molecule rather than all possible isomers (using either prior knowledge or 'backward' models such as CSI:FingerID [12] and MIST [19]). We conduct a side-by-side analysis on a small 500 molecule subset of the test set comparing the setting described above (with 49 decoys) to a setting with no limit on the number of decoys. The results are shown in Table A3, showing that SCARF still performs well in this setting.

A.5 Model details

Here, we describe details of our model's training setup, architecture, and hyperparameters that were omitted from the main text. Definitive details can also be found in the code at https://github. com/samgoldman97/ms-pred .

A.5.1 Training

We train each of our models on a single RTX A5000 NVIDIA GPU (CUDA Version 11.6), making use of the Torch Lightning [15] library to manage the training. SCARF-Thread and SCARF-Weave take on the order of 1.5 and 2.5 hours of wall time to train respectively.

A.5.2 Molecule encoding

Within both SCARF-Thread and SCARF-Weave , a key component is an encoding of the molecular graph using a message passing graph neural network, gnn ( M ) . Such graph neural network models are now well described [3, 7, 20], so we will skip a detailed explanation of them here. In our experiments, we use gated graph sequence neural networks [30]. We made use of the implementation of this network in the DGL library [49] and use as atom features those shown in Table A9 (which are computed using RDKit [39] or DGL [49]).

A.5.3 Molecular formulae representations

When forming representations of formulae (including formulae prefixes) we use a count-based encoder, counts ( f ) . This encoder takes in as input the counts of all individual elements in the formula (which also can be 'undefined' for counts of elements not yet specified - indicated as ' ∗ ' in Figure 3B) and returns a vector representation in R d . The encoder is based upon the Fourier feature mapping proposed by Tancik et al. [43], but using only sin basis functions (to reduce the number of parameters required by our networks). Tancik et al. [43] has shown that such features perform better than encoding integers directly; furthermore, compared to learned representations, using Fourier features enables us (at least in principle) to deal with counts at test time that have not been seen during training.

To be precise, each possible count, v ∈ N 0 , is encoded by our counts-based encoder into the vector:

where the periods ( T 1 , T 2 , etc.) are set at increasing powers of two that enable us to discriminate all possible element counts given in the input, and abs ( · ) is the absolute value function such that we get positive embeddings. For the 'undefined' count we learn a separate encoding of the same dimensionality.

A.5.4 Further details of SCARF-Thread

Pseudo-code for the SCARF-Thread model is shown in Algorithm A.1. Note that the second for loop (on the line marked ‡ ) does not depend on previous iterations of the loop, so that in practice we perform this computation in parallel. At training time we use teacher forcing (§3.3), meaning the first for loop (marked † ) is only run sequentially at inference time.

The function scarf-thread-net ( · ) represents the network shown in Figure 3B and generates the set of subsequent valid element counts given a prefix (i.e., the child nodes of a given prefix node). As discussed in the main text, we treat this as a multi-label binary classification task and predict the binary label for each possible count using forward and difference MLPs (Eq. 3). We fix a maximum possible element count (i.e., the number of possible classes in this classification problem), N = 160 . We do not allow product formulae to have more of a given element than is present in the precursor formula, F , and we achieve this by setting the probability of these classes to zero.

A.5.5 Further details of SCARF-Weave

As discussed in the main text, SCARF-Weave is based off Lee et al. [29]'s Set Transformer. After forming the input encoding using the molecule and count-based encoder (§A.5.2 & §A.5.3), we further refine this embedding using an MLP (multi-layer perceptron) network. The output of this is passed into a series of l 3 Transformer [44] layers (§A.5.6 defines the exact number used in the experiments) with 8 attention heads each.

We use a cosine distance loss to train the parameters of SCARF-Weave . This loss is also used for the FFN and GNN baselines (Table 2). To ensure consistency with the baselines, we first project the output of SCARF-Weave into a binned histogram representation (§A.5.6 defines the number of bins used); for each bin we take the max intensity across all applicable formulae. Given a predicted binned spectra, ˆ s , and the ground-truth binned spectra, s , the cosine distance is defined as the negative of the cosine similarity (computed using PyTorch's torch.cosine_similarity function [36]):

where /epsilon1 = 1 × 10 -8 is used to ensure numerical stability.

A.5.6 Hyperparameters

To enable fair comparison across models, hyperparameters were tuned for SCARF , the FFN binned prediction baseline, and the GNN binned prediction baseline. Parameters were tuned using RayTune [31] with Optuna [1] and an ASHAScheduler. Each model was allotted 50 different hyperoptimization trials for fitting. Models were hyperparameter optimized on a smaller 10 , 000 spectra subset of NIST20 . Parameters are detailed in Table A10.

A.6 Limitations and future work

We outline several potential directions for future work to address limitations of this work.

Improving the gold standard training annotation pre-processing. Because SCARF is flexible in that it can match distributions of formula assignments, a key step to improving and building upon this approach is to develop more robust assignments of formula to training spectra. This includes adding complexity and removing potential assumptions, such as allowing annotations to account for rearrangement, elimination, or charge transfer. A second goal is to identify potentially low quality training spectra, such as ones that emerge from mixtures, and remove these from the inputs. Another potential way to handle such cases would be to model each spectrum peak as an ensemble of potential equivalent-mass formulae, which would be particularly helpful in relating SCARF to inverse models such as MIST [19] in which the structure of the molecule and formula identity of each peak cannot be known a priori .

Incorporating other model covariates. Incorporating collision energy features explicitly into the model, as well as negative-ion mode inputs, will increase its usability. This could be enabled by aggregating public data containing these annotations.

Featurizing molecule inputs using different or more powerful molecular encoders. Recent and simultaneous work to this used a pretrained graph encoder as part of a binned spectrum prediction approach, MassFormer [55]. It is possible to include more powerful molecule or formula encoders into SCARF .

Consideration of interpretability by subgraph attribution and combination with ICEBERG. Following this initial work, we developed a second model, ICEBERG [18], that uses a similar two step modeling approach, but instead encodes fragments, not formula. This increases accuracy and robustness, especially for retrieval, but substantially slows the model. In comparison to ICEBERG, SCARF still has several benefits including speed, the lack of required substructure labeling, and ability to capture potential skeletal rearrangements of molecules (i.e., discontinuities in structure that may not be possible to model by only breaking bonds). An open question and exciting opportunity in the future is to combine these two levels of abstraction and make formula-level predictions with graph-level attribution or featurization.

Retrieval-specific loss functions to enhance retrieval performance. A significant finding of this work was the noise associated with the retrieval task and lack of correlation with spectrum prediction performance as measured by cosine similarity. Future work may consider how to more directly define loss functions that reflect the task of retrieval.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel method for predicting a tandem mass spectrum from a given molecular graph.  The key idea is to subdivide the problem into predicting fragmentation of the graph and then separately predicting intensities associated with each fragment.  Relative to other recent methods that have adopted this approach, this method uses a prefix tree to make the identification of subgraphs more efficient.  Experimental results show that the proposed method performs better than state-of-the-art methods.\n\nSTRENGTHS:\nThe paper addresses an important problem.\n\nThe paper is extremely well written, with a nice intro to tandem mass spectrometry targeted to a NeurIPS audience.\n\nThe treatment of related work is well organized and clear.\n\nThe main idea is well supported by empirical results.\n\nWEAKNESSES:\nOn lines 37-42, I wasn't super convinced that \"physically-inspired\" (which, incidentally, doesn't need to be hyphenated) belongs in this list of desiderata.  I think the main thing is that the method is fast and accurate.  It seems more like a hypothesis to say that in order to achieve speed and accuracy, a method should be physically inspired.  Perhaps the notion of \"accurate\" should be segregated into various dimensions: e.g., accurate in the sense that it doesn't just predict binned m/z values, or accurate in the sense that it doesn't predict extra peaks that can't be explained by the formula.  I would argue that if someone came along with a method that was very accurate and fast, there is no scenario in which an end user would prefer a slightly slower or slightly less accurate method just because it comes with a story about how it is physically plausible.\n\nFigure 1A is the key idea, and I think it needs to be expanded.  As drawn, it's not clear how, e.g., you decide that the root should have three children (C7, C12 and C14).  Why not C1, C2, etc.?  The answer presumably has to do with the molecular graph, which is the primary input but is not shown here.  I would also like to hear a bit more about this prefix tree step in the text.  E.g., does it matter what order the atom types are introduced?   If so, how is the order chosen?\n\nRelated to the first point above, the claim on lines 100-102 that binning methods are less interpretable was hard for me to understand.  The reduced accuracy due to the lack of shared information across fragments is plausible, but that seems like an empirical claim that requires support.  Maybe just say that you hypothesize that this may lead to reduced accuracy. I also found it strange that you didn't explicitly call out the most obvious source of inaccuracy, which is that the binning means that the peak m/z locations are inherently inaccurate.\n\nFor the SCARF-Weave predictor, please indicate how large the sets can be come, both in principle and in practice.  I wonder whether this is a potential problem for the method.\n\nWhen you present the various competing methods on lines 248-255, I think it would be helpful if you could introduce them in terms of the three groups from Section 2.2.\n\nThe closest competing methods are references [24] and [45].  In your experimental comparisons, why did you use a method based on MS-GRAFF ([24]) rather than just using MS-GRAFF itself?  And why didn't you use method [45] at all?\n\nOn a related note, if FixedVocab falls into the third category of methods, then why aren't its predictions 100% valid in Table 2?\n\nQUESTIONS:\nIs there a concrete sense in which binning methods are less interpretable? \n\nHow large are the sets handled by SCARF-Weave?\n\nWhy are MS-GRAFF and FormulaNet/SubsetNet not included in the empirical comparison?\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe manuscript presents a method called SCARF for the prediction of tandem mass spectra from molecular structures. This allows for building up large in-silico libraries that, together with measured spectra, can be used in spectral library search workflows for metabolite identification. From a general machine learning perspective, the method is an interesting approach for the problem of predicting sets of multisets (in this case: sets of molecular formulas). The author note that autoregression is not well suited for this task, as it is not ordering invariant. Instead, they suggest an approach that is quite similar to the way molecular formulas are decomposed from masses: they step-by-step build a prefix-tree where each node represents a subset of the molecular formula space with certain elements are fixed to specific abundancies. The child nodes are then possible abundancies for the next chemical element, restricting the formula space further until the the leafs of the tree describe only a single molecule formula. The resulting set of molecular formulas is then transformed into a spectrum, by utilizing a second transformer based network for intensity prediction.\n\nSTRENGTHS:\n- it is the first paper that predicts high resolution spectra without involving combinatorial fragmentation\n\n- use of prefix tree for the prediction of molecular formulas (or sets of multisets) is very innovative\n\n- use of random Fourier features for molecular formula embedding is innovative and seem to work much better than the common approaches of encoding formulas as frequency vectors or one-hot vectors/dictionaries\n\n- evaluation on the spectrum prediction part is done very well. In particular, certain methods are retrained (or even re-implemented) and parameter-optimized to ensure fair comparison\n\n- manuscript is clearly written, the method is described in detail with all hyper parameters given in the supplement\n\n- source code is available\n\nWEAKNESSES:\n- there is already a successor method (ICEBERG) by the same authors for the same task. This is not a problem in general, as both methods have very different strategies to solve the problem. I still think it would have strengthen the manuscript if pros and cons of both methods and their synergistic potential is discussed \n\n- the manuscripts list three applications for the predicted spectra: molecule identification, learning about fragmentation, and training machine learning methods on spectra. However, in contrast to ICEBERG the method presented here cannot give any insights about the fragmentation process (as it is not a fragmenter). I also doubt that synthetic data (predicted machine learning) is well suited as training data for other machine learning methods. This is some kind of self-training or self-supervised learning which is a difficult and challenging problem on its own. Thus, the only convincing application for the spectrum prediction task is the molecule identification. Beside spectrum prediction, there are other methods that allow for molecule identification (including methods from the same authors such as MIST). Although the evaluation on metabolite identification contains many methods for spectrum prediction, it does not contain any of these alternative approaches.\n\n- the observation that identification rate is decoupled from database retrieval rate is concerning, as this is the main application for spectrum prediction\n\n- although all methods perform worse on the NPLIB1 data, it is strange that NEIMS(FFN) and SCARF perform so very different on the  retrieval task. It would be interesting to find out why the fingerprint based method performs and the dictionary based method both perform so well here but not in NIST.\n\n- besides its application in metabolomics,  the presented approach might be interesting for other areas with sets of multisets as target variable. Unfortunately, the baseline method (autoregression) is not evaluated against the more sophisticated prefix tree decoding method\n\nQUESTIONS:\n- does the cosine similarity calculation takes the precursor ion into account? For many spectra, the precursor can have high intensity (and, thus, heavily effect cosine similarity) while it does not provide any new information. If so, I would suggest to recalculate cosine similarity and leave out the precursor\n- I do not understand why the spectrum retrieval task is performed on a subset of 49 isomers (although the most similar ones) instead of just all isomers\n\nLIMITATIONS:\nThe main limitation of the method is in my opinion its limited performance on the metabolite identification task. This and other limitations are discussed properly in the manuscript.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a new method for predicting mass spectra from molecules called SCARF, which stands for Subformulae Classification for Autoregressively Reconstructing Fragmentations. Mass spectra are sets of peaks that represent the masses and intensities of fragments of molecules after they are ionized and broken down in a mass spectrometer. Predicting mass spectra from molecules is useful for identifying unknown molecules from experimental data, as well as for understanding the fragmentation process and generating virtual spectra libraries.\n\nSCARF predicts mass spectra in two steps: first, it generates the set of chemical formulae for the fragments, which define the locations of the peaks on the mass-to-charge axis; second, it assigns intensities to these formulae, which define the heights of the peaks. The key innovation of SCARF is that it uses prefix trees to efficiently generate the set of formulae, overcoming the combinatorial challenge of enumerating all possible subformulae of a given molecule. SCARF also ensures that all predicted peaks are physically plausible, meaning that they correspond to valid subformulae of the original molecule.\n\nThe paper evaluates SCARF on two datasets of molecules and their experimentally measured spectra, NIST20 and NPLIB1. The paper shows that SCARF outperforms existing methods based on fragmentation rules or binned prediction in terms of accuracy, physical sensibility, and speed. The paper also demonstrates that SCARF can improve the retrieval of unknown molecules from new spectra by comparing them to predicted spectra from a database of candidate molecules. The paper concludes by discussing the limitations and future directions of SCARF.\n\nSTRENGTHS:\nThe paper makes several contributions. \n\nFirst, the paper introduces a new method for predicting mass spectra from molecules, which is based on a novel combination of subformulae classification and autoregressive reconstruction. The method overcomes the limitations of existing methods based on fragmentation rules or binned prediction, which are either too restrictive or too coarse-grained. It achieves state-of-the-art performance in terms of accuracy, physical sensibility, and speed, as demonstrated on two datasets of molecules and their experimentally measured spectra. The method is based on a simple yet powerful idea of using prefix trees to generate the set of formulae for the fragments efficiently. It has the potential to revolutionize the field of mass spectrometry by enabling more accurate and efficient identification of unknown molecules from experimental data.\n\nSecond, the paper provides a thorough evaluation of the proposed method on two datasets of molecules and their experimentally measured spectra, NIST20 and NPLIB1. It uses a new metric called physical sensibility, which measures how well the predicted spectra match the physical constraints of mass spectrometry. SCARF outperforms existing methods based on fragmentation rules or binned prediction in terms of accuracy, physical sensibility, and speed. The paper provides detailed descriptions of the datasets, metrics, baselines, and results. The evaluation is significant because it demonstrates the effectiveness and robustness of SCARF across different datasets and scenarios.\n\nThird, the paper discusses the limitations and future directions of SCARF. It identifies several open problems and challenges in mass spectrometry that SCARF or its variants can address and provides insightful analyses and suggestions for future research. \n\nOverall, the paper represents a significant advance in mass spectrometry by introducing a new method for predicting mass spectra from molecules.\n\nWEAKNESSES:\nI list some weaknesses below:\n\n1. The paper assumes that the fragmentation process is purely additive, meaning that each fragment is formed by adding one or more atoms to the previous fragment. This assumption may not hold for some molecules that undergo complex fragmentation pathways, such as rearrangement, elimination, or charge transfer. To address this weakness, future work could explore more general models of fragmentation that can capture these pathways, such as machine learning models or expert systems.\n\n2. It assumes that the mass spectra are measured under ideal conditions, meaning there is no interference from other molecules or ions. This assumption may not hold for some real-world scenarios, such as complex mixtures or dirty samples. To address this weakness, future work could investigate how to incorporate prior knowledge or external data sources to improve the accuracy and robustness of mass spectra prediction.\n\n3. The assumption is that the molecules are represented by their molecular formulae, which are discrete and symbolic. This representation may not capture the continuous and structural features of molecules that affect their fragmentation patterns and mass spectra. To address this weakness, future work could explore more expressive and flexible representations of molecules that can capture these features, such as molecular graphs or descriptors.\n\n4. The fragmentation patterns are assumed independent of each other, meaning that each fragment is formed independently of the others. This assumption may not hold for some molecules that undergo correlated fragmentation pathways, such as cleavage of adjacent bonds or the formation of cyclic structures. To address this weakness, future work could investigate how to model these correlations explicitly or implicitly in the prediction process.\n\n5. The paper assumes that the mass spectra are measured with high resolution and accuracy, meaning each peak is resolved and assigned a precise mass-to-charge ratio. This assumption may not hold for some low-quality or noisy spectra that have overlapping or shifted peaks. To address this weakness, future work could develop methods for denoising or deconvolving mass spectra before or after prediction.\n\nOverall, these weaknesses suggest several directions for future research in mass spectrometry and related fields.\n\nQUESTIONS:\n- How does SCARF handle molecules that contain elements that are not in the predefined element set? How does it handle molecules that have unknown or ambiguous formulae?\n- Can SCARF handle spectra that have multiple precursor ions or multiple charge states? How does it handle spectra that have different adduct types or ionization modes?\n- Will SCARF handle molecules that have multiple conformers or stereoisomers? How does it handle molecules with different fragmentation patterns depending on their conformation or configuration?\n- How sensitive is SCARF to the choice of hyperparameters, such as the number of predicted peaks, the prefix tree depth, or the neural network architecture? How did the authors tune these hyperparameters, and what are the trade-offs involved?\n- How generalizable is SCARF to other datasets or domains, such as proteomics, metabolomics, or natural products? How would the authors adapt SCARF to these domains or datasets?\n- How interpretable is SCARF in terms of explaining its predictions or providing chemical insights? How would the authors improve the interpretability or visualization of SCARF?\n\nLIMITATIONS:\nThe authors have discussed some of the limitations of their work in Section 5, such as the data dependency, the quality of product formula annotation, and the assumptions and simplifications made by their model. However, they could also mention some of the other limitations I pointed out in my previous comments, such as the interference from other molecules or ions, the continuous and structural features of molecules, the correlated fragmentation pathways, and the low-quality or noisy spectra. They could also provide some empirical evidence or analysis to support their claims about the limitations and future directions of their work.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a model, SCARF, that predicts Tandem-MS spectra in a two-step process: 1) predicting atomic subsets, 2) predicting peaks given the subsets. The model's performance is evaluated on two datasets (NIST20, Natural Product Library) using on spectral similarity, coverage, validity, and retrieval, and compared with relevant baseline models.\n\nSTRENGTHS:\n+ Clear figures and descriptions of the modeling approach. \n+ Clear explanation of background/previous work. The main difference between the SCARF model and other models which use subset prediction is that the SCARF model uses prefix trees to predict subgraphs, so this model considers all relevant subformulae and does not have a limit on the number of valid substructures considered.\n+ Compares against relevant modeling baselines. Evaluates models on coverage and validity to judge the interpretability of the spectra.\n\nWEAKNESSES:\n- It would be helpful to provide some more context for the contents of the two datasets, NIST20 and NPLIB. For example, what classes of molecules are present in each dataset (Classyfire might be a helpful tool for this purpose: http://classyfire.wishartlab.com/). What are the molecular weights of compounds of molecules in this dataset? \n\n- Retrieval task: If I understand correctly, the retrieval task here is set up such that the 'library' or search space is 50 candidates total, 1 target and 49 other decoys, which are selected from the set of chemical isomers with the highest Tanimoto similarity. Each model is used to predict the spectra for all 50 candidates. \n\nWhy not perform the retrieval task on the entire NIST 20 library? Or if NIST20 is too large, why not use the set of all structures with the same chemical formula or precursor formula, rather than limiting the task to 50 isomers? I believe that by limiting the search library to a  set of 50 molecules that are most chemically similar to the target, the retrieval task might become easier than the real world version. In particularly, typically when one wants to identify a compound, you have few assumptions about what your target molecule is, other than the chemical formula, so you would not be able to filter by chemical similarity to the target. Retrieval tasks in previous works such as Wei et al 2019 were performed on the entire NIST17 dataset.\n\nQUESTIONS:\n- It's great that the SCARF algorithm, and subset prediction generation in general, allows for easy interpretation of peaks. How does the peak annotation produced by SCARF compare to the assignments given by NIST's MS Interpreter tool?\n\n- In comparing predicted spectra with ground truth spectra such as in figures 5B&C and Figure A.1: Do you notice any classes of molecules or other patterns where SCARF is particularly good at making predictions? Or particularly bad at making predictions? What is your guess for why the SCARF model generates some extra peaks for some of the molecules in Figure A.1?\n\n- How does the performance of SCARF on the retrieval task depend on the size of the molecule, and therefore the number of plausible isomers.\n\n- In the retrieval task section, it is mentioned that 'database and model biases can skew retrieval results'. Could elaborate more on the biases that are referred to here?\n\n- It could be helpful to provide what you perceive the use case of the SCARF model in real world applications. Would you expect experimental chemists to predict all candidate structures for their target molecule using SCARF, and then perform the retrieval task to identify the right molecule?\n\nLIMITATIONS:\nI do not identify any negative broader societal impacts associated with this work.\n\nI agree with the author's statement that the model's performance will be data dependent. If you plan to release your model (which I would highly encourage), it would be helpful to give an overview of the types of molecules that you expect to be in or out of distribution for the model.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-20"}