Deep learning prediction of electrospray ionization tandem mass spectra of chemically derived molecules

Domain: earth_science
Published: 2024-09-27
Editorial outcome: major_revision

Abstract
Chemical derivatization is a powerful strategy to enhance sensitivity and selectivity of liquid chromatography-mass spectrometry for non-targeted analysis of chemicals in complex mixtures. However, it remains impossible to obtain large sets of reference spectra for chemically derived molecules (CDMs), representing a major barrier in real-world applications. Herein, we describe a deep learning approach that enables accurate prediction of electrospray ionization tandem mass spectra for CDMs (DeepCDM). DeepCDM is established by transfer learning from a generic spectrum predicting model using a small set of experimentally acquired tandem mass spectra of CDMs, which converts a generic model with low predictability for CDMs into a specialized model with high predictability. We demonstrate DeepCDM by predicting electrospray ionization tandem mass spectra of dansylated molecules. The success in establishing Dns-MS further enables the development of DnsBank, a dansylation-specialized in silico spectral library. DnsBank achieves significant increases of accurate annotation rates of dansylated molecules, facilitating discovery of new hazardous pollutants from an environmental study of leather industrial wastewater. DeepCDM is also highly versatile for other classes of CDMs. Therefore, we envision that DeepCDM will pave a way for high-throughput identification of CDMs in non-targeted analysis to dig unknowns with potential health impacts from emerging anthropogenic chemicals.

Introduction
Chemical derivatization is a common practice in analytical chemistry to enable the analysis of compounds that otherwise cannot be easily separated or detected using chromatographic or spectroscopic technologies1–3. When deployed into liquid chromatography-mass spectrometry (LC-MS) analysis that has been widely used in biomarker discovery4–6, untargeted metabolomics7–9 and environmental research10–12, high-throughput identification of chemically derived molecules (CDMs) presents an unmet challenge in data mining and molecule identification. This is because typical pipelines to interpret MS and tandem MS (MS/MS) data rely on spectrum matching in libraries (Fig. 1a), and only molecules for which a reference sample has been measured and its spectrum has been recorded can be confidently annotated13,14. However, there are very limited number of CDM reference spectra and there is no specialized spectral library for CDMs so far. Moreover, CDMs often generate significantly different MS/MS spectra from the original molecules. For instance, low similarity with weighted cosine similarity (WCS) value of only 0.02±0.01 (median, 95% CI) was found between the reference spectra of 231 underived molecules recorded in the MassBank of North America (MoNA), MassBank and Human Metabolome Database (HMDB) and the experimentally obtained spectra of their dansylated forms (Fig. S1a). Consequently, non-targeted analysis of CDMs often results in low confidence levels (level 4–5)15,16. Therefore, there is an urgent need to build specialized spectral libraries for CDMs but experimentally measuring MS/MS spectra of CDMs at a repository scale remains unrealistic.

Recently, in silico methods were developed, allowing rapid prediction of MS/MS spectra in a scalable manner (Fig. 1b). Current predicting tools are developed based on quantum chemistry simulation or machine learning. The quantum chemistry approaches such as quantum chemistry electron ionization mass spectra (QCEIMS) are cost intensive, thus hamper efficiency and application17–19. Machine learning approaches are more efficient17. However, existing machine learning tools including neural electron-ionization mass spectrometry (NEIMS)20 and competitive fragmentation modeling for metabolite identification (CFM-ID)21–23 are general for underived molecules, offering low predictability for CDMs (Fig. S1b). Herein, we describe a deep learning approach that enables accurate prediction of electrospray ionization tandem mass spectrum (ESI-MS/MS) for CDMs, named as DeepCDM. Rather than developing a new predicting tool from scratch, DeepCDM focuses on re-training existing algorisms using a small set of experimentally obtained CDM spectra via transfer learning. We demonstrate that DeepCDM is accurate and scalable, enabling the establishment of a dansylation-specialized model, Dns-MS, which improves upon the generic tools, such as NEIMS and CFM-ID, in the compound-to-spectrum (C2MS) benchmark of dansylated molecules, and allowing the rapid construction of a specialized in silico spectral library containing 294,647 MS/MS spectra for dansylated molecules (Fig. 1c). Moreover, we also demonstrate that DeepCDM is highly versatile, which can be generalized for predicting MS/MS spectrum of other classes of CDMs by switching the training set of CDMs. We further demonstrate that DeepCDM is fully adaptable with non-targeted analysis, which enables the rapid discovery of new hazardous chemicals in authentic wastewaters from leather industry.

Results
DeepCDM workflow for dansylated molecules
As a proof-of-concept, we aim to develop and validate DeepCDM for predicting MS/MS spectra of dansylated molecules. Dansylation is one of the most widely used chemical derivatization approaches in LC-MS based non-targeted analysis9,12,24, but there are only 315 MS/MS spectra of dansylated molecules available in MYCompoundID25 and 6 in MassBank26. To accurately predict MS/MS spectra of dansylated molecules, DeepCDM retrained the multilayer perceptron (MLP) of NEIMS into a dansylation-specialized MS/MS predicting model (Dns-MS). The goal of Dns-MS is to predict collision-energy-merged ESI-MS/MS spectra of dansylated molecules because such spectra are most often acquired in non-targeted analysis27–32. As NEIMS is an electron ionization (EI) spectrum predicting model20, we first trained the MLP of NEIMS using 6790 ESI-positive MS/MS spectra collected from MoNA, so that this pre-trained model (ESI-MLP) was suitable for ESI-MS/MS spectrum prediction (Table S1, S2). Additionally, multiple MoNA spectra of one molecule at different collision energies were merged to generate collision-energy-merged spectra for training. ESI-MLP was further fine-tuned into Dns-MS, where a training set containing 507 experimentally obtained spectra of dansylated molecules synthesized by labeling chemicals with amine, hydroxyl, and carboxyl groups using dansyl chloride (DnsCl) and/or dansylhydrazine (DnsHz) (Fig. 2a) was employed. The original molecular weights of the training set were ranged from 59.0735 to 665.6982 Da, containing 62 classes of molecules such as benzene and substituted derivatives, carboxylic acids and derivatives, and phenols. The molecules had an average pairwise Tanimoto similarity of 0.11 on their Extended-Connectivity Fingerprints (ECFP4) (Fig. S2). For each dansylated molecule, spectra were collected at multiple collision energies ranging from 10 eV to 50 eV and merged automatically by HESI Q-Exactive Plus Orbitrap (Thermo Scientific, USA).

Fine-tuning at the last layer of ESI-MLP showed slightly better performance over fine-tuning at all layers and was thus employed as an optimal fine-tuning strategy for Dns-MS (Fig. S3). Besides Dns-MS, we also developed Dns-MS-β by directly training the MLP of NEIMS using a combined set of 6790 MoNA spectra and 507 experimental spectra of dansylated molecules (Fig. S4). Nevertheless, Dns-MS demonstrated much better predictability over Dns-MS-β (Figs. S4, S5), suggesting that fine-tuning using even a small set of spectra of CDMs is critical to ensure the success of DeepCDM. Moreover, Dns-MS showed equally high spectrum predictability for dansylated molecules synthesized through labeling chemicals with amine, hydroxyl and carboxyl groups (Fig. 2c), suggesting that Dns-MS can be broadly used with no structural bias.

In a 4-fold cross-validation, Dns-MS demonstrated robust performances for 4 randomly split dansylated dataset (Fig. S6a). The Dns-MS predicted spectrum and experimental spectrum of every dansylated molecule in the test set was also matched with the experimental spectrum of a structurally most similar molecule in the training set. The predicted spectra of query molecules had low similarity with the experimental spectra of their structurally similar molecules with a median WCS of 0.27±0.02 (95% CI, n=167). Similar observation was made between the experimental spectra of query molecules in the test set with those of structurally similar molecules in the training set (median WCS=0.22±0.03, 95% CI, n=167, Fig. S6b, c). That confirmed the good predictability of Dns-MS for dansylated molecules that haven’t been exposed to the model.

Then, we compared the performance Dns-MS with two widely used spectrum predicting tools, NEIMS and CFM-ID 4.0 (web server), using the test set of 167 experimental spectra. Figure 2d and Fig. S7 revealed better predictability of Dns-MS with a median WCS value of 0.69 over two other machine learning-based tools with median WCS values of 0.38 for NEIMS and 0.46 for CMF-ID 4.0, respectively. Figure 2e shows the predicted spectra of a representative molecule, dansylated 4-isopropyl-3-methylphenol using Dns-MS, NEIMS, and CMF-ID 4.0, as well as matching with the experimental spectrum of this dansylated standard. NEIMS had low predictability of dansylated 4-isopropyl-3-methylphenol with over-prediction of small fragments (WCS=0.36). CFM-ID 4.0 produced a clean spectrum but with several mismatches (WCS=0.54). By contrast, the predicted spectrum by Dns-MS showed an excellent match with the standard, which returned a WCS value of 0.94. The spectra matching results of all 167 dansylated molecules were summarized in Supplementary Data-01, and 73.65% of predicted spectra by Dns-MS were of high quality with WCS values over 0.6.

To evaluate the capability of DeepCDM to selectively enhance the MS/MS spectrum predictability of CDMs, we harnessed Dns-MS to predict spectrum for underived molecules in the MoNA dataset (n=6790). As shown in Fig. S8, the predictability of Dns-MS was drastically decreased for underived molecules with a median WCS of 0.38, confirming that our DeepCDM approach is specialized for desired CDMs.

Versatility of DeepCDM
To demonstrate that DeepCDM is a generalizable approach for different classes of CDMs, we also established a spectrum predicting model for benzoylation, another widely used chemical derivatization method for amine- and hydroxyl-containing molecules33–35. This model, termed Bz-MS, was first trained using the MoNA dataset containing 6790 merged ESI-MS/MS positive spectra of underived molecules, and then fine-tuned using a training set containing 503 experimental spectra of benzoylated molecules (Fig. S9). Evaluated with a hold-out test set containing 168 benzoylated molecules, Bz-MS achieved a median WCS value of 0.74±0.03 (95% CI, n=168) in predicting spectra for benzoylated molecules, which was significantly improved from the pre-trained model, ESI-MLP (median WCS=0.53±0.02, 95% CI, n=168) (Fig. S10a). It also outperformed NEIMS and CFM-ID 4.0 with over 1.5-fold increase of median WCS in a C2MS benchmark (Figs. S10, S11). Moreover, there was no bias in spectrum prediction for bezoylated amine- and hydroxyl-containing molecules (Fig. S10b). The robustness of Bz-MS was confirmed in a 4-fold cross-validation with 4 randomly split benzoylated datasets (Fig. S12). Bz-MS was also a highly specialized tool for benzoylated molecules with predictability significantly better than underived molecules and dansylated molecules (Fig. S13). Collectively, our success in establishing Bz-MS demonstrates that DeepCDM is a highly versatile approach generalizable from one class of CDMs to another.

DeepCDM-based spectral library for dansylated molecules
Having demonstrated that DeepCDM is a highly robust and versatile approach to enable MS spectrum prediction models for desired class of CDMs, we next aim to establish a database that specialized for a given class of CDM. Here, a dansylation-specialized spectral library (DnsBank) was established, which contained 294,647 ESI-MS/MS spectra predicted by Dns-MS. To construct DnsBank, we first screened 178,629 molecules from a public chemical database, Distributed Structure-Searchable Toxicity (DSSTox)36, with three criteria, including (1) molecules that were composed by 11 elements: H, C, O, N, P, S, Cl, F, Br, I, Si; (2) molecules that were not ionized or containing ionized adducts; (3) molecules that contained amine, hydroxyl or carboxyl groups. The 178,692 molecules then underwent virtual dansylation reactions, where compounds containing amine or hydroxyl groups were derived using DnsCl and those containing carboxyl groups were derived using DnsHz. The virtual reactions were achieved by converting the Simplified Molecular-Input Line-Entry System (SMILES) of each molecule through “SMART reaction” by RDKit (http://www.rdkit.org/) (Fig. 3a). A total of 294,647 dansylated molecules were generated through virtual reactions, the spectra of which were then predicted using Dns-MS and stored in DnsBank. As such, DnsBank contained 50,395 spectra for dansylated amines (17.10%), 110,024 spectra for dansylated hydroxyls (37.34%), 116,348 predicted spectra for dansylated carboxyls (39.49%), and 17,880 spectra for didansylated molecules (6.07%) (Fig. 3b and Table S4).

The spectrum-to-compound (MS2C) capability of DnsBank was evaluated against the 167 experimental spectra of dansylated molecules using WCS as the measure. DnsBank retuned correct annotation rates of 63.47% (106/167) in top 5 output and 91.02% in top 25, respectively (Fig. 3c). The highest correct annotation rate was found for dansylated amines (74.51% in top 5), followed by dansylated hydroxyls (60.81% in top 5) and carboxyls (54.76% in top 5) (Fig. 3d). By contrast, public databases (HMDB, MassBank JP/MassBank EU and MoNA) paired with CFM-ID 4.0 as a compound identification module failed to correctly annotate any dansylated molecules in the test set. Failure was also made when in silico spectral libraries (ChEBI, DSSTox and STOFF-IDENT) generated by CFM-ID 4.0 were used, though 3 dansylated molecules were found in these libraries upon manual checking (Supplementary Data-02).

We also compared the searching result of DnsBank with that obtained using SIRIUS 4, which computes possible molecular fingerprints based on the MS/MS spectrum through a fragmentation tree and then searches the predicted fingerprints in chemical databases, such as PubChem37. Despite SIRIUS 4 drastically widens the coverage of chemicals, it only correctly annotated 7 dansylated molecules in top 5 output and 8 in top 25. Among the 159 incorrect annotations, 7 dansylated molecules were found in PubChem but not annotated by SIRIUS 4 and 13 were unable to be computed by SIRIUS (Supplementary Data-02). Collectively, our results demonstrated the superior MS2C performance of DnsBank over other tools for dansylated molecules, demonstrating the power of our DeepCDM for annotating CDMs.

Wastewater analysis using DeepCDM-assisted non-targeted analysis
DeepCDM allows us to expand the annotation of CDMs beyond the space of known molecules, making it possible to explore novel chemical processes. To demonstrate this, we used DeepCDM to facilitate the discovery of new hazardous chemicals in authentic environmental samples.

Anthropogenic organic chemicals have improved the quality of human life but many also pose inherent dangers to human and environment. Leather industry is an important contributor to pollutants emission due to the booming global market of leather goods38. Although industrial wastewater must be treated before emitting into environment, a significant amount of chemicals persist and can be further transformed during the treatment39. Therefore, high-throughput identification of massive organics in wastewater before and after treatments without prior knowledge is the first and foremost step for discovering chemicals that may eventually expose to human and biota.

We explored chemicals in wastewaters of leather industry before and after treatments by collecting and analyzing three types of samples, including untreated wastewater (UW), wastewater treated by active carbon adsorption (ACD), and that treated by ACD and ozonation (ACD/ozonation) (Fig. 4a). All water samples were first dansylated to enhance the retention and ionization of acids, phenolics, and aromatic amines that were heavily involved in the production of leather goods40,41. LC-MS based non-targeted analysis was then performed, where DnsBank was employed for spectrum matching and molecule annotation. Using WCS≥0.6 as a threshold42 since it provided reasonable number of annotations for further analysis (Fig. S14), we tentatively annotated 614 dansylated molecules in UW, 365 in ACD, and 369 in ACD/ozonation with confidence level 3 (Table 1). The average annotation efficiency was determined to be 19.42% (Table S5). A total of 991 unique dansylated molecules were annotated throughout all samples (Supplementary Data-03). As comparisons, SIRIUS 4 returned 34 tentative annotations (average annotation efficiency: 0.56%) and mzCloud (https://www.mzcloud.org/) paired with Compound DiscovererTM 3.3 (CD3.3, Thermo Fisher Scientific) annotated only 7 dansylated molecules (average annotation efficiency: 0.22%) for the same set of samples (Fig. 4b and Table S4).

The lower number of annotations in treated wastewater samples by either ACD or ACD/ozonation was consistent with the decreasing chemical oxygen demand (COD) (Fig. 4c and Table 1), suggesting the effective removal of chemicals from wastewater. However, there were 87 tentative annotations detected in all 3 samples, indicating the exist of persistent chemicals stable against ACD and ozonation treatments (Table S6). DeepCDM-assisted non-targeted analysis also revealed significant differences in treatment techniques, where ACD was largely overlapped with UW in uniform manifold approximation and projection (UMAP) analysis but ACD/ozonation showed a significant different profile from the other two samples (Fig. 4d). This result indicated that ozonation made a significant change to chemical compositions in wastewater with 142 unique annotations which were likely to be new transformation products (Table S7).

Of all 991 tentative annotations, there were many known pollutants, such as per- and polyfluoroalkyl substances (PFASs) and halogenated compounds (Table S8 and Fig. S15). We also found 865 chemicals with no toxicity record. A further investigation of this group of chemicals may lead to the discovery of new hazardous pollutants. We selected 4 annotations to confirm their identities using dansylated chemical standards and evaluate their cytotoxicity using HepG2 cells through cell counting kit-8 (CCK-8) (Table S9). Two compounds, 3-fluoro-5-hydroxybenzonitrile and ent-paroxol, were found to be of high cellular toxicity with the 50% inhibitory concentration (IC50) determined to be 545.9 μM and 494.4 μM, respectively (Figs. 4e, f and S16), thus could be new hazardous pollutants.

Methods
DeepCDM dataset
First, spectra were downloaded from public MassBank of North America (MoNA, ucdavis.edu) and filtered with criteria: (1) positive mode, (2) ESI ionization, (3) MoNA rank >4. Then the downloaded spectra was processed to merge all fragments produced at different collision energy for one molecule37,46. That produced a MoNA dataset containing 6790 merged electrospray ionization tandem mass spectra (ESI-MS/MS) of 6790 underived molecules, which was used to train the multilayer perceptron (MLP) of NEIMS20. In this step, MoNA dataset was chosen because it is an open database accessible to researchers across diverse research fields. Second, 585 chemical standards were labeled by dansyl chloride (DnsCl) and/or dansylhydrazine (DnsHz) to produce 674 dansylated molecules, whose experimental merged spectra were acquired using HESI Q-Exactive Plus Orbitrap (Thermo Scientific, USA) at collision energies ranged from 10 to 50 eV. 507 experimental spectra of dansylated molecules were used for fine-tuning the trained MLP as the training set with 10% for validation and 167 spectra were holdout from the training set for model evaluation (test set). Dataset was randomly split without considering structural similarity of dansylated molecules. The MLP was fine-tuned by freezing all but the last layer in Dns-MS.

To modify DeepCDM for benzoylated molecules, the trained MLP by MoNA dataset was fine-tuned using 503 experimental spectra of benzoylated molecules with 10% for validation. A hold-out test set containing 168 experimental spectra of benzoylated molecules was used for model evaluation. The benzoylated dataset was collected on HESI Q-Exactive Plus Orbitrap at collision energies ranged from 10 to 50 eV, using 671 benzoylated molecules labeled by benzoyl chloride (BzCl). Dataset was randomly split without considering structural similarity of benzoylated molecules. The MLP was also fine-tuned by freezing all but the last layer in Bz-MS.

Modelevaluation
3 metrics were tested for model evaluation: weighted cosine similarity (WCS), cosine similarity and spectra entropy. WCS was calculated using Eq. (1)47:

WeightedCosineSimilarity (I q ,I l ) = \frac{\sum_{m=1}^{M_{max}} (I_{q,m}^{0.5} \cdot I_{l,m}^{0.5})^2}{\sum_{m=1}^{M_{max}} I_{q,m}^{0.5} \cdot \sum_{m=1}^{M_{max}} I_{l,m}^{0.5}}

(1)

Cosine similarity was calculated from Eq. (2)47:

CosineSimilarity (I q ,I l ) = \frac{\sum_{m=1}^{M_{max}} (I_{q,m}^{0.5} \cdot I_{l,m}^{0.5})^2}{\sum_{m=1}^{M_{max}} I_{q,m}^{0.5} \cdot \sum_{m=1}^{M_{max}} I_{l,m}^{0.5}}

(2)

Where m k was the mass-to-charge ratio (m/z) of fragment; I qk and I lk was the intensity of fragment (m/z=k) in the query spectrum and in the reference spectrum; M l and M q were the largest non-zero indices of I l and I q; M max was the largest index of M l and M q.

Spectra entropy S was calculated from all ion intensities I p by Eq. (3)48:

S = - \sum I p ln I p (3)

Consistent results were found among the 3 metrics in the comparison of 3 models with slightly better WCSs, using the hold-out test set of 167 dansylated molecules. Therefore, WCS was selected in this study to evaluate the predictability of models.

Virtual reaction and library building
A structural dataset was downloaded from Distributed Structure-Searchable Toxicity (DSSTox, https://epa.figshare.com/articles/dataset/Chemistry_Dashboard_Data_DSSTox_SDF/5491516, accessed on March 31, 2022). The DSSTox dataset were screen for chemicals with amine, hydroxyl and carboxyl groups. Then, the structures of dansylated molecules were computed by RDKit (http://www.rdkit.org/) using a derivatization Python script “SMART reaction” and output in Simplified Molecular Input Line Entry System (SMILES). The “SMART reaction” simulated dansylation to reacting groups using defined templates: DnsCl for amine and hydroxyl, DnsHz for carboxyl. The dansylated SMILES structures were exported in SDF files, then input to Dns-MS to predict collision-energy-merged ESI-MS/MS spectra to 1 Da resolution. The predicted spectra were accumulated for DnsBank (Table S4).

Benchmarks
C2MS benchmarks between Dns-MS, Bz-MS, NEIMS and CFM-ID 4.0 (web server) were performed on hold-out test sets. NEIMS was acquired from ref. 20. and CFM-ID 4.0 (web server, http://cfmid.wishartlab.com/predict) was used for spectrum prediction.

MS2C benchmark was performed between DnsBank and other public spectral libraries accessed by CFM-ID 4.0 compound identification module (http://cfmid.wishartlab.com/identify) and PubChem accessed by SIRIUS 4 on the hold-out test set containing 167 dansylated spectra. In silico spectra libraries of ChEBI, DSSTox and STOFF-IDENT predicted by CFM-ID 4.0 and public spectral libraries including HMDB, MassBank and MoNA were used by CFM-ID 4.0 compound identification module for annotating dansylated molecules through spectrum matching. ESI was selected as the spectrum type, ion mode was positive and adduct type was set to [M+H]+ in CFM-ID 4.0, with 10 ppm mass tolerance and DotProduct as the scoring function. For molecule annotation by SIRIUS 4, parameters were set to be instrument: orbitrap, database: PubChem, possible ionization: [M+H]+. CSI:FingerID was used for fingerprint prediction and structure database search.

Dansylation of chemical standards
For DnsCl reaction, amine- or hydroxyl-containing chemicals were dissolved in 50% acetonitrile to a concentration of 10 mM individually. 30 μL standard solution was mixed with 15 μL acetonitrile, 15 μL sodium carbonate/sodium bicarbonate buffer (0.5 M, pH 9.4) and 30 μL 20 mg·mL−1 DnsCl. The reaction solution was shaken at 60 °C for 60 min before quenching by 10 μL 250 mM sodium hydroxide. The excess sodium hydroxide was neutralized by 50 μL 425 mM formic acid.

For DnsHz reaction, carboxyl chemicals were dissolved in acetonitrile to 10 mM individually. 25 μL standard solution was mixed with 25 μL 50 mM 1-ethyl-3(3-(dimethylamino)propyl)carbodiimide, 25 μL 10 mM 1-hydroxy-7-azabenzotriazole in 2-(N-morpholino)ethanesulfonic acid buffer and 25 μL 10 mg mL−1 DnsHz. The reaction solution was shaken at 25 °C for 90 min before quenching at −80 °C for 10 min.

Benzoylation of chemical standards
For benzoyl chloride (BzCl) reaction, amine- or hydroxyl-containing chemicals were dissolved in 50% acetonitrile to a concentration of 10 mM individually. 200 μL standard solution was mixed with 100 μL sodium carbonate buffer (0.1 M, pH=9.5), 100 μL 2% BzCl and 1 μL formic acid at room temperature for 5 min.

MS analysis
MS and tandem MS (MS/MS) spectra of dansylated molecules were acquired by HESI Q-Exactive Plus Orbitrap (Thermo Scientific, USA) in positive mode. Precursor ions were selected for MS/MS scan with ±0.25 Da mass tolerance. 10 MS/MS scans were accumulated to generate a MS/MS spectrum with collision energies ranged from 10 to 50 eV.

Wastewater treatments
The leather industrial wastewater was first treated with activated carbon adsorption (ACD) and then ozonation. 100 mL leather industrial wastewater was diluted 10 times with high-performance liquid chromatography (HPLC) graded water and then mixed with 20 g activated carbon granules followed by incubating at 300 r min−1 for 24 h at room temperature. After filtration, 500 mL ACD treated wastewater was reacted with 500 mL 64.29±1.36 mg L−1 ozone solution at room temperature for 24 h. The chemical oxygen demand (COD) of wastewater were measured before and after treatments.

Sample handling
The wastewater samples were extracted with Supelclean™ ENVI™-18 solid phase extraction cartridge (500 mg, 3 mL, Sigma–Aldrich, USA). 40 μL eluent was divided into two aliquots: 20 μL was reacted with DnsCl and 20 μL with DnsHz. After dansylation, the samples were desalted using HPLC (Agilent 1260 Infinity II, USA) with C18 column (Supelco, 2.1 mm×50 mm, 5 μm) at 25 °C, flow rate: 1 mL/min. The mobile phase A was 0.1% (v/v) formic acid in water and mobile phase B was 0.1% (v/v) formic acid in 95% acetonitrile. Liquid chromatography gradient was: t=0 min, 5% B; t=3 min, 5% B; t=3.01 min, 95% B; t=5 min, 95% B; t=5.01 min, 5% B; t=8 min, 5% B.

Non-targeted LC-MS analysis
Non-targeted analysis was performed using Dionex Ultimate 3000 UHPLC coupled to HESI Q-Exactive Plus Orbitrap mass spectrometer (Thermo Scientific, USA). A C18 column (Thermo Scientific, 2.1 mm×100 mm, 1.8 μm) was used at 25 °C, with a flow rate of 300 μL min−1. Mobile phase A was 0.1% (v/v) formic acid in water, mobile phase B was 0.1% (v/v) formic acid in acetonitrile. Liquid chromatography gradient was: t=0 min, 5% B; t=3 min, 5% B; t=4 min, 15% B; t=14 min, 65% B; t=16 min, 95% B; t=18 min, 95% B; t=18.01 min, 5% B; t=20 min, 5% B. MS/MS spectra were accumulated by 10 MS/MS scans with collision energies ranged from 10 to 50 eV.

Data processing
Molecule classification based on structural features was performed using ClassyFire49. Peak lists of non-targeted data were extracted using MZmine 350. Molecule annotation at confidence level 3 was achieved by matching exact mass and MS/MS spectra in DnsBank. The annotation confidence level was complied with small molecule identifications defined by Schymanski, et al., in 201414. With the spectra matching threshold WCS≥0.642, annotations were exported in csv file. Annotation efficiency was calculated to evaluate the percentage of query spectra confidently assigned to molecules by spectra matching or database searching22.

AnnotationEfficiency = \frac{number of molecule annotations}{number of query spectra} (4)

Statistics
The unpaired t-test was performed in Prism (GraphPad Software, USA) to compare spectral WCS between two groups of data. For comparisons between three or more groups, ordinary one-way ANOVA was performed. The uniform manifold approximation and projection (UMAP) analysis was performed to analyze the distances between organic profiles of 3 wastewater samples. For all analyses, a two-tailed p value <0.05 was considered to be statistically significant.

In vitro cytotoxicity study
The cell counting kit-8 (CCK-8) assay was used to monitor the cell viability of human hepatocellular cell line (HepG2) in 96-wells incubating with various concentrations of chemicals for 72 h. The endpoint absorbance of each well was measured 6 times at 450 nm by Epoch 2 microplate spectrophotometer (Agilent BioTek, USA).

Implementation details
All DeepCDM models were constructed and trained in Tensorflow51. The hyperparameters were optimized on the hidden units and layers of MLP. Training hyperparameters were listed in Table S2. The optimized MLP architecture using ReLU activation had 3 layers of 500 nodes. Extended-Connectivity Fingerprints (ECFP4) with a length of 4096 and radius of 2 were set for the input of molecules. Further details on hyperparameter configurations were included in the code repository and can be obtained through Github (http://github.com/ADNLab-SCU/DeepCDMs).

Peer review rounds
[{"round": 1, "reviewer_comments": {"reviewer_1": "Dear Authors,\nI have carefully read the manuscript entitled \"Deep Learning Enables High-Quality MS/MS Spectrum Prediction for Chemically Derived Molecules\" submitted to Nature Communications.\nThe aim of this manuscript was to overcome the limitations related to the lack of large reference spectra sets for chemically derivatized molecules (CDMs), which hinder the deployment of chemical derivatization-based NTA.\nIn my opinion this investigation is original and its nature and scientifically relevant in the analytical chemistry field and the scope of this journal. The paper is well written and quite easy to follow. The analysis, in general, seems well done and a very good review of the literature has been presented.\nNevertheless, I think that the paper is not suitable for publication in its present format. Some issues need to be addressed before the manuscript can be considered for publication. Some aspects were noticed during this review and are pointed out below.\n- ABSTRACT\n1) Avoid using acronyms, such as \"ESI-MS/MS,\" \"MS/MS spectra,\" and \"MoNA dataset\" in the\nAbstract.\n2) The authors could highlight the significance and potential impact of these findings, particularly in\nthe context of environmental and public health.\n- INTRODUCTION\n1) Authors should discuss about validation of the method, including comparisons with existing tools\nand benchmarks for accuracy and reliability.\n2) Again, avoid using acronyms without providing explanations for them.\n- METHODS\n1) Authors should include information on validation methods and controls used to ensure the\naccuracy and reliability of the experimental results.\n2) A more comprehensive explanation of the statistical methods employed to analyze the data and\nestablish significance is needed.\n3) The authors did not provide details about the implementation and software used. Could you\nplease provide a detailed description?\n- RESULTS\n1) The dataset used for training and testing, although described as large, is based on a specific subset\nof molecules (dansylated molecules), which might limit the generalizability of the results to other\ntypes of molecules.\n2) While WCS is used as a metric for evaluating the predictability of the models, additional metrics or\nmethods could provide a more comprehensive evaluation of the model's performance.\n- DISCUSSION\n1) While DeepCDM is compared to existing tools in terms of efficiency, a more in-depth comparative\nanalysis, including strengths and weaknesses of each tool, would provide a clearer understanding of\nDeepCDM's advantages.", "reviewer_2": "This study uses deep learning to generate an in silico reference library of spectra from tandem mass\nspectrometry (MS/MS) for chemically derivatized molecules (CDM) typically used in non-targerted\nanalysis (NTA). The reported effectiveness of the resultant reference library DnsBank is incredibly\nstrong and the idea using transfer learning for mass spectra prediction is still pretty novel to my\nknowledge. Transfer learning is an excellent choice in this situation where very little training data is\navailable. I've seen the effectiveness of transfer learning from an initially weak spectral prediction\ntool to a more specialized set in my own work. This should be a strong approach, especially if the\ndansylation process has a specific impact on how the molecules fragment. The trials run for this\nstudy between Dns-MS and Dns-MS-beta emphasize the effectiveness of transfer learning in a\nsituation where general prediction (like that performed by NEIMS) is not accurate enough for use in\nthe field. The experimental study seems very promising as well and adds to the excitement of the\npaper, but I don't have the scope of knowledge to gauge how important those results are.\nI think the argument for this tool would be made stronger by investigating the cosine similarity of\nthe reference spectra and spectra of the dansylated molecules for the 507 molecules in the\nexperimental set. If there are significant differences between a molecule's reference spectra\ntraditionally found in MoNA and the dansylated form, this creation of DnsBank should be\nimmediately helpful in the field of NTA. References 15 and 16 (Liu 2019, Zhao 2024) attribute\nchallenges in NTA to many factors, but if analysts are looking for spectra of dansylated molecules in\nlibraries of original molecules DnsBank seems like a way to remove one obstacle. On the other hand,\nif the reference spectra and the spectra of the dansylated forms have high cosine similarities scores,\nthis outcome might be attributed to some other effect.\nFrom a machine learning stand point, I wonder why this team has chosen to start from a relatively\nold model when there have been several advances in spectral prediction with deep learning**.\nI understand NEIMS is easy because of code availability, but there are models that are more crafted\nfor the task of ESI-MS/MS spectral prediction. Similarly, most of the current efforts train on NIST\n(even NEIMS trained on NIST) due to its size and accessibility. The training set from MoNA is very\nsmall and not well curated. Young, Rost, & Wang 2024 (https://doi.org/10.1038/s42256-024-00816-\n8) give a good overview of why NIST is a preferred training set.\nWhy is similarity determined with a weighted cosine similarity? The reference for this metric is 30\nyears old (41, Stein & Scott 1994). Young, Rost & Wang 2024 give a strong argument for regular\ncosine similarity and Li et al 2021 (https://doi.org/10.1038/s41592-021-01331-z) introduce another\nstrong metric in spectral entropy. Also, while 0.6 (lines 127-128) is a reasonable threshold for how\nwell deep learning generally predicts MS/MS spectra, it is still low for analysts trying to match their\nexperimental spectra to library entries.\nFinally, the use of the phrase 'fine tuning' in machine learning is imprecise and should be defined.\nHow was it tuned? Were there further changes beyond the number of layers and nodes? How were\nthese 'tuned' values selected?\nMore generally, a key strength of ESI-MS/MS is the ability to examine spectra at different collision\nenergies. In that vein, collision-energy-merged spectra (line 82) are not as desirable as the reference\nspectra found in existing libraries (e.g., MoNA or NIST). However, if this is the standard output for\nNTA, creating a reference library that specifically meets the need for collision-energy-merged\nspectra is very helpful. Otherwise, creating these merged spectra does simplify the problem from a\nmachine learning stand point. This also could be skewing the reported performance if they are\nsearching for matches to collision-energy-merged spectra in reference libraries where spectra are\nreported at different collision energies (e.g., using CFM-ID 4.0's compound identification model\nprobably compares against non-merged spectra). More details about the benchmarking would be\nvery helpful.\nThe reported benchmarks and poor performance of CFM-ID and SIRIUS4 seem very surprising, again\nI think more clarity in the benchmarking process would be helpful here. From the Fiehn lab's CASMI\n2022 test (where I think they do use merged spectra), the group behind SIRIUS4 was highly\nsuccessful in compound identification and the CFM-ID team also performed reasonably well. Again,\nthis could be attributed to the differences between a molecule's spectrum before and after it's been\ndansylated. A review of the overlap between the presence of the test molecules in the different\nreference libraries (DSSTox, HMDB, MassBank JP/EU/NA, DnsBank, etc) would also help, to show\nthat the correct results are available and the CFM-ID and SIRIUS4 tools are still not finding them.\nDiscussion of the benchmarking in the caption of Figure 3 is helpful, more detail should be added to\nthe report in the body of the paper. A summarizing table might be appropriate.\nOverall, this seems like a promising approach. I like the potential for transfer learning and if there is\na strong difference between the spectra of regular molecules and CDM, I imagine this will be\nextremely helpful. Doubly so if NTA requires a library of collision-energy-merged spectra, as that\ndoes not seem to be the standard of most reference libraries. I appreciate the opportunity to review\nyour exciting discoveries.\nMinor notes:\n- The citation of Grimme's QCEIMS on line 51 should be updated to refer to Grimme's QCxMS which\nactually does quantum chemical simulations of ESI-MS/MS, not just EI-MS.\n- CFM-ID is not a deep learning tool like NEIMS, they are far apart on the spectrum of machine\nlearning tools. CFM-ID replies on probabilistic models applied to combinatorial generation of\npotential fragments. This approach is why CFM-ID doesn't over-predict small peaks like NEIMS does,\nit can only predict peaks at the locations of true potential fragments.\n- NEIMS's over-prediction of small peaks should not be attributed to its original application to EI-MS,\nthere is nothing specific to EI-MS built into NEIMS. In fact, NEIMS has been retrained many times\nover with ESI-MS/MS data** and still has these features, it is more likely the difficulty of using deep\nlearning for spectral prediction.\n- Why aren't the results of Dns-MS compared in Figure S1?\n**Some references:\nhttps://arxiv.org/abs/2010.04661, https://arxiv.org/abs/2301.11419,\nhttps://www.biorxiv.org/content/10.1101/2023.01.16.524269v1, https://doi.org/10.1038/s42256-\n024-00816-8", "reviewer_3": "The authors present a deep learning approach for predicting MS/MS spectra for chemically\nderivatized molecules. The authors adapted a model, which was originally developed for predicting\nESI MS/MS spectra of chemical molecules, by fine-tuning it on a dataset of chemically derivatized\nmolecules and specifically dansylated molecules. The authors utilized the model for developing a\ndansylation-specific spectra library which was used to annotate molecules in an environmental\nstudy, and they were able to verify experimentally 4 annotations.\nThe experimental validation is interesting, however, the novelty of the method is limited as an\nexisting model has been further tuned in a different dataset/domain.\nAdditionally, the machine learning model is not adequately discussed in the manuscript. Although,\nthe architecture of an existing model has been reused, there are important issues that should be\ndiscussed in the paper. More importantly, an analysis that would demonstrate generalization\ncapabilities of the model and an analysis that would show that the model has not been over-fitted\nshould be included. In particular:\n1) A short paragraph explaining what the input and output of the model should be included as well\nas the representations of molecules and spectra.\n2) The hyper-parameters of the model and the training parameters should be added in the\nsupplementary material.\n3) An analysis of the dataset that was used to fine-tune the model is needed. How diverse is the\ndataset? What is the average pairwise similarity of the molecules in the dataset? What is the\ndistribution of the molecular weight (or other properties that may help the reader understand the\ndataset diversity)?\n4) How was the dataset split into train-test? Is it a random split? Was structural similarity taken into\naccount?\n5) The authors should discuss in the paper how they prevented and how they evaluated model over-\nfitting.\n6) I suggest the following experiment as a very simple baseline which can help us understand\nwhether over-fitting may have occurred: For every query molecule in the test set, return the\nspectrum of the closest molecule (structurally) in the training set (a simple retrieval method). What\nis the performance that we can obtain from this method?\n7) Especially for the molecule depicted in Figure 2e: What is the closest (structurally) molecule in the\ntraining set? What is the WCS between the spectrum of the closest molecule and the one depicted in\nthe figure?\n8) What is the performance of the fine-tuned model on the dataset that was used for pre-training\nthe model? Is there a significant drop in the performance after fine-tuning the model?\nFinally, the paper is overall well-written (with a small number of typos and grammatical errors).\nHowever, there is very extensive use of acronyms making it difficult to read the manuscript. I highly\nrecommend reducing the use of acronyms across the manuscript.\nOther minor issues:\n1) [lines 304-305] \"In fine-tuning, the hidden layers of the trained MLP were modified from 7 layers of 2000 nodes to 3 layers of 500 nodes.\" I would expect that the size of the model was modified in\npre-training, not in fine-tuning. Please clarify.\n2) MoNA and NEIMS are mentioned in the abstract but they have not been defined/explained.\n3) [lines 44 & 243] Annotation confidence is mentioned throughout the manuscript but not\nexplained.\n4) [lines 68-69] \"Current predicting tools are not good at predicting spectra for CDMs ...\" I would\nsuggest rephrasing this and stating that they are not designed/developed specifically for CDMs.\n5) [line 80] \"DeepCDM retrained the multilayer perceptron (MLP) of NEIMS\" The architecture of\nNEIMS has not been discussed earlier in the manuscript.\n6) [line 298] I think the NEIMS reference is wrong\n7) Missing references for SIRIUS 4 and mzCloud and Compounds Discoverer.\n8) Annotation efficiency has not been defined."}, "author_response": "We thank all reviewers for the insightful comments and suggestions. We have\nthoroughly revised our manuscript accordingly. A point-by-point response is provided\nas following.\nReviewer #1:\nDear Authors,\nI have carefully read the manuscript entitled \"Deep Learning Enables High-Quality\nMS/MS Spectrum Prediction for Chemically Derived Molecules\" submitted to Nature\nCommunications.\nThe aim of this manuscript was to overcome the limitations related to the lack of large\nreference spectra sets for chemically derivatized molecules (CDMs), which hinder the\ndeployment of chemical derivatization-based NTA.\nIn my opinion this investigation is original and its nature and scientifically relevant in\nthe analytical chemistry field and the scope of this journal. The paper is well written\nand quite easy to follow. The analysis, in general, seems well done and a very good\nreview of the literature has been presented. Nevertheless, I think that the paper is not\nsuitable for publication in its present format. Some issues need to be addressed before\nthe manuscript can be considered for publication. Some aspects were noticed during\nthis review and are pointed out below.\n- ABSTRACT\nComment-1: Avoid using acronyms, such as \"ESI-MS/MS,\" \"MS/MS spectra,\" and\n\"MoNA dataset\" in the Abstract.\nResponse-1: Acronyms such as “ESI-MS/MS”, “MS/MS spectra” and “MoNA” were\nreplaced by their full names in the revised abstract.\nComment-2: The authors could highlight the significance and potential impact of these\nfindings, particularly in the context of environmental and public health.\nResponse-2: Thank you for your kind suggestion. Short discussion about potential\nimpacts of DeepCDM was included in the revised abstract.\nRevision-2: Short discussion about potential impacts of DeepCDM was added to the\nabstract.\n“Therefore, we envision that DeepCDM will pave a way for high-throughput\nidentification of CDMs in non-targeted analysis to dig unknowns with potential health\nimpacts from emerging anthropogenic chemicals.”\n- INTRODUCTION\nComment-3: Authors should discuss about validation of the method, including\ncomparisons with existing tools and benchmarks for accuracy and reliability.\nResponse-3: A brief summary of benchmarks between DeepCDM and generic\nspectrum prediction tools was added to the introduction of the revised manuscript. More\ndetails of method validation and comparisons were included in Results and Discussions.\nRevision-3: Revision was made in the introduction of the revised manuscript:\n“We demonstrate that DeepCDM is accurate and scalable, enabling the establishment\nof a dansylation-specific model, Dns-MS, which outperforms NEIMS and CFM-ID in\nthe compound-to-spectrum (C2MS) benchmark of dansylated molecules, and allowing\nthe rapid construction of a specialized in silico spectral library containing 294,647\nMS/MS spectrum for dansylated molecules (Figure 1c). Moreover, we also demonstrate\nthat DeepCDM is highly versatile, which can be generalized for predicting MS/MS\nspectra of other classes of CDMs by switching the training set of CDMs.”\nComment-4: Again, avoid using acronyms without providing explanations for them.\nResponse-4: Corrected as suggested.\n- METHODS\nComment-5: Authors should include information on validation methods and controls\nused to ensure the accuracy and reliability of the experimental results.\nResponse-5: Thank you for this suggestion. Detailed information on method validation,\nincluding datasets used to ensure accuracy and reliability, was added to the Methods of\nthe revised manuscript.\nRevision-5: Revision was made to Methods part in the revised manuscript:\n“507 experimental spectra of dansylated molecules were used for fine-tuning the trained\nMLP as the training set with 10% for validation and 167 spectra were holdout from the\ntraining set for model evaluation (test set). Dataset was randomly split without\nconsidering structural similarity of dansylated molecules. The MLP was fine-tuned by\nfreezing all but the last layer in Dns-MS.”\n“To modify DeepCDM for benzoylated molecules, the trained MLP by MoNA dataset\nwas fine-tuned using 503 experimental spectra of benzoylated molecules with 10% for\nvalidation. A hold-out test set containing 168 experimental spectra of benzoylated\nmolecules was used for model evaluation. The benzoylated dataset was collected on\nHESI Q-Exactive Plus Orbitrap at collision energies ranged from 10-50 eV, using 671\nbenzoylated molecules labeled by benzoyl chloride (BzCl). Dataset was randomly split\nwithout considering structural similarity of benzoylated molecules. The MLP was also\nfine-tuned by freezing all but the last layer in Bz-MS.”\nComment-6: A more comprehensive explanation of the statistical methods employed\nto analyze the data and establish significance is needed.\nResponse-6: We included more details of statistical analyses in the Methods part of the\nrevised manuscript. Generally, unpaired t-test was performed to compare WCS values\nbetween two prediction methods. For comparisons between three or more groups,\nordinary one-way ANOVA was performed. The uniform manifold approximation and\nprojection (UMAP) analysis was performed to visualize the organic profiles of 3\nwastewater samples. For all analyses, P value < 0.05 was considered to be statistically\nsignificant.\nRevision-6: Revision was made to Methods part in the revised manuscript:\n“Statistics. The unpaired t-test was performed in Prism to compare spectral WCS\nbetween two groups of data. For comparisons between three or more groups, ordinary\none-way ANOVA was performed. The uniform manifold approximation and projection\n(UMAP) analysis was performed to analyze the distances between organic profiles of\n3 wastewater samples. For all analyses, a two-tailed p value < 0.05 was considered to\nbe statistically significant.”\nComment-7: The authors did not provide details about the implementation and\nsoftware used. Could you please provide a detailed description?\nResponse-7: We added these details in the revised manuscript according to your\nsuggestion.\nRevision-7: Revision was made to Methods part in the revised manuscript:\n“Implementation Details. All DeepCDM models were constructed and trained in\nTensorflow51. The hyperparameters were optimized on the hidden units and layers of\nMLP. The optimized MLP architecture using ReLU activation had 3 layers of 500 nodes.\nExtended-Connectivity Fingerprints (ECFPs) with a length of 4096 and radius of 2 were\nset for the input of molecules. Further details on hyperparameter configurations were\nincluded in the code repository and can be obtained through Github\n(http://github.com/ADNLab-SCU/DeepCDMs).”\n- RESULTS\nComment-8: The dataset used for training and testing, although described as large, is\nbased on a specific subset of molecules (dansylated molecules), which might limit the\ngeneralizability of the results to other types of molecules.\nResponse-8: Thank you for this suggestion. To demonstrate the versatility of\nDeepCDM, we established DeepCDM for benzoylated molecules, another important\nclass of CDMs, during the revision. We termed this new prediction model as Bz-MS.\nLike Dns-MS, Bz-MS demonstrated excellent performance in MS/MS spectra\nprediction of corresponding CDMs, confirming the versatility and robustness of our\nDeepCDM approach. Details of Bz-MS modeling and evaluation were added to the\nrevised manuscript.\nRevision-8: Details of Bz-MS modeling and evaluation were added to the Results part\nof the revised manuscript. Figure S10-S13 were included in the Supporting Information.\nRevision in manuscript:\n“Versatility of DeepCDM. To demonstrate that DeepCDM is a generalizable approach\nfor different classes of CDMs, we also established a spectrum predicting model for\nbenzoylation, another widely used chemical derivatization method for amine- and\nhydroxyl-containing molecules36–38. This model, termed Bz-MS, was first trained using\nthe MoNA dataset containing 6,790 merged ESI-MS/MS positive spectra of underived\nmolecules, and then fine-tuned using a training set containing 503 experimental spectra\nof benzoylated molecules (Figure S9). Evaluated with a hold-out test set containing 168\nbenzoylated molecules, Bz-MS achieved a median WCS value of 0.74 ± 0.03 (95% CI,\nn = 168) in predicting spectra for benzoylated molecules, which was significantly\nimproved from the pre-trained model, ESI-MLP (median WCS = 0.53 ± 0.02, 95% CI,\nn = 168) (Figure S10a). It also outperformed NEIMS and CFM-ID 4.0 with over 1.5-\nfold increase of median WCS in a C2MS benchmark (Figure S10 and S11). Moreover,\nthere was no bias in spectrum prediction for bezoylated amine- and hydroxyl-containing\nmolecules (Figure S10b). The robustness of Bz-MS was confirmed in a 4-fold cross-\nvalidation with 4 randomly split benzoylated datasets (Figure S12). Bz-MS was also a\nhighly specific tool for benzoylated molecules with predictability significantly better\nthan underived molecules and dansylated molecules (Figure S13). Collectively, our\nsuccess in establishing Bz-MS demonstrates that DeepCDM is a highly versatile\napproach generalizable from one class of CDMs to another. ”\nRevision in SI:\nFigure S10. (a) WCS comparisons between predicted spectra and experimental spectra\nof 168 benzoylated molecules in a hold-out test set computed by Bz-MS and ESI-MLP.\n(b) WCSs of Bz-MS predicted spectra for benzoylated molecules with amine and\nhydroxyl in the test set. Amines: 0.74 ± 0.03, hydroxyls: 0.72 ± 0.06 (median, 95% CI).\nNo significant difference was observed between the 2 groups (p = 0.1263) (c) WCSs of\npredicted spectra generated by NEIMS, CFM-ID 4.0 and Bz-MS for benzoylated\nmolecules in the test set matching with the 168 experimental spectra. NEIMS: 0.40 ±\n0.02, CFM-ID 4.0: 0.49 ± 0.02 and Bz-MS: 0.74 ± 0.03 (median, 95% CI). Bz-MS had\nsignificantly enhanced WCSs in comparison with CFM-ID 4.0 (p = 1.26e-25) and\nNEIMS (p = 4.25e-54). (d) The predicted spectra of dansylated 2-amino-6-\nmethoxypyridine generated by NEIMS, CFM-ID 4.0 and Bz-MS, were matched with\nits experimental spectrum. ns: p > 0.05, ****: p ≤ 0.0001.\nFigure S11. (a) WCS comparisons of NEIMS and Bz-MS predicted spectra for 168\nbenzoylated molecules in the test set. (b) WCS comparisons of CFM-ID 4.0 and Bz-\nMS predicted spectra for 168 benzoylated molecules in the test set.\nFigure S12. 4-fold cross validation of Bz-MS. For each data split, no significant\ndifference was observed (p = 0.3196). Random split 1: 0.74 ± 0.03, random split 2: 0.77\n± 0.03, random split 3: 0.73 ± 0.03 and random split 4: 0.76 ± 0.03 (median, 95% CI).\nns: p > 0.05.\nFigure S13. (a) WCS between the reference spectra of 6,790 underived molecules in\nthe pre-training MoNA dataset and their predicted spectra generated by Bz-MS, NEIMS\nand CFM-ID 4.0. Bz-MS achieved WCS of 0.53 ± 0.01 (median, 95% CI), NEIMS\nachieved WCS of 0.28 ± 0.01 (median, 95% CI) and CFM-ID 4.0 had WCS of 0.49 ±\n0.01 (median, 95% CI). (b) WCSs of predicted spectra generated by Bz-MS and Dns-\nMS for benzoylated molecules in the test set matching with the 168 experimental\nspectra. Bz-MS: 0.74 ± 0.03 (median, 95% CI), Dns-MS: 0.34 ± 0.02 (median, 95%\nCI). Bz-MS had significant improved performance in comparison with Dns-MS in\nspectrum prediction for benzoylated molecules (p = 9.44e-63). (c) WCSs of predicted\nspectra generated by Dns-MS and Bz-MS for dansylated molecules in the test set\nmatching with the 167 experimental spectra. Dns-MS: 0.69 ± 0.02 (median, 95% CI),\nBz-MS: 0.43 ± 0.03 (median, 95% CI). Dns-MS had significant improved performance\nin comparison with Bz-MS in spectrum prediction for dansylated molecules (p = 4.93e-\n28). ****: p ≤ 0.0001.\nComment-9: While WCS is used as a metric for evaluating the predictability of the\nmodels, additional metrics or methods could provide a more comprehensive evaluation\nof the model's performance.\nResponse-9: Besides WCS, we also employed two other metrics for evaluating the\npredictability of our model, including regular cosine similarity and spectral entropy. We\nincluded details of the two other metrics in the revised manuscript.\nRevision-9: Model Evaluation describing methods of WCS, cosine similarity and\nspectral entropy was added to the Methods of the revised manuscript. Table S2 was\nadded in the Supporting Information.\nRevision in manuscript:\n“Model Evaluation. 3 metrics were tested for model evaluation: weighted cosine\nsimilarity (WCS), cosine similarity and spectra entropy. WCS was calculated using\nequation (1):\n……….….. (1)48\n𝑀𝑚𝑎𝑥 0.5 0.5 0.5 0.5\n∑𝑘=1 𝑚𝑘 𝐼𝑞𝑘∙𝑚𝑘 𝐼𝑙𝑘\n𝑆𝑖𝑚𝑖𝑙𝑎𝑟𝑖𝑡𝑦(𝐼𝑞,𝐼𝑙) = 𝑀𝑞 0.5 0.5 2 𝑀𝑙 0.5 0.5 2\nCosine similarity was calculated from √ eq ∑ u𝑘=a1t(io𝑚n𝑘 (𝐼2𝑞)𝑘:) √∑𝑘=1(𝑚𝑘 𝐼𝑙𝑘 )\n…………...... (2)48\n𝑀𝑚𝑎𝑥 0.5 0.5\n∑𝑘=1 𝑚𝑘𝐼𝑞𝑘∙𝑚𝑘𝐼𝑙𝑘\n𝑆𝑖𝑚𝑖𝑙𝑎𝑟𝑖𝑡𝑦(𝐼𝑞,𝐼𝑙) = 𝑀𝑞 0.5 2 𝑀𝑙 0.5 2\nWhere m was the mass-to-charge ratio √ ∑(𝑘m=/1z(𝑚) 𝑘 o𝐼f𝑞 𝑘fr)a √ g∑m𝑘e=n1(t𝑚; I 𝑘𝐼𝑙𝑘 a)nd I was the intensity\nk qk lk\nof fragment (m/z = k) in the query spectrum and in the reference spectrum; M and M\nl q\nwere the largest nonzero indices of I and I; M was the largest index of M and M .\nq l max l q\nSpectra entropy S was calculated from all ion intensities I by equation (3):\np\n……………….……..…...……... (3)49\nConsistent results were found a\n𝑆\nm\n=\non−g\n∑\nth𝑝e𝐼 𝑃3 lmn𝐼e𝑃trics in the comparison of 3 models with\nslightly better WCSs, using the hold-out test set of 167 dansylated molecules. Therefore,\nWCS was selected in this study to evaluate the predictability of models.”\nRevision in SI:\nTable S2. Three metrics to evaluate the spectra similarity between predicted spectra\ngenerated by 3 models and 167 experimental spectra of dansylated molecules in test set.\nMedian ± 95%CI\nMethods\nDns-MS NEIMS CFM-ID 4.0\nweighted cosine similarity 0.69±0.02 0.39±0.01 0.46±0.02\ncosine similarity 0.65±0.02 0.31±0.01 0.29±0.02\nspectra entropy 0.63±0.02 0.29±0.02 0.27±0.02\n- DISCUSSION\nComment-10: While DeepCDM is compared to existing tools in terms of efficiency, a\nmore in-depth comparative analysis, including strengths and weaknesses of each tool,\nwould provide a clearer understanding of DeepCDM's advantages.\nResponse-10: Thank you for the suggestion. To provide a clearer understanding of the\nadvantages of DeepCDM, we further expanded our discussion on the relationship and\ncomparison of DeepCDM with existing tools.\nRevision-10: Revision was made to the Discussion of the revised manuscript:\n“Unlike existing MS/MS spectrum predicting tools, such as NEIMS and CFM-ID 4.0,\nthat are generic for all molecules, DeepCDM aims to enhance the spectrum\npredictability of a more focused group of molecules undergone given chemical\nderivatizations. Both Dns-MS and Bz-MS are highly specific to dansylated molecules\nand benzoylated molecules, respectively, which outperformed generic tools in\npredicting ESI-MS/MS spectra for CDMs with over 150% enhanced WCS values.\nThe superior performance of DeepCDM is achieved upon transfer learning that converts a\ngeneric tool into a highly specific one for CDMs. For example, although both Dns-MS\nand Bz-MS were built upon the architecture of NEIMS, each of the two models\ndemonstrated high predictability of its corresponding CDMs rather than the other group.\nTherefore, DeepCDM is a powerful addition to the current toolbox for MS/MS\nspectrum prediction. It is also worth noting that DeepCDM is not bound to NEIMS but\ncan be adapted to any other exiting prediction algorithm for addressing specific\nchallenges.”\n“Because of the remarkable MS2C capability, DeepCDM facilitates the annotation of\nCDMs. Current compound identification tools, such as SIRIUS 4, CFM-ID 4.0\ncompound identification module and CD 3.3 access databases containing limited\ninformation of CDMs. Due to the enormous differences between MS/MS spectra of\nCDMs and spectra of underived molecules, annotating CDMs using these compound\nidentification tools is difficult and low in efficiency.”\nReviewer #2:\nThis study uses deep learning to generate an in-silico reference library of spectra from\ntandem mass spectrometry (MS/MS) for chemically derivatized molecules (CDM)\ntypically used in non-targerted analysis (NTA). The reported effectiveness of the\nresultant reference library DnsBank is incredibly strong and the idea using transfer\nlearning for mass spectra prediction is still pretty novel to my knowledge. Transfer\nlearning is an excellent choice in this situation where very little training data is available.\nI've seen the effectiveness of transfer learning from an initially weak spectral prediction\ntool to a more specialized set in my own work. This should be a strong approach,\nespecially if the dansylation process has a specific impact on how the molecules\nfragment. The trials run for this study between Dns-MS and Dns-MS-beta emphasize\nthe effectiveness of transfer learning in a situation where general prediction (like that\nperformed by NEIMS) is not accurate enough for use in the field. The experimental\nstudy seems very promising as well and adds to the excitement of the paper, but I don't\nhave the scope of knowledge to gauge how important those results are.\nComment-1: I think the argument for this tool would be made stronger by investigating\nthe cosine similarity of the reference spectra and spectra of the dansylated molecules\nfor the 507 molecules in the experimental set. If there are significant differences\nbetween a molecule's reference spectra traditionally found in MoNA and the dansylated\nform, this creation of DnsBank should be immediately helpful in the field of NTA.\nReferences 15 and 16 (Liu 2019, Zhao 2024) attribute challenges in NTA to many\nfactors, but if analysts are looking for spectra of dansylated molecules in libraries of\noriginal molecules DnsBank seems like a way to remove one obstacle. On the other hand,\nif the reference spectra and the spectra of the dansylated forms have high cosine\nsimilarities scores, this outcome might be attributed to some other effect.\nResponse-1: Thank you for this kind suggestion. We found 231 reference spectra of\nunderived molecules in the training set from public libraries including MoNA,\nMassBank and HMDB. The WCS values of the reference spectra of underived\nmolecules and the experimental spectral of their dansylated forms was 0.02 ± 0.01\n(median, 95% CI) (Figure S1a). The low spectra similarities between underived and\ndansylated molecules demonstrated the challenge in the identification of CDMs using\ncurrent spectral libraries.\nRevision-1: Figure S1a was added in the Supporting Information and corresponding\ndiscussion was made in the revised manuscript:\n“Moreover, CDMs often generate significantly different MS/MS spectra from the\noriginal molecules. For instance, low similarity with WCS value of only 0.02 ± 0.01\n(median, 95% CI) was found between the reference spectra of 231 underived molecules\nrecorded in the MassBank of North America (MoNA), MassBank and Human\nMetabolome Database (HMDB) and the experimentally obtained spectra of their\ndansylated forms (Figure S1a).”\nRevision in SI:\nFigure S1. (a) The weighted cosine similarity (WCS) between MoNA, MassBank and\nHMDB reference spectra of original molecules, and experimental spectra of dansylated\nmolecules. Median WCS = 0.02 ± 0.01, 95% CI, n = 231. (b) WCSs between 167\nexperimental spectra of dansylated molecules and predicted spectra by NEIMS, CFM-\nID 4.0 and Dns-MS. NEIMS achieved 0.38 ± 0.01 (median, 95% CI) WCS, CFM-\nID 4.0 had 0.46 ± 0.02 (median, 95% CI) WCS and Dns-MS achieved 0.69 ± 0.02 (median,\n95% CI).\nComment-2: From a machine learning stand point, I wonder why this team has chosen\nto start from a relatively old model when there have been several advances in spectral\nprediction with deep learning**. I understand NEIMS is easy because of code\navailability, but there are models that are more crafted for the task of ESI-MS/MS\nspectral prediction. Similarly, most of the current efforts train on NIST (even NEIMS\ntrained on NIST) due to its size and accessibility. The training set from MoNA is very\nsmall and not well curated. Young, Rost, & Wang 2024\n(https://doi.org/10.1038/s42256-024-00816-8) give a good overview of why NIST is a\npreferred training set.\nResponse-2: We chose NEIMS because it is an open-source machine learning model\nshowing reasonably good performance in spectra prediction. NEIMS has also been used\nin many studies for performance comparisons (Anal. Chem., 2023, 95, 2653–2663; Anal.\nChem., 2024, 96, 4835–4844; Int. J. Mass Spectrom., 2022, 475, 116817;\nBioinformatics, 2023, 39, btad354), and this model is thus well understood and accepted\nby the community. Nevertheless, as a transfer learning approach, DeepCDM is not\nbound to NEIMS and can, in principle, be adapted to other machine learning models,\nsuch as MassFormer. We will try this idea in our future work. MoNA dataset was used\nin DeepCDM is because MoNA is open to public. Building DeepCDM on an open\ndatabase would benefit researchers from diverse research fields, such as environment\nsciences and biomedical research.\nComment-3: Why is similarity determined with a weighted cosine similarity? The\nreference for this metric is 30 years old (41, Stein & Scott 1994). Young, Rost & Wang\n2024 give a strong argument for regular cosine similarity and Li et al 2021\n(https://doi.org/10.1038/s41592-021-01331-z) introduce another strong metric in\nspectral entropy. Also, while 0.6 (lines 127-128) is a reasonable threshold for how\nwell deep learning generally predicts MS/MS spectra, it is still low for analysts trying to\nmatch their experimental spectra to library entries.\nResponse-3: Weighted cosine similarity (WCS) was used to evaluate spectral similarity\nin this work because this metric is widely accepted and used in numerous studies since\nits introduction in 1994 (J. Mass Spectrom., 2010, 45, 703–714; Anal. Chem., 2016, 88,\n7689–7697; Nat. Commun., 2023, 14, 3722). We also examined the two suggested\nmetrics, including regular cosine similarity and spectral entropy, in the C2MS\nbenchmarks between Dns-MS, NEIMS and CFM-ID 4.0 using a hold-out test set.\nConsistent results were found among the three metrics with slight better WCSs (Table\nS2).\nWe chose WCS at 0.6 as a threshold for matching experimental spectra to library entries.\nThis is because for practical uses of spectra predicting tools in nontargeted analyses,\nwe must balance the spectra similarity and the number of tentative annotates. We added\nFigure S9 in the revised manuscript, revealing that the number of tentative annotates\ndecreased as we increased the threshold value. Moreover, we aim to profile annotates\nat the confidence level 3, which is defined as evidence exists for possible structure(s)\nbut insufficient information for one exact structure (Environ. Sci. Technol.,2014, 48,\n2097–2098). With the need to ensure total number of tentative annotates and the\nstringency at the confidence level 3, we reason that WCS at 0.6 is a good trade-off to\nmatch experimental spectra to library entries in our nontarget analyses of environmental\nsamples.\nIn the revised manuscript, we added all relevant data, methods, and discussions.\nRevison-3: Methods used for WCS, regular cosine similarity and spectral entropy\ncalculation were included in Methods of the revised manuscript. Table S2 and Figure\nS9 was included in the Supporting Information.\nRevision in manuscript:\n“Using WCS ≥ 0.6 as a threshold44 since it provided reasonable number of annotations\nfor further analysis (Figure S14)”\n“Model Evaluation. 3 metrics were tested for model evaluation: weighted cosine\nsimilarity (WCS), cosine similarity and spectra entropy. WCS was calculated using\nequation (1):\n…….…... (1)48\n𝑀𝑚𝑎𝑥 0.5 0.5 0.5 0.5\n∑𝑘=1 𝑚𝑘 𝐼𝑞𝑘∙𝑚𝑘 𝐼𝑙𝑘\n𝑆𝑖𝑚𝑖𝑙𝑎𝑟𝑖𝑡𝑦(𝐼𝑞,𝐼𝑙) = 𝑀𝑞 0.5 0.5 2 𝑀𝑙 0.5 0.5 2\nCosine similarity was calculated from √ eq∑u𝑘=a1t(io𝑚n𝑘 (𝐼2𝑞)𝑘:) √∑𝑘=1(𝑚𝑘 𝐼𝑙𝑘 )\n….……...... (2)48\n𝑀𝑚𝑎𝑥 0.5 0.5\n∑𝑘=1 𝑚𝑘𝐼𝑞𝑘∙𝑚𝑘𝐼𝑙𝑘\n𝑆𝑖𝑚𝑖𝑙𝑎𝑟𝑖𝑡𝑦(𝐼𝑞,𝐼𝑙) = 𝑀𝑞 0.5 2 𝑀𝑙 0.5 2\nWhere m was the mass-to-charge ratio √ ∑ (𝑘m=/1z(𝑚) 𝑘 o𝐼f𝑞 𝑘fr)a √ g ∑ m𝑘e=n1(t𝑚; I 𝑘𝐼𝑙𝑘 a)nd I was the intensity\nk qk lk\nof fragment (m/z = k) in the query spectrum and in the reference spectrum; M and M\nl q\nwere the largest nonzero indices of I and I; M was the largest index of M and M .\nq l max l q\nSpectra entropy S was calculated from all ion intensities I by equation (3):\np\n……………….………...……... (3)49\nConsistent results were found a\n𝑆\nm\n=\non−g\n∑\nth𝑝e𝐼 𝑃3 lmn𝐼e𝑃trics in the comparison of 3 models with\nslightly better WCSs, using the hold-out test set of 167 dansylated molecules. Therefore,\nWCS was selected in this study to evaluate the predictability of models”\nRevision in SI:\nFigure S14. Total annotation numbers of 3 wastewater samples by spectra matching\nin DnsBank using different WCS thresholds.\nTable S2. Three metrics to evaluate the spectra similarity between predicted spectra\ngenerated by 3 models and 167 experimental spectra of dansylated molecules in test set.\nMedian ± 95%CI\nMethods\nDns-MS NEIMS CFM-ID 4.0\nweighted cosine similarity 0.69±0.02 0.39±0.01 0.46±0.02\ncosine similarity 0.65±0.02 0.31±0.01 0.29±0.02\nspectra entropy 0.63±0.02 0.29±0.02 0.27±0.02\nComment-4: Finally, the use of the phrase 'fine tuning' in machine learning is imprecise\nand should be defined. How was it tuned? Were there further changes beyond the\nnumber of layers and nodes? How were these 'tuned' values selected?\nResponse-4: The layers and nodes of MLP were first optimized in the pre-training\nmodel using the MoNA dataset. The goal is to obtain a reasonable WCS (Table S1). The\npre-trained model, ESI-MLP, was then fine-tuned by freezing all but the last layer of\nMLP to generate Dns-MS. During fine-tuning, the weights and bias were tuned.\nRelevant discussion about layers and nodes optimization was included in the revised\nmanuscript.\nRevision-4: Details of optimization of layer and node number were included in the\nrevised manuscript. Table S1 was added to the Supporting Information.\nRevision in manuscript:\n“The hidden layers of ESI-MLP were optimized from 7 layers of 2000 nodes to 3 layers\nof 500 nodes (Table S1).”\nRevision in SI:\nTable S1. Hidden layers and units of MLP optimization using the MoNA dataset.\nHidden layers = 3 Hidden layers = 5 Hidden layers = 7\nHidden units\nWCS WCS WCS\n500 0.56 ± 0.02 0.57 ± 0.02 0.54 ± 0.02\n1000 0.55 ± 0.02 0.57 ± 0.02 0.56 ± 0.02\n1500 0.57 ± 0.02 0.57 ± 0.02 0.57 ± 0.02\n2000 0.57 ± 0.02 0.56 ± 0.02 0.57 ± 0.02\n500/6790 molecules in the MoNA dataset were used for MLP optimization.\nComment-5: More generally, a key strength of ESI-MS/MS is the ability to examine\nspectra at different collision energies. In that vein, collision-energy-merged spectra\n(line 82) are not as desirable as the reference spectra found in existing libraries (e.g.,\nMoNA or NIST). However, if this is the standard output for NTA, creating a reference\nlibrary that specifically meets the need for collision-energy-merged spectra is very\nhelpful. Otherwise, creating these merged spectra does simplify the problem from a\nmachine learning stand point. This also could be skewing the reported performance if\nthey are searching for matches to collision-energy-merged spectra in reference libraries\nwhere spectra are reported at different collision energies (e.g., using CFM-ID 4.0's\ncompound identification model probably compares against non-merged spectra). More\ndetails about the benchmarking would be very helpful.\nResponse-5: In LC-MS based non-targeted analysis of complex samples, merged\nspectra are often acquired at stepped collision energies to ensure effective\nfragmentation of compounds having diverse chemical structures (Sci. Rep., 2024, 14,\n51; Anal. Chem., 2011, 83, 7651–7656; Anal. Chem., 2019, 91, 2155–2162; TrAC\nTrends Anal. Chem., 2023, 160, 116966; Metabolomics, 2020, 16, 107; Metabolites,\n2020, 10, 135). So, yes, the use of collision-energy-merged spectra is quite standard in\nnon-targeted analysis and creating a reference library with collision-energy-merged\nspectra is quite necessary to improve annotation efficiency in non-targeted analysis. We\nalso added more details on conditions of benchmarking with CFM-ID 4.0 compound\nidentification module and SIRIUS 4.\nRevision-5: Revision was made in Methods of the revised manuscript:\n“Benchmarks. C2MS benchmarks between Dns-MS, Bz-MS, NEIMS and CFM-ID\n4.0 (web server) were performed on hold-out test sets. NEIMS was acquired from ref.\n20 and CFM-ID 4.0 (web server, http://cfmid.wishartlab.com/predict) was used for\nspectra prediction.\nMS2C benchmark was performed between DnsBank and other public spectral libraries\naccessed by CFM-ID 4.0 compound identification module\n(http://cfmid.wishartlab.com/identify) and PubChem accessed by SIRIUS 4 on the\nhold-out test set containing 167 dansylated spectra. In silico spectra libraries of ChEBI,\nDSSTox and STOFF-IDENT predicted by CFM-ID 4.0 and public spectral libraries\nincluding HMDB, MassBank and MoNA were used by CFM-ID 4.0 compound\nidentification module for annotating dansylated molecules through spectrum matching.\nESI was selected as the spectrum type, ion mode was positive and adduct type was set\nto [M+H]+ in CFM-ID 4.0, with 10 ppm mass tolerance and DotProduct as the scoring\nfunction. For molecule annotation by SIRIUS 4, parameters were set to be instrument:\norbitrap, database: PubChem, possible ionization: [M+H]+. CSI:FingerID was used for\nfingerprint prediction and structure database search.”\nComment-6: The reported benchmarks and poor performance of CFM-ID and\nSIRIUS4 seem very surprising, again I think more clarity in the benchmarking process\nwould be helpful here. From the Fiehn lab's CASMI 2022 test (where I think they do\nuse merged spectra), the group behind SIRIUS4 was highly successful in compound\nidentification and the CFM-ID team also performed reasonably well. Again, this could\nbe attributed to the differences between a molecule's spectrum before and after it's been\ndansylated. A review of the overlap between the presence of the test molecules in the\ndifferent reference libraries (DSSTox, HMDB, MassBank JP/EU/NA, DnsBank, etc) would also help,\nto show that the correct results are available and the CFM-ID and SIRIUS4 tools are still not finding them.\nDiscussion of the benchmarking in the caption of Figure 3 is helpful, more detail should be added to\nthe report in the body of the paper. A summarizing table might be appropriate.\nResponse-6: The presence of the test molecules in reference libraries used by CFM-ID\n4.0 compound identification module and SIRIUS 4 was manually checked and reported\nin the manuscript. 3 out of 167 dansylated molecules were found in the libraries used\nby CFM-ID 4.0, and 7 were found in PubChem, which was used by SIRIUS 4. More\ndetails of their presences in reference libraries were summarized in Supporting Data-\n02. Detailed parameters used in CFM-ID 4.0 and SIRIUS 4 benchmarking was added\nto Methods in the revised manuscript as shown in Comment-5. A short description about\nthe benchmarking was also added to the caption of Figure 3c.\nRevision-6: Revision was made to the manuscript:\nFigure 3 (c) “CFM-ID 4.0 failed to annotate any dansylated molecules in the test set,\nalthough 3 molecules were found in its libraries by manual checking. SIRIUS 4\ncorrectly annotated 7 dansylated molecules (4.19%) in top 5 output and 8 (4.79%) in\ntop 25. In total, 15 dansylated molecules in the test set were found in PubChem by\nmanual checking, 8 of them were annotated by SIRIUS 4.”\nMinor notes:\nComment-7: The citation of Grimme's QCEIMS on line 51 should be updated to refer\nto Grimme's QCxMS which actually does quantum chemical simulations of ESI-\nMS/MS, not just EI-MS.\nResponse-7: The reference was updated as suggested.\nComment-8: CFM-ID is not a deep learning tool like NEIMS, they are far apart on the\nspectrum of machine learning tools. CFM-ID replies on probabilistic models applied to\ncombinatorial generation of potential fragments. This approach is why CFM-ID doesn't\nover-predict small peaks like NEIMS does, it can only predict peaks at the locations of\ntrue potential fragments.\nResponse-8: Thank you for this reminder. We found that CFM-ID was reported as the\nmachine-learning-based competitive fragmentation modeling method by Wang, et al.\nin Anal. Chem. 2021, 93, 11692−11700. Based on this previous report, we still\nconsidered it as a machine learning but is open to further suggestions by the reviewer.\nComment-9: NEIMS's over-prediction of small peaks should not be attributed to its\noriginal application to EI-MS, there is nothing specific to EI-MS built into NEIMS. In\nfact, NEIMS has been retrained many times over with ESI-MS/MS data** and still has\nthese features, it is more likely the difficulty of using deep learning for spectral\nprediction.\nResponse-9: The discussion of why NEIMS overpredicted small fragments in the\nspectrum prediction of dansylated 4-isopropyl-3-methylphenol was removed from the\nrevised manuscript as it’s not related to our work.\nComment-10: Why aren't the results of Dns-MS compared in Figure S1?\nResponse-10: The results of Dns-MS was added in Figure S1b as suggested.\nComment-11: For the most part, the code is from the NEIMS paper (Wei, 2019) which\nmy team has used and retrained so in that sense the code is a usable resource. There are\nsmall changes in (1) gather_similarities.py to allow for batching, (2)\nmake_spectra_prediction.py to perform the transfer learning, and (3)\nmolecule_predictors.py/molecule_predictors_transfer.py to perform the transfer\nlearning. I didn't see a specific README file to explain how to use DeepCDMs instead\nof NEIMS. The original codes in the SI Codes folder could use more complete\ndocumentation. 1_dansylation_filtering.py and 4_sort_match_res.py are very straight\nforward. 2_SMART_reaction.py requires chemistry understanding beyond my\nexpertise, but the code seems straight forward enough. The authors say\n3_spectra_matching.py is from the NEIMS original script, but seems to be pieced\ntogether and could use additional documentation/commenting. I did not attempt to run\nthe code because their computed weights for the model are not included in the\nrepository.\nResponse-11: A more detailed README file explaining codes used in DeepCDM and\nits workflow was uploaded to Github (http://github.com/ADNLab-SCU/DeepCDMs)\nReviewer #3:\nThe authors present a deep learning approach for predicting MS/MS spectra for\nchemically derivatized molecules. The authors adapted a model, which was originally\ndeveloped for predicting ESI MS/MS spectra of chemical molecules, by fine-tuning it\non a dataset of chemically derivatized molecules and specifically dansylated molecules.\nThe authors utilized the model for developing a dansylation-specific spectra library\nwhich was used to annotate molecules in an environmental study, and they were able to\nverify experimentally 4 annotations.\nComment-1: The experimental validation is interesting, however, the novelty of the\nmethod is limited as an existing model has been further tuned in a different\ndataset/domain.\nResponse-1: Thank you for this comment. However, we do not fully agree with the\nreviewer regarding the novelty of our work. Indeed, there were existing spectra\npredicting tools, including quantum chemistry models and machine learning models.\nThe quantum chemistry models are cost intensive and have to be built from scratch for\ndifferent tasks, thus are limited in applications. Machine learning models are efficient,\nsuch as CFM-ID and MassFormer. But they are designed for underived molecules,\ntherefore none of existing tools were able to predict the mass spectra of chemically\nmodified molecules (CDMs), a class of highly important compounds in non-targeted\nanalysis. As far as we know, DeepCDM specially tailors machine learning models for\nCDMs spectra prediction, which is the first approach aiming to remove the obstacle\nlying in CDM identification. The outstanding performance of DeepCDM also\ndemonstrates the possibility to engineer a generic model for a specific class of\ncompounds and the power of transfer learning enabling accurate prediction based on a\nsmall pool of experimental data. Our work directs a new solution for improving mass\nspectra prediction, creates a new tool for predicting mass spectra for CDMs, and new\napplications in non-targeted analysis. Therefore, we believe that our work is novel and\nsignificant, and is a critical addition to the current toolbox for mass spectra prediction.\nComment-2: Additionally, the machine learning model is not adequately discussed in\nthe manuscript. Although, the architecture of an existing model has been reused, there\nare important issues that should be discussed in the paper.\nResponse-2: We added more details of the machine learning model in the revised\nmanuscript. Details of fine tuning was also included in the revised manuscript.\nRevision-2: The architecture of the machine learning model was included in the revised\nSI:\n“1.2 NEIMS architecture and the input & output of the model\nNEIMS uses multilayer perceptron architecture with residual network connections\nbetween the layers to predict spectra from an input molecule, mapped to Extended-\nConnectivity Fingerprints (ECFPs).”\nDetails of fine tuning was also added in the manuscript:\n“The hidden layers of ESI-MLP were optimized from 7 layers of 2000 nodes to 3 layers\nof 500 nodes (Table S1).”\nRevision in SI:\nTable S1. Hidden layers and units of MLP optimization using the MoNA dataset.\nHidden layers = 3 Hidden layers = 5 Hidden layers = 7\nHidden units\nWCS WCS WCS\n500 0.56 ± 0.02 0.57 ± 0.02 0.54 ± 0.02\n1000 0.55 ± 0.02 0.57 ± 0.02 0.56 ± 0.02\n1500 0.57 ± 0.02 0.57 ± 0.02 0.57 ± 0.02\n2000 0.57 ± 0.02 0.56 ± 0.02 0.57 ± 0.02\n500/6790 molecules in the MoNA dataset were used for MLP optimization.\nComment-3: More importantly, an analysis that would demonstrate generalization\ncapabilities of the model and an analysis that would show that the model has not been\nover-fitted should be included.\nResponse-3: Thank you for the two suggestions. First, to demonstrate the versatility of\nDeepCDM, we established DeepCDM for benzoylated molecules, another important\nclass of CDMs, during the revision. We termed this new prediction model as Bz-MS.\nLike Dns-MS, Bz-MS demonstrated excellent performance in MS/MS spectra\nprediction of corresponding CDMs, confirming the versatility and robustness of our\nDeepCDM approach. Details of Bz-MS modeling and evaluation were added to the\nrevised manuscript.\nSecond, to determine whether the model has been over-fitted or not, a 4-fold cross\nvalidation was performed. Both models established by DeepCDM, Dns-MS and Bz-\nMS, demonstrated robust performances against different data splits, confirming that our\nmodel was not overfitted (Figure S6a and S12). Moreover, for every query molecule in\nthe test set, we returned the spectrum of the structurally closest molecule in the training\nset. The median WCS value was only 0.27 ± 0.02 (95% CI, n = 167) between the\npredicted spectra in the test set and the experimental spectra of structurally closest\nmolecules in the training set, further confirming that our model has the capability to\ndifferentiate structurally similar molecules and our model was not overfitted. We also\nincluded all validation and comparison results in the revised manuscript.\nRevision-3: (a) Generalization capabilities: Details of Bz-MS modeling and\nevaluation were added to the Results part of the revised manuscript. Figure S10-S13\nwere included in the Supporting Information.\nRevision in manuscript:\n“Versatility of DeepCDM. To demonstrate that DeepCDM is a generalizable approach\nfor different classes of CDMs, we also established a spectrum predicting model for\nbenzoylation, another widely used chemical derivatization method for amine- and\nhydroxyl-containing molecules36–38. This model, termed Bz-MS, was first trained using\nthe MoNA dataset containing 6,790 merged ESI-MS/MS positive spectra of underived\nmolecules, and then fine-tuned using a training set containing 503 experimental spectra\nof benzoylated molecules (Figure S9). Evaluated with a hold-out test set containing 168\nbenzoylated molecules, Bz-MS achieved a median WCS value of 0.74 ± 0.03 (95% CI,\nn = 168) in predicting spectra for benzoylated molecules, which was significantly\nimproved from the pre-trained model, ESI-MLP (median WCS = 0.53 ± 0.02, 95% CI,\nn = 168) (Figure S10a). It also outperformed NEIMS and CFM-ID 4.0 with over 1.5-\nfold increase of median WCS in a C2MS benchmark (Figure S10 and S11). Moreover,\nthere was no bias in spectrum prediction for bezoylated amine- and hydroxyl-containing\nmolecules (Figure S10b). The robustness of Bz-MS was confirmed in a 4-fold cross-\nvalidation with 4 randomly split benzoylated datasets (Figure S12). Bz-MS was also a\nhighly specific tool for benzoylated molecules with predictability significantly better\nthan underived molecules and dansylated molecules (Figure S13). Collectively, our\nsuccess in establishing Bz-MS demonstrates that DeepCDM is a highly versatile\napproach generalizable from one class of CDMs to another. ”\nRevision in SI:\nFigure S10. (a) WCS comparisons between predicted spectra and experimental spectra\nof 168 benzoylated molecules in a hold-out test set computed by Bz-MS and ESI-MLP.\n(b) WCSs of Bz-MS predicted spectra for benzoylated molecules with amine and\nhydroxyl in the test set. Amines: 0.74 ± 0.03, hydroxyls: 0.72 ± 0.06 (median, 95% CI).\nNo significant difference was observed between the 2 groups (p = 0.1263) (c) WCSs of\npredicted spectra generated by NEIMS, CFM-ID 4.0 and Bz-MS for benzoylated\nmolecules in the test set matching with the 168 experimental spectra. NEIMS: 0.40 ±\n0.02, CFM-ID 4.0: 0.49 ± 0.02 and Bz-MS: 0.74 ± 0.03 (median, 95% CI). Bz-MS had\nsignificantly enhanced WCSs in comparison with CFM-ID 4.0 (p = 1.26e-25) and\nNEIMS (p = 4.25e-54). (d) The predicted spectra of dansylated 2-amino-6-\nmethoxypyridine generated by NEIMS, CFM-ID 4.0 and Bz-MS, were matched with\nits experimental spectrum. ns: p > 0.05, ****: p ≤ 0.0001.\nFigure S11. (a) WCS comparisons of NEIMS and Bz-MS predicted spectra for 168\nbenzoylated molecules in the test set. (b) WCS comparisons of CFM-ID 4.0 and Bz-\nMS predicted spectra for 168 benzoylated molecules in the test set.\nFigure S12. 4-fold cross validation of Bz-MS. For each data split, no significant\ndifference was observed (p = 0.3196). Random split 1: 0.74 ± 0.03, random split 2: 0.77\n± 0.03, random split 3: 0.73 ± 0.03 and random split 4: 0.76 ± 0.03 (median, 95% CI).\nns: p > 0.05.\nFigure S13. (a) WCS between the reference spectra of 6,790 underived molecules in\nthe pre-training MoNA dataset and their predicted spectra generated by Bz-MS, NEIMS\nand CFM-ID 4.0. Bz-MS achieved WCS of 0.53 ± 0.01 (median, 95% CI), NEIMS\nachieved WCS of 0.28 ± 0.01 (median, 95% CI) and CFM-ID 4.0 had WCS of 0.49 ±\n0.01 (median, 95% CI). (b) WCSs of predicted spectra generated by Bz-MS and Dns-\nMS for benzoylated molecules in the test set matching with the 168 experimental\nspectra. Bz-MS: 0.74 ± 0.03 (median, 95% CI), Dns-MS: 0.34 ± 0.02 (median, 95%\nCI). Bz-MS had significant improved performance in comparison with Dns-MS in\nspectrum prediction for benzoylated molecules (p = 9.44e-63). (c) WCSs of predicted\nspectra generated by Dns-MS and Bz-MS for dansylated molecules in the test set\nmatching with the 167 experimental spectra. Dns-MS: 0.69 ± 0.02 (median, 95% CI),\nBz-MS: 0.43 ± 0.03 (median, 95% CI). Dns-MS had significant improved performance\nin comparison with Bz-MS in spectrum prediction for dansylated molecules (p = 4.93e-\n28). ****: p ≤ 0.0001.\n(b) Validation of fitting robustness: 4-fold cross validation of Dns-MS was added to\nthe Results part of the revised manuscript. Figure S6 were included in the revised SI.\nRevision in manuscript:\n“In a 4-fold cross-validation, Dns-MS demonstrated robust performances for 4\nrandomly split dansylated dataset (Figure S6a). To evaluate the capability of Dns-MS\nin distinguishing molecules with similar structures, the Dns-MS predicted spectrum and\nexperimental spectrum of every dansylated molecule in the test set was matched with\nthe experimental spectrum of a structurally most similar molecule in the training set.\nThe predicted spectra of query molecules had low similarity with the experimental\nspectra of their structurally similar molecules with a median WCS of 0.27 ± 0.02 (95%\nCI, n = 167), suggesting that Dns-MS was highly robust and capable of distinguishing\nstructurally similar molecules in spectrum prediction (Figure S6b and S6c). Similar\nobservation was also made between the experimental spectra of query molecules in the\ntest set with those of structurally similar molecules in the training set (median WCS =\n0.22 ± 0.03, 95% CI, n = 167). That further confirmed the robustness of our model.”\nRevision in SI:\nFigure S6. (a) 4-fold cross validation of Dns-MS. For each data split, no significant\ndifference was observed (p = 0.9915). Random split 1: 0.69 ± 0.02, random split 2: 0.73\n± 0.02, random split 3: 0.70 ± 0.02 and random split 4: 0.71 ± 0.02 (median, 95% CI).\n(b) WCS between predicted & experimental spectra of dansylated molecules in test set\nand experimental spectra of the structurally most similar molecules in training set. The\npredicted spectra of dansylated molecules in test set had a median WCS of 0.27 ± 0.02\n(95% CI, n = 167) when matching with the experimental spectra of their structurally\nmost similar molecules in training set. The experimental spectra of dansylated\nmolecules in test set had a median WCS of 0.22 ± 0.03 (95% CI, n = 167) when\nmatching with that of their structurally most similar molecules in training set. However,\nthe predicted and experimental spectra of dansylated molecules achieved a median\nWCS of 0.69 ± 0.02 (95% CI, n = 167). Both the predicted and experimental spectra of\ndansylated molecules in test set were significantly different from the experimental\nspectra of their structurally most similar molecules in training set ((p = 2.97e-75 for\npredicted vs closet experimental and p = 1.36e-60 for experimental vs closet\nexperimental). (c) Matching the predicted and experimental spectra of dansylated 4-\nisopropyl-3-methylphenol with the experimental spectrum of its structurally most\nsimilar molecules in training set: dansylated 3-methyl-4-(methylthio)phenol. ns:\np > 0.05, ****: p ≤ 0.0001.\nComment-4: A short paragraph explaining what the input and output of the model\nshould be included as well as the representations of molecules and spectra.\nResponse-4: The input of Dns-MS was the ECFP features of molecules, and the output\nwas predicted peak lists providing information of fragments m/z and intensity. Based\non the peak lists, MS/MS spectra of molecules were generated.\nRevision-4: The input and output of the model was included in S1.2 of the Supporting\nInformation.\nRevision in SI:\n“1.2 NEIMS architecture and the input & output of the model\nNEIMS uses multilayer perceptron architecture with residual network connections\nbetween the layers to predict spectra from an input molecule, mapped to Extended-\nConnectivity Fingerprints (ECFPs). The output of NEIMS is a vector that represents\nthe intensity at every integral m/z bin. The input and output of Dns-MS and Bz-MS are\nthe same with NEIMS.”\nComment-5: The hyper-parameters of the model and the training parameters should be\nadded in the supplementary material.\nResponse-5: The hyperparameters of the model were added in Methods of the revised\nmanuscript. The optimization of training parameters was summarized in Table S1.\nRevision-5: The hyperparameters of the model were briefly described in Methods.\nTable S1 was added to the Supporting Information.\nRevision in manuscript:\n“The hyperparameters were optimized on the hidden units and layers of MLP. The\noptimized MLP architecture using ReLU activation had 3 layers of 500 nodes.”\n“Further details on hyperparameter configurations were included in the code repository\nand can be obtained through Github (http://github.com/ADNLab-SCU/DeepCDMs).”\nRevision in SI:\nTable S1. Hidden layers and units of MLP optimization using the MoNA dataset.\nHidden layers = 3 Hidden layers = 5 Hidden layers = 7\nHidden units\nWCS WCS WCS\n500 0.56 ± 0.02 0.57 ± 0.02 0.54 ± 0.02\n1000 0.55 ± 0.02 0.57 ± 0.02 0.56 ± 0.02\n1500 0.57 ± 0.02 0.57 ± 0.02 0.57 ± 0.02\n2000 0.57 ± 0.02 0.56 ± 0.02 0.57 ± 0.02\n500/6790 molecules in the MoNA dataset were used for MLP optimization.\nComment-6: An analysis of the dataset that was used to fine-tune the model is needed.\nHow diverse is the dataset? What is the average pairwise similarity of the molecules in\nthe dataset? What is the distribution of the molecular weight (or other properties that\nmay help the reader understand the dataset diversity)?\nResponse-6: The original molecular weights of Dns-MS training set before dansylation\nwere ranged from 59.0735 Da to 665.6982 Da, containing 62 classes of molecules with\nan average pairwise similarity of 0.11 (Figure S2). For Bz-MS, the training set\ncontained 65 classes of molecules, weight ranged from 57.0578 Da to 504.3172 Da\nbefore benzoylation. The average pairwise similarity of molecules in the training set\nwas 0.10 (Figure S9).\nRevision-6: Figure S2 and S9 were added in the revised SI. Corresponding discussion\nwas made in the revised manuscript:\n“The original molecular weights of the training set were ranged from 59.0735 to\n665.6982 Da, containing 62 classes of molecules with an average pairwise similarity of\n0.11 (Figure S2).”\nRevision in SI:\nFigure S2. (a) Molecular weight distribution (range from 59.0735 to 665.6982 Da) of\n507 underived molecules in the training set of dansylation. (b) Heat map of pairwise\nsimilarity for the 507 underived molecules. The average pairwise similarity was 0.11.\n(c) 10 × 10 dot plot of 62 chemical classes for 507 underived molecules in the training\nset for Dns-MS.\nFigure S9. (a) Molecular weight distribution (range from 57.0578 to 504.3172 Da) of\n503 underived molecules in the training set of benzoylation. (b) Heat map of pairwise\nsimilarity for the 503 underived molecules. The average pairwise similarity was 0.10.\n(c) 10 × 10 dot plot of 65 chemical classes for 503 underived molecules in the training\nset for Bz-MS.\nComment-7: How was the dataset split into train-test? Is it a random split? Was\nstructural similarity taken into account?\nResponse-7: The dataset was randomly split into a training set and a test set. Structural\nsimilarity was not considered in dataset splitting.\nRevision-7: Details of dataset splitting was added in the Methods of revised manuscript:\n“Dataset was randomly split without considering structural similarity of dansylated\nmolecules.”\nComment-8: The authors should discuss in the paper how they prevented and how they\nevaluated model over-fitting.\nResponse-8: The robustness of the model was evaluated by a 4-fold cross validation as\ndetailed in Comment-3.\nComment-9: I suggest the following experiment as a very simple baseline which can\nhelp us understand whether over-fitting may have occurred: For every query molecule\nin the test set, return the spectrum of the closest molecule (structurally) in the training\nset (a simple retrieval method). What is the performance that we can obtain from this\nmethod?\nResponse-9: Thank you for this suggestion. As detailed in Comment-3, we performed\nthe suggested experiment and confirmed that no overfitting was observed for either\nDns-MS or the newly developed Bz-MS. The test results were summarized in Figure\nS6.\nRevision-9: Figure S6 was included in the revised SI. The corresponding discussion\nwas made in the revised manuscript:\n“To evaluate the capability of Dns-MS in distinguishing molecules with similar\nstructures, the Dns-MS predicted spectrum and experimental spectrum of every\ndansylated molecule in the test set was matched with the experimental spectrum of a\nstructurally most similar molecule in the training set. The predicted spectra of query\nmolecules had low similarity with the experimental spectra of their structurally similar\nmolecules with a median WCS of 0.27 ± 0.02 (95% CI, n = 167), suggesting that Dns-\nMS was highly robust and capable of distinguishing structurally similar molecules in\nspectrum prediction (Figure S6b and S6c). Similar observation was also made between\nthe experimental spectra of query molecules in the test set with those of structurally\nsimilar molecules in the training set (median WCS = 0.22 ± 0.03, 95% CI, n = 167).\nThat further confirmed the robustness of our model.”\nRevision in SI:\nFigure S6. (a) 4-fold cross validation of Dns-MS. For each data split, no significant\ndifference was observed (p = 0.9915). Random split 1: 0.69 ± 0.02, random split 2: 0.73\n± 0.02, random split 3: 0.70 ± 0.02 and random split 4: 0.71 ± 0.02 (median, 95% CI).\n(b) WCS between predicted & experimental spectra of dansylated molecules in test set\nand experimental spectra of the structurally most similar molecules in training set. The\npredicted spectra of dansylated molecules in test set had a median WCS of 0.27 ± 0.02\n(95% CI, n = 167) when matching with the experimental spectra of their structurally\nmost similar molecules in training set. The experimental spectra of dansylated\nmolecules in test set had a median WCS of 0.22 ± 0.03 (95% CI, n = 167) when\nmatching with that of their structurally most similar molecules in training set. However,\nthe predicted and experimental spectra of dansylated molecules achieved a median\nWCS of 0.69 ± 0.02 (95% CI, n = 167). Both the predicted and experimental spectra of\ndansylated molecules in test set were significantly different from the experimental\nspectra of their structurally most similar molecules in training set ((p = 2.97e-75 for\npredicted vs closet experimental and p = 1.36e-60 for experimental vs closet\nexperimental). (c) Matching the predicted and experimental spectra of dansylated 4-\nisopropyl-3-methylphenol with the experimental spectrum of its structurally most\nsimilar molecules in training set: dansylated 3-methyl-4-(methylthio)phenol. ns:\np > 0.05, ****: p ≤ 0.0001.\nComment-10: Especially for the molecule depicted in Figure 2e: What is the closest\n(structurally) molecule in the training set? What is the WCS between the spectrum of\nthe closest molecule and the one depicted in the figure?\nResponse-10: The closest structures of dansylated 4-isopropyl-3-methylphenol was\ndansylated 3-methyl-4-(methylthio)phenol in the training set. The matching between\nthe experimental and Dns-MS predicted spectrum of dansylated 4-isopropyl-3-\nmethylphenol with the experimental spectrum of dansylated 3-methyl-4-\n(methylthio)phenol had WCSs = 0.31 (Figure S6c).\nRevision-10: Matching between the experimental and Dns-MS predicted spectra of\ndansylated 4-isopropyl-3-methylphenol with the experimental spectrum of dansylated\n3-methyl-4-(methylthio)phenol was shown in Figure S6c in Supporting Information.\nRevision in SI:\nFigure S6. …(c) Matching the predicted and experimental spectra of dansylated 4-\nisopropyl-3-methylphenol with the experimental spectrum of its structurally most\nsimilar molecules in training set: dansylated 3-methyl-4-(methylthio)phenol. ns:\np > 0.05, ****: p ≤ 0.0001.\nComment-11: What is the performance of the fine-tuned model on the dataset that was\nused for pre-training the model? Is there a significant drop in the performance after\nfine-tuning the model?\nResponse-11: Thank you for this suggestion. We input the molecules in the MoNA\ndataset (used in pre-training the model) to Dns-MS to predict their MS/MS spectra.\nAs MoNA dataset was not dansylated, predictability of Dns-MS was drastically decreased\n(Figure S8), confirming that the fine-tuned model was specialized for dansylated\nmolecules. We added this observation in the revised manuscript as Figure S8.\nRevision-11: Figure S8 was added in the revised SI. Corresponding discussion\nwas made in the revised manuscript:\n“To evaluate the capability of DeepCDM to selectively enhance the MS/MS spectrum\npredictability of CDMs, we harnessed Dns-MS to predict spectrum for underived\nmolecules in the MoNA dataset (n = 6,790). As shown in Figure S8, the predictability\nof Dns-MS was drastically decreased for underived molecules with a median WCS of\n0.38, confirming that our DeepCDM approach is specialized for desired CDMs.”\nRevision in SI:\nFigure S8. WCS between the reference spectra of 6,790 underived molecules in the\npre-training MoNA dataset and their predicted spectra generated by Dns-MS, NEIMS\nand CFM-ID 4.0. Dns-MS achieved WCS of 0.38 ± 0.01 (median, 95% CI), NEIMS\nachieved WCS of 0.28 ± 0.01 (median, 95% CI) and CFM-ID 4.0 had WCS of 0.49 ±\n0.01 (median, 95% CI).\nComment-12: Finally, the paper is overall well-written (with a small number of typos\nand grammatical errors). However, there is very extensive use of acronyms making it\ndifficult to read the manuscript. I highly recommend reducing the use of acronyms\nacross the manuscript.\nResponse-12: The use of acronyms was reduced according to your suggestion.\nOther minor issues:\nComment-13: [lines 304-305] \"In fine-tuning, the hidden layers of the trained MLP\nwere modified from 7 layers of 2000 nodes to 3 layers of 500 nodes.\" I would expect\nthat the size of the model was modified in pre-training, not in fine-tuning. Please clarify.\nResponse-13: Indeed, the size of the model was modified in pre-training and it was\ncorrected in the revised manuscript.\nRevision-13: Correction was made in the revised manuscript.\nRevision in manuscript:\n“The hidden layers of ESI-MLP were optimized from 7 layers of 2000 nodes to 3 layers\nof 500 nodes (Table S1).”\nComment-14: MoNA and NEIMS are mentioned in the abstract but they have not been\ndefined/explained.\nResponse-14: Both were defined in the revised manuscript.\nComment-15: [lines 44 & 243] Annotation confidence is mentioned throughout the\nmanuscript but not explained.\nResponse-15: Annotation confidence level was first published by the Metabolomics\nStandards Initiative (MSI) and then refined by Schymanski, et al., in 2014 (Environ.\nSci. Technol., 2014, 48, 2097–2098) to align the rigor of small molecule identifications\nin chemical practices. Annotation confidence level 1 represents the confirmed structure\nvia appropriate measurement of a chemical standard with MS, MS/MS and retention\ntime matching. Level 2 is probable structure indicates that it was possible to propose an\nexact structure with different evidences, including unambiguous MS/MS spectra\nmatching and additional evidences like retention behaviors and diagnostic characters.\nLevel 3 is tentative annotation, which is defined as evidence exists for possible\nstructure(s) but insufficient information for one exact structure only, usually assigned\nby MS/MS matching with reference spectra in libraries. Level 4 is unequivocal\nmolecular formula, assigned by exact mass and signature MS, such as adduct and\nisotope. Level 5 is exact mass measured in a sample without a formula.\nRevision-15: Explanation of annotation confidence level 3 was explained in the\nMethods of the revised manuscript. The reference was added as well.\n“Molecule annotation at confidence level 3 was achieved by matching exact mass and\nMS/MS spectra in DnsBank. The annotation confidence level was complied with small\nmolecule identifications defined by Schymanski, et al., in 201414.”\nComment-16: [lines 68-69] \"Current predicting tools are not good at predicting spectra\nfor CDMs ...\" I would suggest rephrasing this and stating that they are not\ndesigned/developed specifically for CDMs.\nResponse-16: The sentence was rephrased as suggested.\nComment-17: [line 80] \"DeepCDM retrained the multilayer perceptron (MLP) of\nNEIMS\" The architecture of NEIMS has not been discussed earlier in the manuscript.\nResponse-17: The architecture of NEIMS was added in section S1.2 of the revised SI.\nRevision-17: Revision in SI:\n“1.2 NEIMS architecture and the input & output of the model\nNEIMS uses multilayer perceptron architecture with residual network connections\nbetween the layers to predict spectra from an input molecule, mapped to Extended-\nConnectivity Fingerprints (ECFPs). The output of NEIMS is a vector that represents\nthe intensity at every integral m/z bin. The input and output of Dns-MS and Bz-MS are\nthe same with NEIMS.”\nComment-18: [line 298] I think the NEIMS reference is wrong\nResponse-18: Corrected as suggested.\nComment-19: Missing references for SIRIUS 4 and mzCloud and Compounds\nDiscoverer.\nResponse-19: References for SIRIUS 4 and mzCloud and Compound Discoverer were\nadded in the revised manuscript.\nComment-20: Annotation efficiency has not been defined.\nResponse-20: Annotation efficiency was the percentage of query spectra which were\nconfidently assigned to molecules by spectra matching or database searching. The\ncalculating equation of annotation efficiency was provided in Methods.\nRevision-20: Annotation efficiency was defined in the Methods of revised manuscript:\n“Annotation efficiency was calculated to evaluate the percentage of query spectra\nconfidently assigned to molecules by spectra matching or database searching.\n…….…(4)22”\n𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑚𝑜𝑙𝑒𝑐𝑢𝑙𝑒 𝑎𝑛𝑛𝑜𝑡𝑎𝑡𝑖𝑜𝑛𝑠\n𝐴𝑛𝑛𝑜𝑡𝑎𝑡𝑖𝑜𝑛 𝐸𝑓𝑓𝑖𝑐𝑖𝑒𝑛𝑐𝑦 = 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑞𝑢𝑒𝑟𝑦 𝑠𝑝𝑒𝑐𝑡𝑟𝑎\nComment-21: The README file is not sufficient. There is no explanation on the\ncontent of the repository and how to run the code.\nResponse-21: The updated README file was available on Github\n(http://github.com/ADNLab-SCU/DeepCDMs).\nComment-22: Does the folder named \"Ref Codes\" contain the code from NEIMS\nadjusted for DeepCDM? If that's the case, then the README within that folder should\nbe modified accordingly. At the moment it appears as it is the code of the NEIMS paper.\nThe README file should indicate clearly the source of the original code (as it already\nthe case) but it should be adjusted for the DeepCDM model.\nResponse-22: We updated the README file with more the explanation of repository\nand codes. It’s now available on Github (http://github.com/ADNLab-SCU/DeepCDMs)\nComment-23: Will the trained model be made available?\nResponse-23: The trained model would be available upon request after publication."}, {"round": 2, "reviewer_comments": {"reviewer_1": "Dear Authors,\nI have carefully read again the manuscript entitled \"Deep Learning Enables High-Quality\nMS/MS Spectrum Prediction for Chemically Derived Molecules\" submitted to Nature\nCommunications.\nThe aim of this manuscript was to overcome the limitations related to the lack of large\nreference spectra sets for chemically derivatized molecules (CDMs), which hinder the\ndeployment of chemical derivatization-based NTA.\nBased on my review, I am satisfied with the authors response to the questions that I raised in the\nfirst revision round.", "reviewer_2": "The authors have made a strong effort to address my concerns, I appreciate all their efforts to add\nclarity to the metrics and discussion of the model tuning. I do feel that the explanation of why the\nauthors chose to train on MoNA data instead of NIST is important and would add to the larger\ndiscussion of ML prediction of spectra if it was included in the final version of the paper.\nMinor note: In line 52, the acronym WCS is used without initially defining it.", "reviewer_3": "The authors have addressed all the reviewers' questions and they have carefully considered all\nsuggestions for improving the manuscript. They have improved clarity of the manuscript and\nenhanced the experiments and performance analysis. However, after reviewing the authors'\nresponses to the reviewers' questions, as well as the revised manuscript, there are a few additional\ntopics to be clarified or improved before the final publication of the manuscript.\nMore specifically:\n[line 52] WCS has not been defined up to this point.\n[line 114 and SI Figure S2] how is structural similarity inferred? Is it tanimoto similarity on Morgan\nfingerprints? Please add this information in the manuscript.\n[lines 50-55] I am wondering if the spectra from MoNA, MassBank and HMDB have the same\nconditions (collision energy, ionization) with the spectra of the dansylated forms and whether this\ncomparison is accurate.\n[lines 74-75] \"... Dns-MS, which outperforms NEIMS and CFM-ID in the compound-to-spectrum\n(C2MS) benchmark of dansylated molecules...\" I would suggest using the phrase \"improves upon\ngeneric tools\" instead of \"outperforms\" or even \"outperforms generic tools...\" given that NEIMS and\nCFM-ID are not designed for dansylated molecules and therefore the comparison is not fair.\n[line 113] \"62 classes of molecules\" what classes? can you name a few?\n[lines 107-108] \"The hidden layers of ESI-MLP were optimized from 7 layers of 2000 nodes to 3\nlayers of 500 nodes (Table S1)\" It is not clear what you mean here. Do you mean that you did hyper-\nparameter tuning to select the optimal number of layers and nodes? I suggest rephrasing this\nsentence or removing it because it seems that it is repeated in line 516.\nA follow-up question on the fine-tuning hyper-parameters: Were the training hyper-parameters\n(learning rate, number of epochs) kept the same as in the original NEIMS implementation? Is it only\nthe hidden layer size that is different with respect to NEIMS? I would expect that the training hyper-\nparameters would be different for the purpose of fine-tuning the model.\n[lines 137-140] This additional experiment was suggested to the authors in order to get an\nunderstanding of the difficulty of the problem. The idea was to use a very simple model as a baseline\nand see how it performs on this dataset. The model that was suggested was one that would simply\nreturn the spectrum of the closest structurally molecule in the training set. I do not think that this\nexperiment is good for \"evaluating the capability of Dns-MS in distinguishing molecules with similar\nstructures\".\n[line 208] Please make sure you are giving a reference for the databases and tools you are\nmentioning in the manuscript. For example DSSTox database does not have a reference.\n[lines 409, 411]: Instead of using the same name \"similarity\" in both equations, you could name the\nfirst one as weighted cosine similarity and the second as cosine. Given that the two equations are\nvery similar, this would help the reader.\n[line 504] Please add reference for Prism.\nFinally, the structure of the Git repository is still confusing to me. The folder Ref Codes contains the\nread-me file of the original NEIMS implementation but this is the same folder where you have the\nmodified files of your own model. I suggest decoupling the two things, that is, having one folder with\nthe original implementation and read-me file and a separate folder with your modifications / python\nfiles that are specific to your model."}, "author_response": "We thank all reviewers for the insightful comments and suggestions. We have\nrevised our manuscript accordingly. A point-by-point response is provided\nas following.\nReviewer #1:\nDear Authors,\nI have carefully read again the manuscript entitled \"Deep Learning Enables High-\nQuality MS/MS Spectrum Prediction for Chemically Derived Molecules\" submitted to\nNature Communications.\nThe aim of this manuscript was to overcome the limitations related to the lack of large\nreference spectra sets for chemically derivatized molecules (CDMs), which hinder the\ndeployment of chemical derivatization-based NTA.\nBased on my review, I am satisfied with the authors response to the questions that I\nraised in the first revision round.\nResponse: Thank you for the insightful comments and suggestions to help us\nimproving our work.\nReviewer #2:\nComment-1: The authors have made a strong effort to address my concerns, I\nappreciate all their efforts to add clarity to the metrics and discussion of the model\ntuning. I do feel that the explanation of why the authors chose to train on MoNA data\ninstead of NIST is important and would add to the larger discussion of ML prediction\nof spectra if it was included in the final version of the paper.\nResponse-1: A short discussion about using MoNA dataset in DeepCDM was added to\nMethods of the revised manuscript.\nRevision-1: Revision in manuscript:\n“In this step, MoNA dataset was chosen because it is an open database accessible to\nresearchers across diverse research fields.”\nComment-2: Minor note: In line 52, the acronym WCS is used without initially\ndefining it.\nResponse-2: The acronym WCS was defined in line 52.\nReviewer #3:\nThe authors have addressed all the reviewers' questions and they have carefully\nconsidered all suggestions for improving the manuscript. They have improved clarity\nof the manuscript and enhanced the experiments and performance analysis. However,\nafter reviewing the authors' responses to the reviewers' questions, as well as the revised\nmanuscript, there are a few additional topics to be clarified or improved before the final\npublication of the manuscript.\nMore specifically:\nComment-1: [line 52] WCS has not been defined up to this point.\nResponse-1: The acronym WCS was defined in line 52.\nComment-2: [line 114 and SI Figure S2] how is structural similarity inferred? Is it\ntanimoto similarity on Morgan fingerprints? Please add this information in the\nmanuscript.\nResponse-2: The pairwise structural similarity of molecules in the training set were\ncalculated using Tanimoto similarity on the Extended-Connectivity Fingerprints\n(ECFP4). The information was added in the revised manuscript and in Figure S2 as\nsuggested.\nRevision-2: Information about structural similarity was added in the revised manuscript\nand the Supporting Information.\nRevision in manuscript:\n“The molecules had an average pairwise Tanimoto similarity of 0.11 on their Extended-\nConnectivity Fingerprints (ECFP4)”\nRevision in SI:\nFigure S2: (b) …“and calculated using Tanimoto similarity on molecular Extended-\nConnectivity Fingerprints (ECFP4).”\nComment-3: [lines 50-55] I am wondering if the spectra from MoNA, MassBank and\nHMDB have the same conditions (collision energy, ionization) with the spectra of the\ndansylated forms and whether this comparison is accurate.\nResponse-3: The reference spectra of 231 underived molecules from MoNA,\nMassBank and HMDB matched with experimental spectra of their dansylated forms\nwere manually selected to have same ionization method: electrospray ionization.\nSpectra of one molecule at different collision energies in these databases were also\nmerged to generate collision-energy-merged spectra as described in Method, to ensure\nconsistent spectral conditions in matching.\nRevision-3: The information of reference spectra of underived molecules was added to\nFigure S1.\nRevision in SI:\nFigure S1: (a)…“To ensure consistent spectra acquiring conditions in matching,\nelectrospray ionization tandem mass spectra (ESI-MS/MS) of original molecules were\nselected and spectra of one molecule collected at different collision energies were\nmerged.”\nComment-4: [lines 74-75] \"... Dns-MS, which outperforms NEIMS and CFM-ID in\nthe compound-to-spectrum (C2MS) benchmark of dansylated molecules...\" I would\nsuggest using the phrase \"improves upon generic tools\" instead of \"outperforms\" or\neven \"outperforms generic tools...\" given that NEIMS and CFM-ID are not designed\nfor dansylated molecules and therefore the comparison is not fair.\nResponse-4: The sentence was rephrased in the revised manuscript as suggested.\nComment-5: [line 113] \"62 classes of molecules\" what classes? can you name a few?\nResponse-5: The molecules were classified according to their structural features using\nClassyFire (J. Cheminf. 2016, 8, 61). The 62 classes of molecules included benzene and\nsubstituted derivatives, carboxylic acids and derivatives, phenols, organooxygen\ncompounds, fatty acyls, and so on. The information of molecule classification was\nadded to the revised manuscript.\nRevision-5: Examples of molecular classes were added in the revised manuscript.\nRevision in manuscript:\n“62 classes of molecules such as benzene and substituted derivatives, carboxylic acids\nand derivatives, and phenols.”\n“Molecule classification based on structural features was performed using ClassyFire51.”\nComment-6: [lines 107-108] \"The hidden layers of ESI-MLP were optimized from 7\nlayers of 2000 nodes to 3 layers of 500 nodes (Table S1)\" It is not clear what you mean\nhere. Do you mean that you did hyper-parameter tuning to select the optimal number\nof layers and nodes? I suggest rephrasing this sentence or removing it because it seems\nthat it is repeated in line 516.\nResponse-6: The sentence was removed as suggested.\nComment-7: A follow-up question on the fine-tuning hyper-parameters: Were the\ntraining hyper-parameters (learning rate, number of epochs) kept the same as in the\noriginal NEIMS implementation? Is it only the hidden layer size that is different with\nrespect to NEIMS? I would expect that the training hyper-parameters would be different\nfor the purpose of fine-tuning the model.\nResponse-7: Some of the training hyperparameters were modified from the original\nNEIMS implementation, such as number of epochs, hidden layer size and so on. Details\nof hyperparameters used in DeepCDM was summarized in Table S2.\nRevision-7: Table S2 was added to the Supporting Information.\nTable S2. Training hyperparameters in DeepCDM.\nHyperparameter Setting\nMLP # Hidden Layers 3\nMLP # Hidden Nodes 500\nActivation Function ReLU\n# Fingerprint Length 4096\n# Epochs 1000\nLearning Rate 0.001\nComment-8: [lines 137-140] This additional experiment was suggested to the authors\nin order to get an understanding of the difficulty of the problem. The idea was to use a\nvery simple model as a baseline and see how it performs on this dataset. The model that\nwas suggested was one that would simply return the spectrum of the closest structurally\nmolecule in the training set. I do not think that this experiment is good for \"evaluating\nthe capability of Dns-MS in distinguishing molecules with similar structures\".\nResponse-8: Thank you for the clarification. In the suggested experiment, we found\nthe experimental spectra of molecules in the test set were low in spectral similarity with\nthe spectra of their structurally closest molecules in the training set, having a median\nWCS of 0.22 ± 0.03 (95% CI, n = 167). The predicted spectra of the 167 molecules in\ntest set using Dns-MS reached a median WCS of 0.27 ± 0.02 when matching with the\nspectra of the structurally closest molecules in the training set. But the Dns-MS\npredicted spectra of molecules in the test set achieved greatly improved WCSs (median\nWCS = 0.69 ± 0.02) in matching with their own experimental spectra, showing the Dns-\nMS had good predictability for a different group of molecules which hadn’t been\nexposed to the model. We agree this experiment wouldn’t be able to evaluate the\ncapability of Dns-MS in distinguishing molecules with similar structure, therefore we\nremoved the statement in the revised manuscript.\nComment-9: [line 208] Please make sure you are giving a reference for the databases\nand tools you are mentioning in the manuscript. For example DSSTox database does\nnot have a reference.\nResponse-9: Reference was added to DSSTox database in the revised manuscript.\nComment-10: [lines 409, 411]: Instead of using the same name \"similarity\" in both\nequations, you could name the first one as weighted cosine similarity and the second as\ncosine. Given that the two equations are very similar, this would help the reader.\nResponse-10: The names of the two equations were changed as suggested.\nComment-11: [line 504] Please add reference for Prism.\nResponse-11: Reference for Prism was added in the revised manuscript.\nComment-12: Finally, the structure of the Git repository is still confusing to me. The\nfolder Ref Codes contains the read-me file of the original NEIMS implementation but\nthis is the same folder where you have the modified files of your own model. I suggest\ndecoupling the two things, that is, having one folder with the original implementation\nand read-me file and a separate folder with your modifications / python files that are\nspecific to your model.\nResponse-12: The Ref Codes folder in our Github repository was decoupled into\noriginal NEIMS and modifications."}, {"round": 3, "reviewer_comments": {"reviewer_1": "Dear Authors,\nI have carefully read again the manuscript entitled \"Deep Learning Enables High-Quality\nMS/MS Spectrum Prediction for Chemically Derived Molecules\" submitted to Nature\nCommunications.\nThe aim of this manuscript was to overcome the limitations related to the lack of large\nreference spectra sets for chemically derivatized molecules (CDMs), which hinder the\ndeployment of chemical derivatization-based NTA.\nBased on my review, I am satisfied with the authors response to the questions that I raised in the\nfirst revision round.", "reviewer_2": "My concerns have been addressed.", "reviewer_3": "The authors have addressed all my questions, improved clarity of the manuscript and improved the\nstructure of the code repository.\nOne additional suggestion to the authors:\nLines 137-146 under the Results section: I believe that from the experiment you are describing in\nthis part of the manuscript, we cannot draw any conclusions on the robustness of the model. I think\nthe connection to model robustness is not valid and I would encourage the authors to carefully\nrephrase this paragraph."}, "author_response": "We thank all reviewers for the insightful comments and suggestions. We have\nrevised our manuscript accordingly. A point-by-point response is provided\nas following.\nReviewer #2:\nMy concerns have been addressed.\nResponse: Thank you for the insightful comments and suggestions to our work.\nReviewer #3:\nThe authors have addressed all my questions, improved clarity of the manuscript and\nimproved the structure of the code repository.\nOne additional suggestion to the authors:\nLines 137-146 under the Results section: I believe that from the experiment you are\ndescribing in this part of the manuscript, we cannot draw any conclusions on the\nrobustness of the model. I think the connection to model robustness is not valid and I\nwould encourage the authors to carefully rephrase this paragraph.\nResponse: The suggested experiment described in lines 137-146 compared the\npredicted and experimental spectra of dansylated molecules in the test set with the\nexperimental spectra of their structurally most similar molecules in the training set.\nBoth the Dns-MS predicted and experimental spectra of molecules in the test set had\nlow similarities with the experimental spectra of their structurally closest molecules in\nthe training set, but the predicted and experimental spectra of query molecules in the\ntest set had good spectral similarity, that showed Dns-MS had good predictability for a\ndifferent group of molecules which hadn’t been exposed to the model. Therefore, we\nremoved the sentence connecting this experiment and model robustness as suggested\nby the reviewer.\nRevision in manuscript:\n“That confirmed the good predictability of Dns-MS for dansylated molecules that\nhaven’t been exposed to the model.”"}]

Decision letter