MolSpectra: Pre-training 3D Molecular Representation with Multi-modal Energy Spectra
Authors: Liang Wang
Shaozhen Liu
Yu Rong
Deli Zhao
Qiang Liu
Shu Wu
Liang Wang
Venue: ICLR 2025 Poster (2025)
Keywords: 3D molecular representation learning
molecular spectra
pre-training
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: We propose incorporating molecular spectra into the pre-training of 3D molecular representations, thereby infusing the knowledge of quantum mechanical principles into the representations.

Abstract: Establishing the relationship between 3D structures and the energy states of molecular systems has proven to be a promising approach for learning 3D molecular representations. However, existing methods are limited to modeling the molecular energy states from classical mechanics. This limitation results in a significant oversight of quantum mechanical effects, such as quantized (discrete) energy level structures, which offer a more accurate estimation of molecular energy and can be experimentally measured through energy spectra. In this paper, we propose to utilize the energy spectra to enhance the pre-training of 3D molecular representations (MolSpectra), thereby infusing the knowledge of quantum mechanics into the molecular representations. Specifically, we propose SpecFormer, a multi-spectrum encoder for encoding molecular spectra via masked patch reconstruction. By further aligning outputs from the 3D encoder and spectrum encoder using a contrastive objective, we enhance the 3D encoder's understanding of molecules. Evaluations on public benchmarks reveal that our pre-trained representations surpass existing methods in predicting molecular properties and modeling dynamics.

Published as a conference paper at ICLR 2025
MOLSPECTRA: PRE-TRAINING 3D MOLECULAR REP- RESENTATION WITH MULTI-MODAL ENERGY SPECTRA
Liang Wang! Shaozhen
Shaozeen Liu' Yu Rong"1 Deli Zhao² Qiang Liu Lit Wu! Liang Wang NNw Laboratory of Pattern Recognition (NLPR), State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS),
Institute of Automation, Chinese Academy of Sciences (CASIA)
2School of Artificial Intelligence, University of Chinese Academy of Sciences 3 DDMO Academy, Alibaba Group
ABSTRACT
Establishing the relationship between 3D structures and the energy states of molecular systems has proven to promising approach for learning 3D molecu- lar representations. However, existing methods are limited to modeling the molec- ular energy states from classical mechanics. This limitation results in a significant oversight of quantum mechanical effects, such as quantized (discrete) energy level structures, which offer a more accurate estimation of molecular energy and can be experimentally measured through energy spectra. In this paper, propose to utilize the energy spectra to enhance the pre-training of 3D molecular repre- sentations (MolSpectra), therehy infusing the knowledge of quantum mechanics into the molecular representations. Specifically, we propose SpecFormer, multi- spectrum encoder for encoding molecular spectra via masked patch reconstruc- tion. By further aligning outputs from the 3D encoder and spectrum encoder using a contrastive objective, we enhance the 3D encoder's 's understanding of molecules. Evaluations on public benchmarks reveal that our pre- pre-trained representations sur- pass existing methods in predicting molecular properties and modeling dynamies.
INTRODUCTION
Learning 3D molecular representations from geometric conformations offers a promising approach for understanding molecular geometry and predicting quantum properties and interactions, which significant in drug discovery and materials science (Musaelian et al., 2023; Batatia et al., 2022; Liao & Smidt, 2023; Wang et al., 2023b; Du et al., 2023b). Given the scarcity of molecular prop- erty labels, self-supervised representation pre-training has been proposed and utilized to provide generalizable representations (Hu et al., 2020; Rong al., 2020; Ma et al., 2024).
In contrast to contrastive learning (Wang et al., 2022; Kim ct al., 2022) and masked modeling (Hou et al., 2022; Liu et al., 2023c; Wang et al., 2024b) 2D molecular graphs and molecular lan- guages (e.g.. SMILES), the design of pre-training strategies on 3D molecular geometries is more closely aligned with physical principles. Previous studies (Zaidi et al., 2023; Jiao al., 2023) have guided representation learning through denoising processes on 3D molecular geometries, theoreti- cally demonstrating that denoising 3D geometries is equivalent to learning molecular force fields, specifically the negative gradient of molecular potential energy with respect to position. Essentially, these studies reveal that establishing the relationship between 3D geometries and the energy states of molecular systems is an effective pathway to learn 3D molecular representations.
However, existing methods are limited to the continuous description (i.e., the potential energy func- tion) of the molecular energy states within the classical mechanics, overlooking the quantized (dis- crete) energy level structures from the quantum mechanical perspective. From the quantum per- spective, molecular systems exhibit quantized energy level structures, meaning that energy states
" "Correspondence Liang Wang: 1iang. wang@cripac.
1 'Corresonding authors: Yu Rong and Qiang Liu
'The code is released at https: / / thub. com Azurel eon /Mo Spectra

---

Published as a conference paper at ICLR 2025
Principle
Knowledge
Data
Encoding
Pre-training Objective
Add Noise
3D Encoder
Denoising
3D Representation
(Continuous)
Classical Mechanics
Potential Energy Function
Conformation
Contrastive
Add Mask
Spectra
Masked Patch Reconstruction
Encoder
Spectra
Quantum Mechanics
(Quantized) Energy Level Strueture
Spoctra
Representation
The conceptual view MolSpectra, which leverages both molecular conformation and spectra for pre-training. Prior works only model classical mechanies by denoising conformations.
Figure I:
only assume specific discrete values. Specifically, different types of molecular motion, such as electronic, vibrational, and rotational motion, correspond to different energy level structures. Knowledge of these energy levels is crucial in molecular physics and quantum chemistry, they determine the spectroscopic characteristics, chemical reactivity, and many other important molec- ular properties. Fortunately, experimental measurements of molecular energy spectra can reflect these structures. Meanwhile, there are many molecular spectra data obtained through experimen- tal measurements or simulations (Zou et al., 2023; Alberts et al., 2024). Therefore, incorporating the knowledge of energy levels into molecular representation learning is expected to facilitate the development of more informative molecular representations.
In this paper, we propose MolSpectra, framework that incorporates molecular spectra into the pre-training of 3D molecular representations, thereby infusing the knowledge of quantized energy level structures into the representations, shown in Figure 1. In MolSpectra, we introduce a multi- spectrum encoder, SpecFormer, to capture both intra-spectrum and inter-spectrum peak correlations by training with a masked patches reconstruction (MPR) objective. Additionally, we employ contrastive objective to distills the spectral features and its inherent knowledge into the learning of 3D representations. After pre-training, the resulting 3D encoder can be fine-tuned for downstream tasks, providing expressive 3D molecular representations without the need for associated spectral data. Extensive experiments over different downstream molecular property prediction benchmarks shows the superiority of MolSpectra.
In summary, contributions are follows:
We introduce quantized energy level structures and molecular spectra into 3D molecular repre- sentation pre-training for the first time, surpassing previous work that relied solely on physical knowledge within the scope of classical mechanics.
We propose SpecFormer as expressive multi-spectrum encoder, along with the masked patches reconstruction objective for spectral representation learning.
We propose a contrastive objective to align molecular representations in the 3D modality and spectral modalities, enabling the pre-trained 3D encoder to infer molecular spectral features in downstream tasks without relying on spectral data.
Experiments across different downstream benchmarks demonstrate that our method effectively enhances the expressiveness of the pre-trained 3D molecular representations.
PRELIMINARIES
2.1 NOTATIONS
Consider molecule characterized by its 3D structure and spectra, represented as M (a. S). Here, a E (1,2...., 118} specifies the atomic numbers, indicating the types atoms within the molecule. The vector R3N describes the conformation of the molecule, while S represents its
2

---

Published as a conference paper at ICLR 2025
spectra. The parameter N denotes the number of atoms in the molecule. Note that the atoms arranged in the same order in both a and x, ensuring consistency between the atomic numbers and their corresponding spatial coordinates.
= (......,, 8|s)) represents the set of spectra for a molecule, where S denotes the number of spectrum types considered. our study, we focus on three types, so |S] = 3. The first spectrum, E |R is the UV-Vis spectrum, which spans from 1.5 to 13.5 eV with 601 data points at intervals of 0.02 eV. The second spectrum, $2 E R3501, is the IR spectrum, covering a range from 500 to 4000 em with 3501 data points at intervals of 1 cm The third spectrum, $3 E RR3501, is the Raman spectrum, with the same range and intervals as the IR spectrum. Together, these spectra provide a comprehensive description of the molecular characteristics across different spectral modalities.
2.2 PRE-TRAINING 3D MOLECULAR REPRESENTATION VIA DENOISING
Denoising has emerged as a prominent pre-training objective in 3D molecular representation learn- ing, excelling various downstream tasks. This method involves training models to predict and remove noise introduced deliberately into molecular structures. This approach is physically inter- pretable due its proven equivalence to learning the molecular force field.
Equivalence between denoising and learning molecular force fields. The equivalence between coordinate denoising and force field learning is established by Zaidi al. (2023). For given molecule M. perturb its equilibrium structure æn according to the distribution p(z](z)), where æ the noisy conformation. Assuming the molecular distribution adheres to the energy-based Boltz- mann distribution with respect to the energy function E(-), then
Dennissing (MM (((2) ( - (a: 2=))l
(1)
p ( (2) - (- V ((=))))
where GNNo(2) denotes a graph neural network parameterized by 6 which processes the confor- mation to produce node-level predictions. The notation signifies the equivalence of different objectives. The proof of this equivalence is provided in the Appendix A. In prior research, the energy function E(- has been defined in several forms. Below three representative studies.
Energy function I: mixture of isotropic Gaussians. In Coord (Zaidi et al., 2023), the energy function is approximated using a mixture of isotropic Gaussians centered at the known equilibrium structures to replace the Boltzmann distribution, since these structures are local maxima of the Boltz- man distribution. Leveraging the equivalence between the ecore-matching objective and denoising autoencoders (Vincent, 2011), the following denoising bbased energy function Ecuon(.)) is derived:
Eccord (x) =
(x
xo)
(xx -xp)-
2r2
Note that this objective is derived under the assumption of isotropic Gaussian noise, i.e., p(x: xo) ~ N(x0, J, where JaN represents the identity matrix of size 3N, and the subseript indicates the coordinate denoising approach.
Energy function II: mixture of anisotropic Gaussians. Considering rigid and flexible com- ponents in molecular structures, isotropic Gaussian can lead to significant approximation errors. To address the anisotropic distribution, Frad (Feng et al., 2023) introduces hybrid noise on di- hedral angles of rotatable bonds and atomic coordinates. incorporating fractional denoising of the coordinate noise. The equilibrium structure œp initially perturbed by dihedral angle noise NWon Im), followed by coordinate noise p(z(zx) ~ NNxx,, IaN). Here, Ba sho E [0, 2Tm represent to the dihedral angles of rotatable bonds in structures æ. and vo spectively, with m denoting the number of rotatable bonds. The subscript f indicates the fractional denoising approach. Subsequently, the energy function is induced:
Fvrat (x) R (2x x0) T
Tyjay (x 2o),
(3)
2
where E.y.ay = 7 LaN + C.C. and C xaNxm matrix used to linearly transform the dihedral angle noise into coordinate change, expressed as Ar = CAsb.
Energy function III classical potential energy theory. SliDe (Ni et al., 2024) derives energy function from classical molecular potential energy theory (Alavi, 2020; Zhou & Liu, 2022). In this
3

---

Published as a conference paper at ICLR 2025
Denosing
Conntaatt
MPR
3D Structure View
Spectra View
Molecular 3D
Molecular Spectra
Reconstructed
Predicted
Representation
Representation
Macked Patches
Noise
Flation Representastinn Head
Reconstruction Heads
Add & Numo
Demoising Head
Representation Head
Feed Forward
SpecFormer (Single-stin
Spectrum Encoder)
Ads Norm
+
j...
3D Encoder
Pawnkaagy Pootaker
PacchEasaw indiingg
Pradnny
Add Mask
Add Mask
Add Mask
Add Noise
Patchinguv
Patching
Patching
UV-Vis
IR
Raman
Figure 2: Overview of the MolSpectra pre-training framework. Our pre-training framework com- prises three sub-objectives: the denoising objective and the MPR objective, which respectively guide the representation learning of the 3D and spectral modalities, and the contrastive objective, which aligns the representations of both modalities.
form, the total intramolecular potential energy is mainly attributed to three types of interactions: bond stretching, bond angle bending, and bond torsion. The following energy function derived:
EsuDe 0,
JkB o (r ro)l
(r (r)]
ro) A o (0 0o)) 0o)
k (Φ Φo)1 ( go),
where (R>0))=11 E [0, (z, " 10, 2n) m represent vectors of the bond lengths, bond
angles, and bond torsion angles of the molecule, respectively. ro, Og. Φ correspond to the respective equilibrium values. The parameter vectors B kA, k² determine the interaction strength.
THE PROPOSED MOLSPECTRA METHOD
Considering the complementarity of different spectra, introduce multiple spectra into molecular representation learning. To effectively comprehend molecular spectra, we designed a Transformer- based multi-spectrum encoder, SpecFormer, along with masked reconstruction objective guide its training. Finally, a contrastive objective is employed to align the 3D encoding guided by the denoising objective with the spectra encoding guided by the reconstruction objective, endowing the 3D encoding with the capability to understand spectra and the knowledge they encompass.
3.1 SPECFORMER: A MINGLE-STREAL ENCODER FOR MULTI-MODAL ENERGY SPECTRA
For different types of spectra, each spectrum is independently patched and initially encoded. Then, all the resulting patch embeddings are concatenated and encoded using Transformer-basee encoder.
Patching. Compared to directly encoding individual frequency points, we divided each spectrum into multiple patches. This approach offers two distinct advantages: (i) By forming patches from adjacent frequency points, local semantic features, such as absorption peaks, can captured more effectively. (ii) It reduces the computational overhead of subsequent Transformer layers. Techni- cally, each spectrum 8; E Rh where = 1.... .|S] is first divided into patches according to the patch length P; and the stride D,. When 0 < D. < P., the consecutive patches will be overlapped

---

Published as a conference paper at ICLR 2025
with overlapping region length P. - D,. When D, Ps. the consecutive patches will be non- overlapped. L, denotes the length of 8,. The patching process on each spectrum will generate a the number of patches.
sequence patches p. E RNNxP, where N= L
Patch encoding and position encoding, Prior to be fed into the encoder, the patches of the i-th spectrum are mapped to the latent space of dimension d via a trainable linear projection W; E RAxdd A learnable additive position encoding Wi E RRNxxd is applied to maintain the order of the patches: p/ P: P.W: + W where P E TRN, denotes the latent representation of the spectrum 8; that will be fed into the subsequent SpecFormer encoder.
multi-spectrum Transformer encoder. Although several encoders have been pro- posed to map molecular spectrum into implicit representations, such as the CNN-AM (Tao et al., 2024) based on one-dimensional convolution, these encoders are designed to encode only sin- gle type of spectrum. In our approach, multiple molecular spectra (UV-Vis, IR, Raman) are jointly considered. When encoding multiple spectra of molecule simultancously, tion caught our attention and led us to adopt a Transformer- encoder with multiple spec- tra input, similar the single-stream Transformer in multi-modal learning (Shin et al., 2021). The observation refers to the fact that the same functional group not only causes multiple peaks within a single spectrum, but also generates peaks across different spectra. As shown on the left of Figure 3, the different vi-
SpecFormer:
observa-
Intra-spectrum Dependencies
Inter-spectrum Dependencies
Imaradannn Co.
strutching
vibéation
chanetiii vilggttoon
Trammiitani
TR. Sprectrom
brational modes of the methyl group
i-- ansicion
(-CHa) in methanol (CH; OH) result in three peaks in the IR spectrum, indicating intra-spectrum dependen- cies among these peaks. A simi- lar phenomenon occurs with the hy- droxyl group (-OH) in methanol. Ad- ditionally, the aromatic ring in phe- nol OH), shown on the right of Figure 3, not only produces multiple peaks in the IR spectrum due to different vibrational modes but also causes an absorption peak near 270 nm in the UisVis spectrum due to the x -> x transition in the aromatic ring, demonstrating the existence of inter-spectrum dependencies. Such dependencies have been theoretically studied, for example, in the context of vibronic coupling (Kong al., 2021).
2001
3000
2001
2000
1861
199
539
Speetrum
Wavmumbbr (cm
UV-Vis Specikinn
T
Illustration of intra-spectrum (left) and inter- sppcctum (right) dependencies.
Figure 3:
To capture intra-spectrum and inter-spectrum dependencies. we concatenate the embeddings ob- tained from patch encoding and position encoding of different spectra: D = p' Il. (.ll...Ilpss)
|1z
(EE) x. and then input them into the Transformer encoder depicted in Figure 2. Then each head h I,..., H in multi-head attention will transform them into query matrices Qs pw? key matrices K,, pw, and value matrices V, = pWW where / w.K Rdxx2 and W g $ Afterward, scaled product is utilized to obtain the attention output O₁ E ( NJ T
QuKK Vdly
O Attention ( Kn, Vv.) Softmax
Vn.
(5)
The multi-head attention block also includes BatchNorm layers and feed forward network with residual connections as shown in Figure After combining the outputs of all heads. it generates the representation denoted as z R Nd)xd Finally, flatten layer with representation projection head used to obtain the molecular spectra representation 2, Rd.
3.2 MASKED PATCHES RECONSTRUCTION PRE- TRAINING FOR SPECTRA
Before distilling the spectra information into 3D molecular representation learning, we need first ensure that the spectrum encoder can effectively comprehend molecular spectra and generate spec- tral representations. Considering the success of masking modeling across various domains (Devlin et al., 2019; He et al., 2022; Hou et al., 2022; Xia et al., 2023; Wang et al., 2024b; Nie et al., 2023), propose masked patches reconstruction (MPR) objective to guide the training of SpecFormer.
5

---

Published as a conference paper at ICLR 2025
After the patching step, randomly select portion of patches according the mask ratio and replace them with zero vectors to implement the masking. Subsequently, the masked patches undergo patch encoding and position encoding. In this way, the semantics of the masked patches (the absorption intensity at specific wavelengths) are obscured during patch encoding, while the positional information is retained facilitate the reconstruction of the original semantics.
After encoding by SpecFormer, the encoded results corresponding to the masked patches are input into spectrum-specific reconstruction head reconstruct the original spectral values that were masked. The mean squared error (MSE) between the reconstruction results and the original masked spectra serves the loss function for the MPR task, guiding the training of SpecFormer:
[S]
LMPR
Σ E EP, |
-P:.j
Pi.j
(6)
where P. denotes the set of masked patches in the i-th type of molecular spectra, and p., denotes the reconstructed patch corresponding the masked patch pij.
3.3 CONTRASTIVE LEARNING BETWEEN 3D STRUCTURES AND SPECTRA
Under the guidance of the denoising objective for 3D representation learning and the MPR objec- tive for spectral representation learning, we further introduce contrastive objective align the representations across these two modalities. We treat the 3D representation representation 2, E Rdd of the same molecule positive samples, and negative samples otherwise. Subsequently, the consistency between positive samples and the discrepancy between negative sam- ples are maximized through the contrastive objective. Given the theoretical and empirical effective- ness, employ InfoNCE (van den Oord et al., 2018) as the contrastive objective:
E Ie and spectral
exp(fa (2, 2.)) Coontrast - pp(...... log exp(f.((.....))t5) (x.... + log 2x, exp(I.(E..2.))
(2....))
where 2/, 2 are randomly sampled 3D and spectra views regarding to the positive pair (zz, z.). fz 2.) and fa(2., Ro) are scoring functions for the two corresponding views, with flexible for- mulations. Here we adopt (= z,) fa(z,, 2x) (Zz, zo).
Note that the denoising objective can utilize any form from existing 3D molecular representation pre-training studies, enabling seamless integration of our method into these frameworks.
3.4 TWO-STAGE PRE- TRE-NIGIN PIPELINE
Previous pre-training efforts for 3D molecular representation have been conducted on unlabeled datasets using denoising objective. These datasets typically provide only equilibrium 3D structures without offering spectra for all molecules. To enhance the pre-training effect by incorporating spec- tra while leveraging denoising pre-training, we employ a two-stage pre-training approach. The first stage involves training on larger dataset (Nakata & Shimazaki, 2017) without spectra using only the denoising objective. Subsequently, the second stage involves training on dataset that includes spectra using the complete objective as follows:
L DDenoising Densisiing + SMPR + BContrust CContrust
(8)
where Dennising AMPR and BContrast denote the weights of each sub-objective.
EXPERIMENTS
To comprchensively evaluate the impact of molecular spectra molecular tasks, first verify the effectiveness of molecular spectra in the training-froms from--crrach method for the downstream task. Furthermore, we evaluate the effectiveness of our pre-training framework MolSpectra.
4.1 EFFECTIVENESS OF MOLECULAR SPECTRA IN TRAINING FROM SCRATCH
This pilot experiment aims to demonstrate the rationality for incorporating molecular spectra into pre-training. We introduce additional spectral features into train-from-scratee molecular property

---

Published as a conference paper at ICLR 2025
prediction model to observe the impact of spectral information prediction outcomes. We employ EGNN (Satorras et al., 2021), a representative 3D molecular encoder, equipped with an MLP-based prediction head as the baseline model. While EGNN encodes the 3D representations, the UV-Vis spectrum of each molecule provided by the QM9S (Zou et al., 2023) dataset is encoded into spectral representations by spectrum encoder. Before making predictions with the final MI.,, we concate- nate the spectral and 3D representations for prediction. The results are presented in Table 1.
Table Performance (MAE 4) when training from scratch on QM9 dataset.
Task Units
homo
lumo (mcV)
gap
R12
ZPVE
Ug
U
H
G
C
(D)
(ng)
(meV)
(mcV)
(xg)
(meV)
(neV)
(neV)
(meV)
(meV)
w/o spectra
0.029
0.071
29
25
48
0.106
1.55
11
12
12
12
0.031
spectra
0.027
0.049
28
0.084
1.45
11
10
10
0.030
We observe that by directly concatenating spectral representations, the performance of molecular property prediction can be effectively enhanced. This indicates that the information from molecular spectra beneficial for downstream molecular property prediction. Further incorporating moleeu- lar spectra into the pre-training phase of molecular representation has the potential to enhance the informativeness and generalization capability of the representations, thereby broadly improving the performance of downstream tasks.
4.2 EFFECTIVENESS OF MOLECULAR SPECTRA IN REPRESENTATION PRE-TRAINING
We conduct experiments to evaluate MolSpectra by first introducing spectral data into the pre- training of 3D representations, followed by evaluating the performance on downstream tasks. For a comprehensive comparison, two types of baselines are adopted: (1) training-fromm-cratt methods, including SchNet (Schütt al., 2017), EGNN, DimeNet (Klicpera et al., 2020b), DimeNet++ (Kliepera et al., 2020a), PaiNN (Schlit et al., 2021), SphereNet (Liu et al., 2021), and TorchMD-Net (Thölke & Fabritiis, 2022); and (2) pre-training methods, including Transformer- M (Luo et al., 2023), SE(3)-DDM (Liu et al., 2023b), 3D-EMGP (Jiao et al., 2023), and Coord.
MolSpectra can be seamlessly plugged into any existing denoising method. To evaluate the enhance- ment provided by our method compared to denoising alone, we select the representative coordinate denoising (Coord) as our denoising sub-objective. This method also serves as our primary baseline.
4.2.1 PRE-TRAINING DATASET.
As described in Section 3.4, we first perform denoising pre-training the PCQM4My2 (Nakata & Shimazaki, 2017) dataset, followed by second stage of pre-training on the QM9Spectra (QM9S) (Zou al., 2023) dataset, which includes multi-modal molecular energy spectra. In both stages, we adopt the denoising objective provided by Coord (Zaidi et al., 2023), as defined in Eq. 2.
The QM9S dataset comprises organic molecules from the QM9 (Ramakrishnan et al., 2014) dataset. The UV-Vis, IR, and Raman spectra of the molecules are calculated at the P3LYP/def-TZVP level of theory, through frequency analysis and time-dependent density functional theory (TD-DFT).
4.2.2 QM9
The QM9 dataset is a quantum chemistry dataset comprising over 134,000 small molecules, each consisting of up to 9 heavy atoms (C, N. O, F) and additional H atoms. This dataset provides an equilibrium geometric conformation for each molecule along with 12 property labels. The dataset is divided into training set of 1 110k molecules, a validation set of 10k molecules. and a test set con- taining the remaining over 10k molecules. Prediction errors are measured using the mean absolute error (MAE). The experimental results are presented in Table 2.
The 3D molecular representations pre-trained using our method are fine-tuned and used for predic- tion across various properties, achieving ofate-of-the-an performance in out 12 properties and outperforms Coord in 10 out of 12 properties. In conjunction with the observations in Section 4.1, the performance improvement can be attributed to our incorporation of an understanding of molec- ular spectra and the knowledge they entail into the 3D molecular representations.

---

Published as a conference paper at ICLR 2025
Table 2:
Performance (MAE.) on QM9 dataset. The compared methods are divided into two groups:
training from scratch and pre-training then fine-tuning. The best results are highlighted in bold.
μ4
homo
lumo
gap
R
ZPVE
Up
U
H
G
CN
(D)
(a))
(meV)
(meV)
(meV)
(a.)
(meV)
(meV)
(meV)
(meV)
(mee))
cal m.c.k
SchNet
0.033
0.235
41.0
34.0
63.0
0.070
1.70
14.00
19.00
14.00
14.00
0.033
EGNN
0.029
0.071
29.0
25.0
48.0
0.106
1.55
11.00
12.00
12.00
12.00
0.031
DimeNet++
0,030
0.044
24.6
19.5
32.6
0.330
1.21
6.32
6.28
6.53
7.56
0.023
PaiNN
0.012
0.045
27.6
20.4
45.7
0.070
1.28
5.85
5.83
5.98
7.35
0.024
Npt
0.025
0.045
22.8
18.9
31.1
0.270
1.12
6,26
6.36
6.33
7.78
0.022
TorchMD-Net
0.011
0.059
20.3
17.5
36.1
0.033
1.84
6.15
6.38
6.16
7.62
0.026
Translormer-M
0.037
0.041
17.5
16.2
27.4
0.075
1.18
9.37
9.41
9.39
9.63
0.022
SE(3)-DDM
0.015
0.046
23.5
19.5
40.2
0.122
1.31
6.92
7.09
7.65
0.024
3D-EMGP
0.020
0.057
18.2
37.1
0.092
1.38
8.60
8.70
9.30
0.026
Coord
0.016
0.052
17.7
14.7
31.8
0.450
1.71
6.57
6.11
6.45
6.91
0.020
MolSpectra
0.011
0.048
15.5
13.1
26.8
0.410
1.71
5.67
5.45
5.87
6.18
0.021
Table 3: Performance (MAEI) on MD17 force prediction (kcal/mol/ Á). The methods are divided
into two groups: training from scratch and pre-training then fine-tuning, The best results are in bold.
Malonal dehydc
Naphtha -lene
Salicy -lic Acid
Aspirin
Benzene
Ethanol
Toluene
Uracil
SphereNet
0.430
0.178
0.208
0.340
0.178
0.360
0.155
0.267
SchNet
1.350
0.310
0.390
0.660
0,580
0,850
0.570
0.560
DimeNet
0.499
0.187
0.230
0.383
0,215
0.374
0.216
0.301
PaiNN
0.338
0,224
0,319
0.077
0.195
0.094
0.139
TorchMD-Net
0.245
0.219
0.107
0.167
0.059
0.128
0.064
0.089
SE(3)-DDM*
0.453
0.166
0,288
0.129
0,266
0.122
0.183
Coord 0.211
0.169
0.096
0.139
0.053
0.109
0.058
0.074
MolSpectra
0.099
0.097
0.052
0.077
0.085
0.093
0.075
0.095
4.2.3 MD17
The MD17 dataset contains molecular dynamics trajectories for eight organic molecules, including aspirin, benzene, and ethanol. It offers 150k to nearly 1M conformations per molecule, with energy and force labels. Unlike QM9, MD17 emphasizes dynamic behavior in addition to static properties. We use standard limited data split: models train on Ik samples, validate 50, and test on the rest. Performance is evaluated using MAE, with results in Table 3.
Our approach also results in the expected performance improvement on MD17. MD17 is a dataset comprising a large number of non-equilibrium molecular structures and their corresponding force fields, which serves evaluate a model's understanding of molecular dynamics. However, previ- ous pre-training methods based solely on denoising have only learned force field patterns at static equilibrium states, failing adequately capture the dynamic evolution of molecular systems. In contrast, our MolSpectra learns the dynamic evolution of molecules by understanding energy level transition patterns, thereby outperforming denoising- based pre-training methods.
4.3 SENSITIVITY ANALYSIS OF PATCH LENGTH P,, STRIDE D,, AND MASK RATIO &
We conduct experiments to evaluate the impact of patch length P,, stride D,, and mask ratio a. Results are summarized in Table 4 and Table 5.
From Table 4, we observe that when consecutive patches have overlap (D; < P.), the performance of pre-training is superior compared to scenarios without overlap (D; = P.). Specifically, the per- formance optimal when the stride is half of the patch length. This is because appropriate overlap better preserve and capture local features, particularly the information at the patch boundaries. Additionally, we find that choosing an appropriate patch length further enhances performance. In experiments, the configuration of P; 20, D; = 10 yields the best results.
8

---

Published as a conference paper at ICLR 2025
Table 4: Sensitivity of patch length and stride.
Table 5: Sensitivity of mask ratio.
patch length
stride
overlap ratio
homo
lumo
gap
mask ratio
homo
lumo
gap
20
7,%%
15.9
13.7
28.0
0.05
15.7
13.4
29.7
50%
15.5
13.1
26.8
0.10
15.5
26.8
20
15
25%
16.1
13.6
28.1
0.15
15.7
13.5
28.0
20
20
0%
15.7
13.5
27.5
0.20
16.0
13.6
28.1
16
50%
16.0
13.4
27.6
0.25
16.3
13.5
28.0
30
15
50%
15.9
14.0
28.1
0.30
16.2
13.7
29.0
Regarding the mask ratio, = 0.10 I is preferable choice. A small mask ratio results in insufficient MPR optimization, hindering SpecFormer training- Conversely, a large mask ratio causes excessive spectral perturbation, degrading performance when aligning with the 3D representations with the contrastive objective. An appropriate mask ratio strikes a balance between these two aspects.
4.4 ABLATION STUDY
To rigorously demonstrate the contributions of masked patches reconstruction, the incorporation of molecular spectra, and each spectral modality, conducted ablation study them.
Ablation study of masked patches reconstruction.
Table 6: Ablation of optimization objectives.
We remove the MPR loss analyze the impact of masked patches reconstruction, referred to as "w/o MPR" in Table Removing the MPR objective leads to performance deterioration. This is consis- tent with the sensitivity analysis of the mask ratio in Section 4.3, as removing MPR is an extreme case where a This decline is due to the lack of ef- fective guidance in training SpecFormer. Using an undertrained SpecFormer for contrastive learning with 3D encoder outputs limits performance improvement.
homo
lumo
gap
MolSpectra w/o MPR w/o MPR, Contrast
15.5
13.1
26.8
16.4
14.1
29.7
17.5
14.4
31.2
Ablation study of molecular spectra. We retain only the denoising loss, removing both the MPR loss and contrastive loss, referred to as "w/o MPR, Contrast" in Table 6. The only difference between this variant and MolSpectra is the incorporation of molecular spectra into the pre-training. The "w/o MPR, Contrast" results are inferior to those of MolSpectra, highlighting that incorporating molecular spectra effectively enhances the quality and generalizability of molecular 3D representations.
Ablation study of each spectral modality, To evaluate the contributions of each spectral modal- ity to the performance, we conduct an ablation study for each modality. The results are presented in Table 7. It can be observed that each spectral modality contributes differently, with the UV-Vis spectrum having the smallest contribution and the IR spectrum the largest, likely due to the varying information content in each modality.
Table 7: Ablation of spectral modalities.
UV-Vis
IR
Raman
homo
lumo
gap
15.5
13.1
26.8
15.8
13.3
27.1
16.6
14.1
28.9
J
16.1
13.9
28.3
RELATED WORK
3D molecular pre-training. Molecular 2D structures are typically represented graphs and mod- eled using graph learning methods (Gilmer et al., 2017; Li et al., 2023; Jiang et al., 2024; Yuan et al., 2025). However, 3D molecular structures provide critical geometric information that es- sential for understanding physicochemical properties (Chen et al., 2023; 2024; Wang et al., 2024a; Sun et al., 2024; Han et al., 2024), which cannot be directly inferred from 2D graphs or SMILES representations (Gong et al., 2024). Designing effective strategies for pre-training 3D molecular representations remains challenging due the geometric symmetries inherent 3D structures and their strong connection to physical knowledge, such potential energy functions.

---

Published as a conference paper at ICLR 2025
Denoising the geometric structure has been demonstrated as effective strategy for 3D representa- tion pre-training (Liu et al., 2023b; Jiao et al., 2023; Kim et al., 2023; Zhou et al., 2023; Wang et al., 2025). Coordinate denoising (Coord) (Zaidi et al., 2023) first theoretically proves that the denois- ing objective is equivalent to learning the gradient of the potential energy with respect to positions, essentially the force field. Building this work, fractional denoising (Frad) (Feng al., 2023) in- troduces dihedral angle noise to optimize the sampling of low-energy structures. Further, SliDe (Ni al., 2024) incorporates more rigorous potential energy from classical mechanics. Another line of research simultaneously leverages both 2D and 3D structures for vre-training molecular repre- sentations, addressing the complementarity of the two modalities (Li al., 2022; Zhu et al., 2022; Liu et al., 2023a; Du et al., 2023a; Yu et al., 2024) the computational complexity of 3D structure determination (Liu al., 2022; Stärk et al., 2022; Wang al., 2023a).
Although these studies elucidate the relationship between molecular 3D structures and their energy states, they remain limited the description of molecular energy states within elassical mechanics, without considering the quantized energy level structures as described by quantum mechanics.
Molecular spectroscopy. Molecular spectroscopy studies interactions between molecules and electromagnetic radiation. Analyzing spectra provides valuable insights into molecular structure, composition, and dynamics (Lancaster et al., 2024). When encountering unknown substances, re- samples and compare the observed spectra with
searchers conduct spectroscopic measurements libraries for identification. To expand library coverage, machine learning methods are widely used predict molecules' spectra (Zou al., 2023; Wei al., 2018; Zong et al., 2024).
Some studies incorporate physical principles into spectra prediction models as inductive biases, including molecular dynamics simulations via equivariant message passing (Schüit al., 2021), fragmentation (Dühhkop et al., 2020; Cao et al., 2020; Goldman et al., 2023a), motifs (Park et al., 2023), and long-distanee atomic interactions (Young al., 2024). Another line of research approach bypasses spectral library comparison and directly performs de novo structure elucidation from spec- tra (Stravs al., 2021; Goldman al., 2023b; Tao al., 2024).
Since different spectroscopic techniques offer complementary advantages, the joint analysis of mul- tiple spectra can provide comprehensive information (Alberts et al., 2024). In this study, we encodes multiple spectra, and introduce them into molecular representation pre-training for the first time.
6 CONCLUSION
In this study. we explore pre-training molecular 3D representations beyond classical mechanics. By leveraging the correlation between molecular energy level structures and molecular spectra in quantum mechanics, we introduce molecular spectra for pre-training molecular 3D representations (MolSpectra). By aligning the 3D encoder trained with a denoising objective and the spectrum encoder trained with masked patch reconstruction objective, enhance the informativeness and transferability of the resulting 3D representations.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This work jointly supported by National Science and Technology Major Project (2033ZD010000)) and National Natural Science Foundation of China (62372454, 62236010).
REFERENCES
Saman Alavi. Intra- and intermolecular potentials in simulations. In Chapter 3, pp. 39-71. John Wiley & Sons, Ltd, 2020. ISBN 9783527699452. doi: 10. 10.102/978352669455...3..
Marvin Alberts, Oliver Schilter, Federico Zipoli, Nina Hartrampf, and Teodoro Laino. Unraveling molecular structure: multimodal spectroscopic dataset for chemistry. NeurlPS Datasets and Benchmarks Track, 2024.
Ilyes Batatia, D'avid P'eter Kov'ace acs, Gregor N. C. Simm, Christoph Ortner, and Gábor Csányi. Mace: Higher order equivariant message passing neural networks for fast and accurate force fields. In NeurlPS, 2022.
Liu Cao, Mustafa Guler, Azat M. Tagirdzhanov, Yi- Lee, Alexey A. Gurevich, and Hosein Mohimani. Moldiscovery; learning mass spectrometry fragmentation of small molecules. Nature Communicationss 12, 2020.
Dingshuo Chen, Yanqiao Zhu, Jieyu Zhang, Yuanqi Du, Zhixun Li, Qiang Liu, Shu Wu, and Liang Wang. Uncovering neural scaling laws in molecular representation learning. In NearlPS, 2023.
Dingshuo Chen, Zhixun Li, Yuyan Ni, Guibin Zhang, Ding Wang, Qiang Liu, Shu Wu, Jeffrey Xu Yu, and Liang Wang. Beyond efficiency: Molecular data pruning for enhanced generalization. In NeurlPS, 2024.
Ting Chen, Simon Korblithh, Mohammad Norouzi, and Geoffrey E. Hinton. simple framework for contrastive learning of visual representations. In ICML, volume 119, pp. 1597-1607, 2020.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT (1), pp. 4171-4186. As- sociation for Computational Linguistics, 2019.
Weitao Du, Jiujiu Chen, Xuecang Zhang. Zhi-Ming Ma, and Shengchao Liu. Molecule joint auto- encoding: Trajectory pretraining with and 3d diffusion. In NeurlPS, 2023a.
Weitao Du, Yuanqi Du, Limei Wang, Dieqiao Feng. Guifeng Wang, Shuiwang Ji, Carla P. Gomes, and Zhi-Ming Ma. A perspective on building efficient and expressive 3d equivariant graph neural networks. In NeurIPS, 2023b.
Kai Dührkop, Louis-Félis Nothias, Markus Fleischauer, Raphael Reher, Marcus Ludwig, Martin A. Hoffmann, Daniel Petrás, William Gerwick, Juho Rousu, Pieter C. Dorrestein, and Sebastian Böcker. Systematic classification of unknown metabolites using high-resolution fragmentation mass spectra. Nature Biotechnology, 39:462 471, 2020.
Shikun Feng, Yuyan Ni, Yanyan Lan, Zhi-Ming Ma, and Wei-Ying Ma. Fractional denoising for 3d molecular pre-training. In ICML, volume 202. PMLR, 2023.
Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. Neural message passing for quantum chemistry. In ICML, 2017.
Samuel Goldman, John Bradshaw, Jiayi Xin, and Connor W. Coley. Prefix-tree decoding for pre- dicting mass spectra from molecules. In NeurIPS, 2023a.
Samuel Goldman, Jeremy Wohlwend, Martin, Strazar, Guy Haroush, Ramnik Xavier, Connor, and Coley. Annotating metabolite mass spectra with domain-inspiree chemical formula trans- formers. Nature Machine Intelligence, 2023b.
Haisong Gong. Qiang Liu, Shu Wu, and Liang Wang. Text-guided molecule generation with diffu- sion language model. In AAAI, 2024.
11

---

Published as a conference paper at ICLR 2025
Jiaqi Han, Jiacheng Cen, Liming Wu, Zongzhao Li, Xiangzhe Kong. Rui Jiao, Ziyang Yu, Tingyang Xu, Fandi Wu, Zihe Wang, Hongteng Xu, Zhewei Wei, Yang Liu, Yu Rong, and Wenbing Huang. A survey of geometric graph neural networks: Data structures, models and applications. arXiv, abs/2403.00488, 00445, 2024.
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross B. Girshick. Masked autoencoders are scalable vision learners. CVPR, pp. 15979-15988. IEEE, 2022.
Zhenyu Hou, Xiao Liu, Yukuo Cen, Yuxiao Dong, Hongxia Yang, Chunjje Wang, and Jie Tang. Graphmae: Self-supervised masked graph autoencoders. In KDD, pp- 594-604. ACM, 2022.
Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay S. Pande, and Jure Leskovec. Strategies for pre-training graph neural networks. In ICLR, 2020.
Xinke Jiang, Rihong Qiu, Yongxin Xu, Wentao Zhang, Yichen Zhu, Ruizhe Zhang, Yuchen Fang, Xu Chu, Junfeng Zhao, and Yasha Wang. Ragraph: A general retrieval-augmennte graph learning framework. In NeurIPS, 2024.
Rui Jiao, Jiaqi Han, Wenbing Huang, Yu Rong, and Yang Liu. Energy-motivated equivariant pre- training for 3d molecular graphs. In AAAI, pp. 8096 8104. AAAI Press, 2023.
Dongki Kim, Jinheon Baek, and Sung Ju Hwang. Graph self-supervised learning with accurate discrepancy learning. In NeurlPS, 2022.
Hyeonsu Kim, Jeheon Woo, Seonghwan Kim, Seokhyun Moon, Jun Hyeong Kim, and Woo Youn Kim. Geotmi: Predicting quantum chemical property with easy-to-obtain geometry via positional denoising, In NeurlPS, 2023.
Johannes Klicpera, Shankari Giri, Johannes Margraf, and Stephan Gunnemann. Fast arXiv,
and uncertainty-aware directional message passing for non-equilibrium molecules. abs/2011.14115, 2020a.
Johannes Klicpera. Janek GroB, and Stephan Günnemann. Directional message passing for molec- ular graphs. In ICLR, 2020b.
Fan-Fang Kong. Xiao-Jun Tian, Yang Zhang, Yun-Jie Yu, Shi-Hao Jing, Yao Zhang, Guangjun Tian, Yi Luo, Jinlong Yang, Zhenchao Dong, and G. Hou. Probing intramolecular vibronic coupling through vibronic-state imaging. Nature Communications, 12, 2021.
Noah M. Lancaster, Pavel Sinitcyn, Patrick Forny, Trenton M. Peters-Clarke, Caroline Fecher, An- drew J. Smith, Evgenia Shishkova, Tabiwang N. Arrey, Anna Pashkova, Margaret Lea Robinson, Nicholas L. Arp. Jing Fan, Julia K. Hansen, Andrea Galmozzi. Lia R. Serrano, Julie Rojas, Au- drey P. Gasch, Michael S. Westphall, Hamish Stewart, Christian Hock, Eugen Damoe, David J. Pagliarini, Vlad Zabrouskov, and Joshua J. Coon. Fast and deep phosphoproteome analysis with the orbitrap astral mass spectrometer. Nature Communications, 15, 2024.
Shuangli Li, Jingho Zhou, Tong Xu, Dejing Dou, and Hui Xiong. Geomgel: Geometric graph contrastive learning for molecular property prediction. In AAAI, pp. 3541-4549, AAAI Press, 2022.
Zhixun Li, Liang Wang, Xin Sun, Yifan Luo, Yanqiao Zhu, Dingshuo Chen, Yingtao Luo, Xiangxin Zhou, Qiang Liu, Shu Wu, Liang Wang, and Jeffrey Xu Yu. GSLB: the graph structure learning benchmark. In NeurIPS, 2023.
Yi-Lun Liao and Tess E. Smidt. Equiformer: Equivariant graph attention transformer for 3d atom- istic graphs. In ICLR, 2023.
Shengehao Liu, Hanchen Wang, Weiyang Liu, Joan Lasenby, Hongyu Guo, and Jian Tang. Pre- training molecular graph representation with 3d geometry. In ICLR, 2022.
Shengchao Liu, Weitao Du, Zhi-Ming Ma, Hongyu Guo, and Jian Tang. A group symmetric stochas- tic differential equation model for molecule multi-modd modal pretraining. In ICML, volume 202, pp. 21497-21526. PMLR, 2023a.
12

---

Published as a conference paper at ICLR 2025
Shengchao Liu, Hongyu Guo, and Jian Tang. Molecular geometry pretraining with (in)-invari denoising distance matching. In ICLR, 2023b.
Yi Liu, Limei Wang, Meng Liu, Xuan Zhang, Bora Oztekin, and Shuiwang Ji. Spherical message passing for 3d molecular graphs. In International Conference Learning Representations, 2021.
Zhiyuan Liu, Yaorui Shi, An Zhang, Enzhi Zhang, Kenji Kawaguchi, Xiang Wang, and Tat-Seng Chua. Rethinking tokenizer and decoder in masked graph modeling for molecules. In NeurIPS, 2023c.
Shengjie Luo, Tianlang Chen, Yixian Xu, Shuxin Zheng, Tie- Yan Liu, Liwei Wang, and Di He. One transformer can understand both 2d 3d molecular data. In JCLR, 2023.
Hehuan Ma, Feng Jiang. Yu Rong, Yuzhi Guo, and Junzhou Huang. Toward robust self-training
paradigm for molecular prediction tasks. Journal of Computational Biology, 31(3)2113-225 2024. doi: 10. wtt.///mm..22...11...
Albert Musaclian, Simon L. Batzner, Anders Johansson, Lixin Sun, Cameron J. Owen, Mordechai Kornbluth, and Boris Kozinsky. Learning local equivariant representations for large-scale atom- istic dynamics. Nature Communications, 14, 2023.
Maho Nakata and Tomomi Shimazaki. Pubchemqe project: A large-scale first-principles electronic structure database for data-driven chemistry. Journal of chemical information and modeling, 57 6:1 6:1300-1308, 2017.
Yuyan Ni, Shikun Feng. Wei- Ying Ma, Zhi-Ming Ma, and Yanyan Lan. Sliced denoising: A physics- informed molecular pre-training method. In ICI.R, 2024.
Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. In ICLR, 2023.
Jiwon Victoria Park, Jeonghee Jo, and Sungroh Yoon. Mass spectra prediction with structural motif- based graph neural networks. Scientific Reports, 14, 2023.
Raghunathan Ramakrishnan, Pavlo O. Dral, Pavlo O. Dral, Matthias Rupp, and O. Anatole von Lilienfeld. Quantum chemistry structures and properties of 134 kilo molecules. Scientific Data, 1, 2014.
Yu Rong, Yatao Bian, Tingyang Xu, Weiyang Xie, Ying Wei, Wenbing Huang, and Junzhou Huang. Self-supervised graph transformer on large-scale molecular data. In NeurlPS, 2020.
Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling. E(n) equivariant graph neural net- works. In ICML, volume 139, pp. 9323 9332. PMLR, 2021.
Kristof Schütt, Oliver T. Unke, and Michael Gastegger. Equivariant message passing for the pre- diction of tensorial properties and molecular spectra. In ICML, volume 139 Proceedings of Machine Learning Research, pp. 9377-9388, PMLR, 2021.
Kristof T. Schütt, Huziel E. Sauceda, P Kindermans, Alexandre Tkatchenko, and Robert Müller. Schnet a deep learning architecture for molecules and materials. The Journal of chemical physics, 148 24:241722, 2017.
Andrew Shin, Masato Ishii, and Takuya Narihira. Perspectives and prospects transformer archi- tecture for cross-modal tasks with language and vision. International Journal of Computer Vision, 130:435 454, 2021.
Hannes Stärk, Dominique Beaini, Gabriele Corso, Prudencio Tossou, Christian Dallago, Stephan Günnemann, and Pietro Lió. 3d infomax improves gnns for molecular property prediction. In ICML, volume 162, pp. 20479-20502. PMLR, 2022.
Michael A. Stravs, Kai Dührkop, Sebastian Böcker, and Nicola Zamboni. Msnovelist: de novo structure generation from mass spectra. Nature Methods, 19:865 870, 2021.
Xin Sun, Liang Wang. Qiang Liu, Shu Wu, Zilei Wang, and Liang Wang. DIVE: subgraph disagree- ment for graph out-of-distributiio generalization. In KDD, 2024.
13

---

Published as a conference paper at ICLR 2025
Shijie Tao, Yi Feng, Wenmin Wang, Tiantian Han, Pieter E S Smith, and Jun Jiang. A machine learning protocol for geometric information retrieval from molecular spectra. Artificial Intelli- gence Chemistry, 2024.
Philipp Thölke and Gianni De Fabritiis. Equivariant transformers for neural network based molecu- lar potentials. In ICLR, 2022.
Aäron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predic- tive coding, arXiv, abs/ 807.03748, 2018.
Pascal Vincent. A connection between score matching and denoising autoencoders. Neural Compu- tation, 23:1661-1613 2011.
Liang Wang, Qiang Liu, Shaozhen Liu, Xin Sun, Shu Wu, and Liang Wang. Pin-tuning: Parameter- efficient in-context tuning for few-shot molecular property prediction. In NewrIPS, 2024a.
Liang Wang. Xiang Tao, Qiang Liu, Shu Wu, and Liang Wang. Rethinking graph masked autoen- coders through alignment and uniformity. In AAAI, 2024b.
Liang Wang, Chao Song, Zhiyuan Liu, Yu Rong, Q. Liu, Shu Wu, and Liang Wang. Diffusion models for molecules: survey of methods and tasks. arXiv, abs/2502.00511, 2025.
Xu Wang, Huan Zhao, Wei-Wei Tu, and Quanming Yao. Automated 3d pre-training for molecular property prediction. In KDD, pp. 1419-24330 ACM, 2023a.
Yiqun Wang, Yuning Shen, Shi Chen, Lihao Wang, Fei Ye, and Hao Zhou. Learning harmonic molecular representations on riemannian manifold. In ICLR, 2023b.
Yuyang Wang, Jianren Wang, Zhonglin Cao, and Amir Barati Farimani. Molecular contrastive learning of representations via graph neural networks. Nature Machine Intelligence, 4(3):279- 287, 2022.
Jennifer N. Wei, Jennifer N. Wei, David Belanger, Ryan P. Adams, and D. Sculley. Rapid prediction of electron-ionization mass spectrometry using neural networks. ACS Central Science, 5:700 708. 2018.
Jun Xia, Chengshuai Zhao, Bozhen Hu, Zhangyang Gao, Cheng Tan, Yue Liu, Siyuan Li, and Stan Z.
Li. Mole-bert: Rethinking pre-training graph neural networks for molecules. In ICLR, 2023.
Adamo Young, Bo Wang, and Hannes Rost. Tandem mass spectrum prediction for small molecules using graph transformers. Nature Machine Intelligence, 2024.
Qiying Yu, Yudi Zhang, Yuyan Ni, Shikun Feng, Yanyan Lan, Hao Zhou, and Jingjing Liu. Multi- modal molecular pretraining via modality blending. In ICLR, 2024.
Chaohao Yuan, Kangfei Zhao, Ercan Engin Kuruoglu, Liang Wang, Tingyang Xu, Wenbing Huang. Deli Zhao, Hong Cheng, and Yu Rong. A survey of graph transformers: Architectures, theories and applications. arXiv, 152/2502..1533,1 2025.
Sheheryar Zaidi, Michael Schaarschmidt, James Martens, Hyunjik Kim, Yee Whye Teh, Alvaro Sanchez-Gonzalez, Peter W. Battaglia, Razvan Pascanu, and Jonathan Godwin. Pre-training via denoising for molecular property prediction. In ICLR, 2023.
Gengmo Zhou, Zhifeng Gao, Qiankun Ding, Hang Zheng, Hongteng Xu, Zhewei Wei, Linfeng Zhang, and Guolin Ke. Uni-mol: A universal 3d molecular representation learning framework. In ICLR, 2023.
Kun Zhou and Bo Liu. Chapter 2 potential energy functions. In Molecular Dynamics Simula- fion, pp. 41-65. Elsevier, 2022. ISBN 978-0-12-816419.. doi: 10. 10100166978800-1281611994 00007-6
Jinhua Zhu, Yingce Xia, Lijun Wu, Shufang Xie, Tao Qin. Wengang Zhou, Houqiang Li, and Tie- Yan Liu. Unified 2d and 3d pre-training of molecular representations. In KDD, pp. 2626-2636. ACM, 2022.
14

---

Published as a conference paper at ICLR 2025
Yu Zong, Yuxin Wang, Xipeng Qiu, Xuanjing Huang, and Liang Qiao. Deep learning prediction of glycopeptide tandem mass spectra powers glycoproteomics. Nature Machine Intelligence, 2024.
Zihan Zou, Yujin Zhang, Lijun Liang, Mingzhi Wei, Jiancai Leng, Jun Jiang, Yi Luo, and Wei Hu. A deep learning model for predicting selected organic molecular spectra. Nature Computional Science, 3(115:957-964 2023.
15

---

Appendix
Contents of the appendix
A Proof of theoretical results
16
Visualization and analysis of spectra
16
Implementation details
17
C.I Hardware and software
17
C.2 Model configuration..
17
More experimental results and discussions
17
E Visualization of attention patterns and learned spectra representations in Spec
18
F Limitations and potential future directions
19
A PROOF OF THEORETICAL RESULTS
Theorem A.1 (Equivalence between the denoising objective and the learning of molecular force fields (Zaidi et al., 2023)), Assume the conformation distribution is a mixture of Gaussian distribu- tion centered the equilibriums:
p(x) = p(a//uu p(ar)zu)p(az)) p(a](a)) ~ N laN)
(A1)
To. x IR are equilibrium and noisy conformation respectively, N is the number of atoms the molecule. It relates to molecular energy by Bolrzmann distribution p(x) x (xx))
Then given a sampled molecule M. the denoising loss on the conformation coordinates an equiv- alent optimization target to force field prediction:
Denoising (M) = Ep((el=o)p((o)) ( - (x æo)l
(A2)
= (=
(A3)
where GNNo( denotes graph neural network with parameters 0 which takes conformation as an input and returns node-level noise predictions, = denotees equivalence.
Proof. According to Boltzmann distribution, Eq. A3 is equal to Ep(a)) [| V. log p( || By using conditional score matching lemma (Vincent, 201 1), the equation above is further equal to Eppæ æ=))((.)) IanlllGNN No(x) - Va log p( (|) + Tı, where Tj is constant independent 0. =| + TT. Finally, since coefficients f do not rely on the input æ. it can be absorbed into GNN,, thus obtaining Eq. A2.
Then with the Gaussian assumption, it becomes (p(m)m)))(mm)) GNNN((x) G.No:ip
B VISUALIZATION AND ANALYSIS OF SPECTRA
In this section, we visualize the three types of spectra we utilize (UV-Vis, IR. Raman) and standard- ize the initial spectral data based on data analysis. In Figure A1, we visualize 20 randomly sampled spectra from QM9S for each type of spectrum. A notable pattern observed that, although each spectrum consists of numerous absorption peaks, there are significant differences in the heights (ab- sorption intensities) of these peaks. For instance, in the IR spectra, the absorption intensity at most peaks is around 200, but few peaks reach an intensity of 800. However, in qualitative analysis, the position and shape of the peaks are more critical than their heights. Therefore, the differences in peak absorption intensities this issue, we pre- pre-process the absorption intensities the spectra by applying log transformation mitigate the interference caused by peak intensity differences.
interfere with model training under the MSE loss metric. To address
16

---

Published as a conference paper at ICLR 2025
UV-Vis Spectra
IR Spectra
Raman Spectra
Relative Intensity
0.20
0.15
Intensity
600
Intonsity
400-
0.10
200
0.05
200
0.00
200
400
2000
3000
1000
2000
3000
Frequency Index
Frequency Index
Frequency Index
Figure A1: Randomly sampled examples of molecular energy spectra.
C IMPLEMENTATION DETAILS
C.I HARDWARE AND SOFTWARE
Our experiments are conducted on Linux servers equipped with 184 Intel Xeon Platinum 8469C CPUs, 920GB RAM, and NVIDIA H20 96GB GPUs. Our model implemented in PyTorch version 2.3.I, PyTorch Geometric version 2.6.1 (https://pyg.org/ with CUDA version 12.1, and Python 1..1..
C.2 MODEL CONFIGURATION
The SpecFormer is implemented using a 3-layer Transformer with 16 attention heads. Following previous works, we both d and dę as 256. TorchMD-Net (Thölke & Fabritiis, 2022) is adopted as the 3D molecular encoder. We tune the mask ratio (i.e., a) in (0.05, 0010, 0.15, 0.20, 0.25, 0.30), tune the "stride/patch length" pair (i.e., D./P,) in {5/20, 10/20, 15/20, 20/20, 8/16, 15/30). and tune the weights of sub-objectives (i.e., Denoising: 8MPR. and Soontrast ) in {0.01, 0.1, 1, 10}. Since our goal is to align the 3D representations and spectra representations of molecules during the pre- training phase, and not rely molecular spectra data during downstream fine-tuning, these hyper- parameters related to molecular spectra are tuned on the pre- pre-training dataset. Based on the results of hyper-parameter tuning, adopt 0.10, D, 10, P. 20, Denoising 1.0, 8MPR 1.0, and @Contrast 1.0.
Following SimCLR (Chen et al., 2020), the contrastive loss in our Eq. 7 is implemented using in-batch contrastive loss, where positive and negative pairs are constructed within each data batch. Therefore, for each anchor representation in batch, there is one positive sample and bs negative samples, where bs is the batch size. In our method, bs = 128.
In both pre-training stages, Coord (Zaidi et al., 2023), specifically energy function as described in Section 2.2. The noise is added atom positions as scaled mixture of isotropic Gaussian noise, with scaling factor of 0.04.
the noise generation method and denoising objective provided by
The denoising objective is defined in Eq. 2.
For baselines, we follow their recommended settings.
D MORE EXPERIMENTAL. RESULTS AND DISCUSSIONS
In addition to Coord, we evaluate the effect of incorporating SliDe (Ni et al., 2024) into our Mol- Spectra. SliDe is also a denoising-
pre-training method, utilizing the TorchMD-Net (Thölke & Fabritiis, 2022) as its encoder backbone, consistent with previous pre-training work (Zaidi et al., 2023; Feng et al., 2023). The results are presented in Table Al and Table A2.
Integrating method with SliDe effectively reduces the error in property prediction on the QM9 dataset and the MD17 dataset. Given that our method enhances both Coord and SliDe, this suggests that our approach is broadly effective across various denoising- based pretraining strategies. Further- more, incorporating molecular spectra can guide the pre-trained model to acquire knowledge beyond what denoising objectives offer, which proves beneficial for downstream property prediction.
17

---

Published as a conference paper at ICLR 2025
Table AI: Performance (MAEL) on QM9 dataset. The better result between the two variants of each pretraining method, w/ and w/o MolSpectre is highlighted in bold.
homo
lumo (meV)
gap (meV)
R2
ZPVE
(D)
(a))
(meV)
(ag)
(meV)
Coord
0.016
0.052
17.7
14.7
31.8
0.450
1.71
Coord w/ MolSpectra
0.011
0.048
15.5
13.1
26.8
0.410
1.71
SliDe
0.015
0.050
18.7
16.2
28.8
0.606
1.78
SliDe w/ MolSpectra
0.012
0.043
17.0
15.8
28.5
0.424
1.73
Performance (MAEJ) on MD17 dataset. The better result between the two variants of each pretraining method, w/ and w/o MolSpectra, is highlighted in bold.
Table A2:
Malonal -dehyde
Naphtha -lene
Salicy -lic Acid
Aspirin
Benzene
Ethanol
Toluene
Uracil
Coord
0.211
0.169
0.096
0.139
0.053
0.109
0.058
0.074
Coord w/ MolSpectra
0.099
0.097
0.052
0.077
0.085
0.093
0.075
0.095
SliDe
0.174
0.169
0.088
0.154
0.048
0.101
0.054
0.083
SliDe MolSpectra
0.160
0.054
0.055
0.088
0.073
0.098
0.077
0.097
E VISUALIZATION OF ATTENTION PATTERNS AND LEARNED SPECTRA REPRESENTATIONS IN SPECFORMER
200
300
300
300
100
200
300
300
200
300
1000
200
300
Attention
Head
Attention (Layer Hend
(c) Attennioo (Layer Head
Representations
Figure A2: (a-c) Attention maps from three attention heads in SpecFormer. Different heads model distinct dependencies. (d) t-SNE visualization of the spectra representations produced by Spec- Former.
We visualize the attention patterns and learned spectra representations in SpecFormer. Based on the visualizations presented in Figure A2, we have made the following observations.
In Figure A2(a-c), we visualize attention maps from three attention heads in SpecFormer's is second layer. The attention weights within the three blocks along the main diagonal indicate intra-spectrum dependencies, while those outside reveal inter-spectrum dependencies, as explained in Section 3.1. can be observed that different attention heads model distinct dependencies: Head 11 focuses on intra-spectrum dependencies, Head 13 focuses on inter-spectrun dependencies, and Head 12 mod- els both types simultaneously. inter-spectrum dependencies, the interaction between spectra and Raman spectra is relatively pronounced, which may be related to their mutual association with vibrational modes. Additionally, because the intensity peaks and dependencies molecular spectra sparse, the attention maps in SpecFormer are generally sparse well.
In Figure A2(d), we use t-SNE to visualize the spectra representations produced by the final layer of SpecFormer. It can be observed that the distribution of representations in the latent space is relatively uniform and forms several potential clusters. This well-shaped distribution of representations reveals effective spectra representation learning and supports the structure-spectruum alignment.
18

---

Published as a conference paper at ICLR 2025
F LIMITATIONS AND POTENTIAL FUTURE DIRECTIONS
One limitation of our method is the availability, scale, and diversity molecular spectral data. Our current dataset comprises geometric structures of 134,000 molecules, each with three types of spec- tra (UV-Vis, IR, Raman). To effectively explore the scaling laws of pre-training methods, larger and more diverse molecular spectral datasets are necessary. Encouragingly, molecular spectroscopy has been gaining increasing attention in the research community, with larger and more diverse datasets being released, such as the recent multimodal spectroscopic dataset (Alberts al., 2024). This development supports advancements in molecular representation learning and other related tasks.
Another limitation is that our proposed SpecFormer can currently only handle one-dimensional molecular spectra. For higher-dimensional spectra, such as two-dimensional NMR and two- dimensional correlation spectra, further development of sophisticated spectrum encoder is needed.
Looking ahead, we envision several future directions in this field. First, there is potential in in- vestigating the scaling laws pre-training on larger and more diverse molecular spectral datasets. Second, expanding the scope of molecular spectrum encoding to include wider range, such as NMR, mass spectra, and two-dimensional spectra, could be highly beneficial. Third, while pre- trained spectral encoder has been developed in method, we have far only applied the pre- trained 3D encoder to downstream tasks. Exploring the use of the pre-trained spectral encoder for molecular spectrum-related downstream tasks, such as automated molecular structure elucida- tion from spectra, represents an promising opportunity. Finally, current molecular 3D pre-training methods designed based on TorchMD-Net (Thölke Fabritiis, 2022). With the development of equivariant message passing neural networks, more expressive backbone architectures, such as Allegro (Musaelian et al., 2023) and MACE (Batatia al.. 2022) have been proposed, improving the prediction of molecular properties when trained from scratch. Extending pre-training strategies these state-of-the- thett-rart architectures holds the promise further advancing downstream tasks.
19


Reviews:
[{"review_id": "haP4jgLgG7", "reviewer": "Reviewer_vWL4", "summary": "Authors introduce a new data modality to the pretraining of 3d molecular structures representation models, namely absorbtion spectra in three different frequency spans: IR, UV, Raman. To encode the spectra they use transformer on top of the spectral patches with positional encoding. They use two-stage pretraining, by first denoising on a larger dataset without spectra; and then using contrastive learning objective coupled with the masked spectral patch reconstruction objective to finish pretraining. Authors demonstrate effectivnes of incorporating spectral information without denoising pretraining on QM9 dataset. Then they show small improvement in prediction quality on downstream tasks for QM9 and MD17 datasets.", "questions": "1. Can the model be learned in one step simultaneously with contrastive objective and denoising one? \n2. How sensitive the results are to different learning rate/transformer configuration given fixed stride/patch/mask/ optimizer parameters.", "limitations": "", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Incorporating additional data is always useful. Although benefits are marginal, this work will be relevant for subsequent methods that learn representations of molecules. The validating experiments show that performance improvements are consistent albeit small.", "weaknesses": "1. The method of pretraining suffers from additional complexity due to two-step pretraining procedure. The choice of such pretraining schedule is not well explained.\n2. Table 2, first column, methods Torch-MD and PaiNN should be bold, similarly second column there methods that perform better or on par with the proposed one.\n3. Table 4 and 5 show that the method is sensitive to the parameters of the patch and stride sizes of spectrum encoder, the difference in performance in predicting homo 0.2, lumo 0.5, gap 0.2 between the closest values of these parameters. This roughly corresponds to the difference between next best performing method homo 0.5, lumo 1.5, gap 0.6. We suspect that row 1 column overlap ration the number is incorrect and should be swapped with row 3 same column. \n4. Authors mention that they fine tuned parameters such as stride/patch/mask and additionally weights of each objective. We hope that the tuning was done using pretraining dataset and not downstream tasks performance. This is not mentioned in appendix C2 or the main text. Please clarify this point.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EyiF5U2YDx", "reviewer": "Reviewer_yfxG", "summary": "- Unlike common approaches, the authors incorporate quantum information to enhance the quality of molecular representations.\n\n - The denoising component of the model is carefully designed to account for rotational and vibrational degrees of freedom in the energy.\n\n - The model consists of two distinct parts connected by contrastive loss: one for denoising and the other a transformer for quantum spectrums.\n\n - Numerous experiments demonstrate the model’s superiority over conventional methods.\n\nThe attempt to incorporate quantum information into the model is impressive. Generally, this approach is believed to enhance prediction performance over conventional models that rely on classical methods. However, as outlined in the questions section, there are still unresolved points. Therefore, the score is not final, and I am open to further discussion with the authors before finalizing it.", "questions": "- The model appears computationally heavy. Is there any analysis of the computational costs for the experiments conducted?\n\n - Based on results from other studies, Frad and SliDe seem to perform better on QM9 and MD17 tasks. Have any additional tests been conducted to compare these recent methods?\n\n - If Frad and SliDe outperform the proposed model—and both already account for specific potential energy forms similar to this model—can it truly be claimed that the inclusion of quantum spectral data contributes to the model's superiority? Is there any analysis or ablation study that demonstrates the usefulness of the quantum data?", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 2, "contribution": 3, "strengths": "- A quantum mechanical approach is considered for molecular representation learning.\n\n - The denoising component of the model is designed with sophistication, incorporating both rotational and vibrational energies into the Boltzmann distribution.\n\n - A contrastive setting enables inference without molecular spectral data, enhancing the model's usability in real-world situations.\n\n - Numerous downstream experiments demonstrate that the model outperforms conventional approaches.", "weaknesses": "- The spectral data has a high dimensionality, and the model’s transformer architecture is quite resource-intensive. Given that the contrastive loss requires numerous data combinations, the model is likely to be computationally demanding.\n\n - The experiments do not include comparisons with recent models, such as SliDe (1) or Frad(2), despite the model’s energy function incorporating potential forms used in both SliDe and Frad.\n\n - The experimental conditions are not rigorously introduced such as number of negative samples in contrastive loss and noise generation parameters.\n\n\n <References>\n\n(1) Yuyan Ni, Shikun Feng, Wei-Ying Ma, Zhi-Ming Ma, and Yanyan Lan. Sliced denoising: A physicsinformed molecular pre-training method. arXiv preprint arXiv:2311.02124, 2023.\n\n(2) Shikun Feng, Yuyan Ni, Yanyan Lan, Zhi-Ming Ma, and Wei-Ying Ma. Fractional denoising for\n3d molecular pre-training. In International Conference on Machine Learning, pp. 9938–9961.\nPMLR, 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "q9OpD0FkxC", "reviewer": "Reviewer_baLq", "summary": "This paper proposes MolSpectra, an approach for pre-training 3D molecular representations by combining a denoising objective with a contrastive loss. The contrastive loss aligns the 3D representations with representations for molecular spectra (UV-Vis, IR, Raman), which are trained via a masked patch reconstruction (MPR) objective. The authors argue that incorporating information from molecular spectra enables MolSpectra to learn the dynamic evolution of molecules by understanding energy level transition patterns. MolSpectra is evaluated on the QM9 and MD17 benchmarks and compared to existing methods.", "questions": "* From the appendix, it seems like TorchMD-Net is the model used for structural representation learning. Is the architecture in any way different to the TorchMD-Net model for which results are reported in Tables 2 and 3? If yes, these differences should be pointed out explicitly. Also, I suggest relabelling the entries for MolSpectra in Tables 2 and 3 as something like \"TorchMD-Net (w/ MolSpectra)\" or similar. Also, the baseline (see my suggestions above) should be clearly labelled as such, so that readers know which numbers to compare to.\n\n* Could the authors elaborate on the choice of the specific spectra types (UV-Vis, IR, and Raman) used in this work? It seems to me like other types of spectra, such as NMR and mass spectra, could provide additional information and further enhance the learned representations.\n\n* Can the authors provide details on the computational complexity and resource requirements of MolSpectra, including training time and memory usage? A comparison with baseline methods would be helpful.\n\n* How does the performance of MolSpectra scale with the size and diversity of the molecular dataset used for pre-training?\n\n* Can the authors provide a more in-depth analysis of the learned representations? For instance, visualizing the latent space or analyzing the attention patterns in SpecFormer could provide insights into the captured features and relationships.\n\n* For the simple test of the effectiveness of molecular spectra (section 4.1/Table 1), where do the spectra used to obtain the spectral representations come from? I assume they are taken from QM9S, but this should be stated explicitly.\n\n* How sensitive is MolSpectra to the choice of hyperparameters in Eq.8 ($\\beta_{\\text{Denoising}}$, $\\beta_{\\text{MPR}}$, and $\\beta_{\\text{Contrast}}$)? The authors mention in the appendix that they tuned these hyperparameters by trying different values. I believe the results for these different runs should also be included (in the appendix is fine), so readers can develop an intuition how changes to the values affect down-stream performance.\n\n* In the appendix, the authors write that they apply a $\\log_{10}$ transform to the peak intensities to mitigate interference causes by peak intensity differences. It seems intuitively more meaningful to me to instead normalize spectra by setting the height of the highest intensity peak to an arbitrary value (say 1) and scaling the remaining peaks proportionally. Have the authors experimented with different \"normalization procedures\" such as the one mentioned?\n\n**Additional Feedback:**\n\n* A discussion of the limitations of the proposed method and potential future directions for research would be interesting.\n\n* There is a typo on p.8 l.427/428: \"yiels\" should be \"yields\".", "limitations": "", "rating": 6, "confidence": 4, "soundness": 1, "presentation": 2, "contribution": 2, "strengths": "+ The paper presents an interesting approach to molecular representation learning by incorporating information from spectra.\n\n+ The motivation for incorporating spectral data is well-justified. The authors argue that quantized energy levels are fundamental to understanding molecular properties and dynamics, and that spectral data provides a direct measurement of these levels.\n\n+ The proposed SpecFormer architecture and the associated MPR and contrastive learning objectives are technically sound and well-designed.", "weaknesses": "- The results on the QM9 and MD17 benchmarks presented in Tables 2 and 3 are misleading for two reasons: (1) In Table 2 (QM9), the caption states that the best results are highlighted in bold, however, this is not true. In fact, the numbers for MolSpectra are highlighted in bold for 9 out of 12 columns, but it only achieves the best results in 3 out of 12 columns. (2) Both tables only include older models that are not SOTA anymore. Newer models such as MACE (https://proceedings.neurips.cc/paper_files/paper/2022/file/4a36c3c51af11ed9f34615b81edb5bbc-Paper-Conference.pdf) and Allegro (https://www.nature.com/articles/s41467-023-36329-y) achieve significantly better results than all listed models (I have not done a literature search to check which model is the current SOTA, those are just two models with better performance I know at the top of my head).\nThe authors should include results for more recent (SOTA) models in both tables and fix the formatting, so that the result that is actually best is highlighted in bold. In addition (this is a minor suggestion that does not affect my rating), I think it would be helpful to readers to include (either in the caption, or within the table itself) an explanation/label what the horizontal separator signifies. I assume this is to distinguish models trained from scratch from models that were pre-trained in an unsupervised manner and then finetuned, but this should probably be made explicit.\n\n- It is difficult to assess how big the improvement of including spectral information into the unsupervised representation learning actually is. This is because it is not immediately clear from Tables 1 and 2 how the same model architecture, (pre-)trained on the same structural data, performs when the *only* difference is whether spectral information was included in pre-training or not. The authors write on p.7 that Coord serves as primary baseline, but it is not clear from the text whether this is actually the same model architecture, and how exactly it was trained. To have an objective baseline, I suggest to pre-train the same architecture using the two-stage pre-training pipeline (section 3.4) twice, with the only difference being that $\\beta_{\\text{MPR}}$ and $\\beta_{\\text{Contrast}}$ are set to zero for one of the models. This way, the only difference is whether spectral information is used or not, allowing a direct assessment of the effectiveness of including this information.\n\n- The manuscript applies MolSpectra only to a single architecture for structural representation learning (TorchMD-Net). It is therefore difficult to judge whether MolSpectra is generally effective, or whether its usefulness is strongly dependent on the underlying architecture used for structural representation learning. To address this shortcoming, the authors should apply MolSpectra to pre-training other architectures (ideally using the method described in my previous point to establish objective baselines for each architecture). This would allow readers to assess MolSpectra in a broader context and would significantly strengthen the paper.\n\n- The authors state that \"MolSpectra learns the dynamic evolution of molecules by understanding energy level transition patterns\", however, this statement is not supported by direct evidence. I think it is a valid hypothesis, but it should be tested explicitly. Fortunately, a very direct test is possible: As the authors correctly state, the denoising objective is equivalent to learning a force field. This means that models trained with/without MolSpectra on a denoising task can be directly used as a force field to run molecular dynamics (MD) simulations. From such MD simulations, it is trivial to extract the power spectrum of a molecule via the velocity autocorrelation function (see e.g. https://doi.org/10.1039/C3CP44302G if the underlying theory is not familiar). The power spectrum contains the same peaks as the IR and Raman spectra, with the only differences being that (1) all internal vibrations are active (in contrast to IR/Raman spectra, where only some vibrations are visible - the power spectrum contains peaks from both!) and (2) the peak intensities are different. The peak positions, however, are directly comparable. If the spectral information actually teaches a model about the dynamics of a molecule, I would expect the power spectrum of a model trained with MolSpectra to show much better agreement (in peak positions) to the \"ground-truth\" IR/Raman spectra. The authors should perform this test, as it would make the paper much more insightful.\n\n- In section 2.2, the authors describe three different energy functions that can be used for pre-training. It is not immediately clear from the text which of these is actually used for MolSpectra. From context, I assume it is variant I, but I think stating this explicitly would make it easier to understand the details of the method.\n\n- The paper lacks an analysis of the computational complexity and resource requirements of the proposed method. A comparison of training time and resource usage with baseline methods would be beneficial.\n\n- While the ablation study provides insights into the importance of the MPR objective, a more comprehensive ablation study is needed to assess the individual contributions of different spectral modalities (UV-Vis, IR, Raman). This would provide a deeper understanding of how each component contributes to the overall performance.\n\n- The paper mainly compares against methods that rely on 3D structure information only. Comparison with other multimodal methods for molecular representation learning would provide a more complete picture of MolSpectra's performance relative to other methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)