Self-Supervised Diffusion Models for Electron-Aware Molecular Representation Learning
Authors: Gyoung S. Na
Chanyoung Park
Venue: ICLR 2025 Poster (2025)
Keywords: Representation learning;Generative models;Molecular science;Scientific applications
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)

Abstract: Physical properties derived from electronic distributions are essential information that determines molecular properties. However, the electron-level information is not accessible in most real-world complex molecules due to the extensive computational costs of determining uncertain electronic distributions. For this reason, existing methods for molecular property prediction have remained in regression models on simplified atom-level molecular descriptors, such as atomic structures and fingerprints. This paper proposes an efficient knowledge transfer method for electron-aware molecular representation learning. To this end, we devised a self-supervised diffusion method that estimates the electron-level information of real-world complex molecules without expensive quantum mechanical calculations. The proposed method achieved state-of-the-art prediction accuracy in the tasks of predicting molecular properties on extensive real-world molecular datasets.

Published as a conference paper at ICLR 2025
SELF-SUPERVISED FLECTRON-AWARE LEARNING
DIFFUSION MOLECULAR REPRESENTATION
MODELS FOR
Gyoung S. Na KRICT, Republic of Korea ngs00krict.. re..cttte....e. re.kr
Chanyoung Park KAIST, Republic of Korea сy . par.kkaisstttt........k..kk....a a...k.. kr
ABSTRACT
Physical properties derived from electronic distributions are essential information that determines molecular properties. However, the electron-level information is not accessible most real-world complex molecules due to the extensive com- putational costs of determining uncertain electronic distributions. For this reason, existing methods for molecular property prediction have remained in regression models on simplified atom-level molecular descriptors, such as atomic structures and fingerprints. This paper proposes an efficient knowledge transfer method for electron-aware molecular representation learning. To this end, we devised self- supervised diffusion method that estimates the electron-level information of real- world complex molecules without expensive quantum mechanical calculations. The proposed method achieved state-of-the-art prediction accuracy in the tasks of predicting molecular properties on extensive real- world molecular datasets.
INTRODUCTION
Machine learning has been widely studied as efficient data-driven method for predicting the phys- ical and chemical properties of molecules (Wigh et al. 2022). In particular, graph neural networks (GNNs) (Kipf & Welling (017)) achieved numerous successes in various molecular representation learning tasks (Bilodeau et al. (002 Duval et 2023). In GNNs, an atom-level molecular structure is defined as graph G (V, D,U,A, X, R), where Vis set of nodes (i.e., atoms), 2/ is set of edges
[JVIXII is an adjacency matrix, X E I | is dddimensional node-feature matrix, and R E 1244 is r-dimensional edge-feature matrix (Wieder et al. 2020
(i.e., chemical bonds), A E {0, 1)
[VI
In addition to traditional GNNs, various methods have been proposed to learn informative molecular representations from different approaches, such as fragmentation-base based learning (Zhang et al., [021 Kim et al. 2023). domain knowledge integration (Wang al. 2022). and hierarchical representation learning (ang et al. 2023). However, existing methods aimed to learn molecular representations from atom- molecular descriptors, while overlooking physical principle that molecular prop- erties are essentially derived from electron-level information, such as electronic distributions and related electronic energies, beyond the atom-level information (Parr & Yang 1995 EEgell & Drei- zier 2011). Therefore, the representation capabilities of existing methods for molecular repre- sentation learning are inherently limited, even though they were sophisticatedly designed to capture latent information from the atom-level molecular descriptors.
In physical science, quantum mechanical calculations have been used as a de facto standard to calculate the clectron-level molecular information (Parr & Yangl 1995). such molecular bital and atomization energy. However, as these methods suffer from cubic or greater time complexities with respect to the number of electrons in a molecule (Engel & Dreizler] 2011] Dawson al., (022), electron-level information about real-world complex and large molecules are usually not accessible in chemical applications. Although there is an efficient solution that uses sophisticatedly designed 3D-GNNs with electron-level information generated by force- field-based and semi-empirical calculations (FFSECs) (Riniker & Landrum 2013), the effec- tiveness of this straightforward solution is questionable due to the low calculation accuracy of the FFSEC methods. To corroborate our argument, we empirically evaluated the effec- tiveness of 3D-GNNs with FFSEC on well-known real-world molecular datasets: Lipop (Wul al. 2018), ESOL (Delaney 2004), and IGC50 (wu & Weil 2018) datasets. Fig. [ shows

---

Published as a conference paper at ICLR 2025
that the experimental evaluations did not demonstrate significant improve- ment by the 3D-GNNs with FFSEC. denoted by PhysChem (Yang et al. 2021), M3GNet (Chen & Ong
Upop
ESOL
IGC38
0.85
0.3
0.80
0.79
0.75
0.70
2022
8
0.3
and FAENet (Duval et al. 2023 while a simple 2D-GNN called AttFP (Xiong al. 2019) rather showed better prediction accuracy. We con- jecture two major reasons for such results: 1) The calculation errors of FFSEC can be propagated to the 3D GNN models, which in turn degrades the prediction accuracy. Complex 3D-GNNs are not effec- tive in representation learning on large molecules due to the overfitting problem (Li et al. 2023a).
0.50
0.30
0.3
20 CMR
30.0MM 6..
0.50
20 GMMs::
MPRX
UrMPP
AIP
30 GMAlic
PlyyChem
Figure 1: R -cooress Nagellerke et al. 1991) of 3D-GNNs with FFSEC on real-world molecular datasets.
In this paper, propose decomposition cinompooiiioo-uuprviiiedd electron-level information diffusion (DELID) for an electron-aware molecular representation learning. The main challenge is that the electron- level information is usually unknown for real-world molecule G. To this end, we propose self- supervised diffusion model for estimating the unknown latent representation So without its ground truth being accessible in the training process. As shown in Fig. 2. DELID consists of two diffusion models. The diffusion model
G aims estimate the original molecule G from the noise Gr, while the diffusion model on S aims to estimate the unknown complete electron-level information So of G from the noise Sy. The main idea of DELID is to consider GT as decomposed substructures of G, and Sr as their electron-leve information, where GT and ST can be casily obtained from molecu- lar decomposition algorithms (Liu et al., (017), and public chemical datahases (Ramakrishnan al.) 2014), respectively, without expensive quantum mechanical calculations. As illustrated Fig. [2h, DELID employs the transition probability p(G,-1 GG; Go) of the diffusion process on G as a self- supervision learn the diffusion process from readily accessible ST unknown Sp. In Section 3.2 we will mathematically show that the diffusion model on s can be optimized by minimizing the KL. divergence between p(G.-1|s;; Go) and p(G+-1|Gr: Go).
In our experiments, we focus on evaluating the prediction capabilities of the machine learning meth- ods biased and relatively small experimental datasets rather than simulated datasets (e.g.. QM9 dataset (Ramakrishnan al., 2014)). Although the simulated datasets are useful for analyzing rough statistics on small molecules, they are not appropriate to evaluate the prediction capabilities of the machine learning methods real-world molecular physies due the following two reasons: 1) The simulated datasets do not contain complex and large molecules due to the large time complexity of the quantum mechanical calculations. 2) The simulated datasets do not sufficiently reflect the quantum mechanical uncertainty in real-world molecules (Sim et al. 2018). For these reasons, we used experimentally collected molecular datasets from physicochemistry toxicity, pharmacokinet- ics, and optical applications to evaluate the practical potential of DELID. For all benchmark molec- ular datasets, DELID achieved state-of-the-an performance in predicting experimentally observed properties of real-world complex molecules. The contributions can be summarized as:
A novel method called DELID for learning electron-aware molecular representations beyond atom-level molecular representations without expensive quantum mechanical calculations.
self-supervised diffusion mechanism to estimate the unknown electron-level information.
The state-of-the-art prediction accuracy of DELID on extensive real-world molecular datasets containing experimentally collected complex molecules and their properties.
2
RELATED WORK
2.1 GRAPH NEURAL NETWORKS ON MOLECULES
2D-GNNs for molecular representation learning the 2D molecular structures have been widely studied in chemical science due to their practicality and efficiency. SchNet (Schütt et al., 2017) and MEGNet (Chen et al.) 2019) are graph convolutional neural networks for molecular representation learning on quantum mechanical principles in chemical bonds and local atomic substructures. They employed an atom-wiss wise representation to learn geometric information of the molecules. MPNN (Gilmer et al. 2017) is message-padding neural network that captures the quantum mechanics
2

---

Published as a conference paper at ICLR 2025
a. The overall architssture of DELID
b. Self-supervised diflusion process to estimate Fa
Diffusion process on
Diflusion
Decompesed motecules: Cy
Orig molecules G
Diffusion process on
Set-superision labels: feu.,,fr
(e))
electron-level information:s.
Incomplete
process
Diffusian
Comp etn electron-level informston: 5.
Predecton model: --109 ((006)
Incomplete information
superviee difusiio:: ((Ge-1 ((Ce_11A;
information
Complite
The overall architecture of DELID,, and b. the self-supervised diffusion process to estimate the complete electron-level information So that represents the ground truth s of G. In the self-supervised diffusion process, Gr, Go. and Sy are given data, whereas So is unknown data.
Figure 2:
between the atoms. Directed MPNN (D-MPNN) is an extension of MPNN for molecular repre- sentation learning based directed message passing scheme (Yang et al., 2019). In addition to the general-purpose GNNs, AttFP (Xiong et al., 2019) was proposed to predict the physical and chemical properties of the molecules for drug discovery.
3D-GNNs that utilize the 3D molecular structures in representation learning have been devised for
more accurate molecular representation learning. DimeNet++ (Gasteiger et al.) 2020).
PhysChem
Yang al. 202I), and M3GNet (Chen & Ong. 2022), and FAENet (Duval al.
2023) were
proposed to learn local and global 3D geometry of the molecules. ConAN (Nguyen et al. 2024) used
the 3D molecular structures of possible conformers of the input molecule in molecular representation learning. In addition to these methods, PaiNN (Schütt et al. (021). GemNet (Gasteiger et al,
2021 Equiformer (Liao & Smidt 2022), MolKGNN (Liu et al. 20223). and ViSNet (Wang et al, 2024 were proposed for molecular property prediction on the 3D structures. However, despite their -tate-of-the-art performances on several benchmark datasets on calculated molecular structures, their applicability is significantly limited in real-world molecular science because calculating an accurate 3D molecular geometry is not feasible in most real-world complex molecules due to the uncertainty of the electronic structures and the large computational costs to calculate them (Krivanek et al. 22100) Suenaga & Koshino 2010; Schuch & Verstraete, 2009). Although ConAN showed state-of-the. prediction accuracy with an efficient geometry calculation, its practicality also limited because we should generate all conformers of the input molecules for each inference process.
2.2 KNOWLEDGE TRANSFER METHODS FOR MOLECULAR REPRESENTATION LEARNING
Machine learning methods usually suffer from the lack of training data and informative features in chemical applications because time-consuming and labor-intensive chemical experiments are quired to collect experimentally generated data (Gromski et al. 2019 Shen et al. 2021). To over- come the lack of experimental data, transfer learning exploit simulated molecular data has been widely studied in physics and chemistry (Jha et al.) 2019 Cai et al., 2020 Zaverkin et al. 2023 Dou et al.) (023b)). However, the practicality of the transter learning methods on simulated molecular data is still limited because the source calculation databases are not able to cover the majority of large molecules in real-world chemical experiments due to the cubic or greater time complexities of the calculation methods with respect the number of electrons in the molecules (Engel & Dreizler 2011 Schuch & Verstraete| 2009). In addition to transfer learning, various molecular representation learning methods have been proposed to exploit fragmented information
moleculat lar representation
learning (Zhang et al. 2021 Wang et al. 2022 Yu & Gao 2022; Chen et al. 2022 Kim et al. 2023]
Feng et 2023). However, the representation capabilities of the existing molecular ar representation learning methods are essentially limited to the atom-level molecular representations because they did not consider how to estimate the electron- information and how to utilize it.
2.3 DIFFUSION MODELS
Diffusion models aim to learn a stochastic process that approximates the probability distribution of given dataset (Kingma et al.) 2021 Song et al., (2020). The diffusion models have achieved remark- able successes in learning physical systems consisting of complex and steg stochastic processes over conventional generative models (Zeni et al. 2023 Yuan et al. 2023 Wu & Li. 2023). The re- verse process of the diffusion models, which restore the data from noise, can be used as generative model generate new data (Ho et al.) [2022] Vignac al. 2022). In addition to the conventional dif- fusion models, conditional diffusion models were devised to generate new data of desired properties
Tashiro et al. 2021 Zhang al. 2023 Zbinden et al. 2023). Since the electron-level information is
3

---

Published as a conference paper at ICLR 2025
usually derived by complex and long-step physical processes (ParT & Yang 1995 Hollingsworth & Dror 2018), DELID employs the variational diffusion models (Huang et al. 2021 rather than shal- low generative models, such as variational autoencoder (Kingmal 2013). However, as the existing diffusion models essentially require the ground truth data learn the forward and reverse processes, they are not applicable to our problem where the electron-level information is not available.
3 PROPOSED METHOD
Vanilla Diffusion Models. The diffusion models essentially aim to learn the reverse process ((S)-1(S;) to restore the original data from noise ST (Ho et al. 2022). The diffusion models are usually optimized by maximizing the following probability (Kingma al. 202T):
p(sr) I1 p(s p(s)-1(s))
p(s) = log Egis or 8o)
(I)
II (Sc
A fundamental assumption of the diffusion models is that Sn to learn the diffusion processes is given in the training dataset (Kingma et al. [202]] Graham et al. 2023). However, since So represents the electron-level information of a real- world molecule, it is assumed to be unavailable in our task.
Challenges in the Diffusion Processes of DELID. In our regression setting, the objective function of the prediction models is given by:
log G) log plyls, G)
1og log] + log
Section
Section 3.2.2
Section
where y the target molecular property corresponding to the atom-level molecular descriptor G, and s is hidden electron-level representation about G. However, calculating p(s]G) based on the vanilla diffusion models is not feasible because the ground truth is unknown in our task. Hence, our proposed DELID adopts self-supervised diffusion process for learning the distribution of Sp, even though is not given in the training process, which will be described in the following subsections. We will also mathematically show how we can approximate p(s[G) based on the diffusion process on p(G) under some mild conditions on set of substructures GT of the original molecule G.
3.1 DELID: DECOMPOSITION RECOMPOSTIONNSSPERVIISS SUPERVISED ELECTRON-LEVE INFORMATION DIFFUSION
Experimental observations on the atomic systems contain measurement noise originated from the uncertainty of the electronic distributions (Robertson) 1929 Najm et al.) 2009). In physical science, quantum mechanical calculations have been widely used quantify the uncertainty of the electronic distributions (Engel & Dreizler [001]] Parr & Yang 1995). Following the convention in physical science, DELID employs the electron-level information calculated by the quantum mechanical cal- culations as supplementary information to correct the measurement noise as:
= S(G) + (bsss
where y the target property of a molecule G. f is structure encoder for the atom-level molecular descriptor G, and ub function to estimate quantum mechanical noise from the electron-level information which is calculated by the quantum mechanical calculations.
To predict the target molecular property y based on Eq. (3), DELID implements f as a deterministic function based on 2D-GNNs and v a stochastic function derived from parameterized normal distribution N (V a where My = (s: G) and oy = fg.o G) are parameterized mean and standard deviation, respectively. However, since s is not accessible in real-world complex and large molecules, devised the seff-supervised diffusion process to estimate unknown from given atom-level molecular descriptor G. In the following sections, we will formally define the self- supervised diffusion process to estimate unknown for (s;; G).
3.2 SELF-SUPERVISED DIFFUSION PROCESSES
3.2.1 DIFFUSION PROCESS ON MOLECULAR GRAPHS
The purpose of the self-supervised diffusion of DELID is to learn the dlectron-award representation s without labeled data for training s. To this end, we define a diffusion process of the molecule G that will be used to guide the diffusion process the unknown The lower bound of p(G) is
4

---

Published as a conference paper at ICLR 2025
given by the variational diffusion model (Kingma et al. 2021) as: log p(G) (GG) [logp(Go[G:]] Dx Dx.(o(Gr//Go)lo((G))))
(4)
where (-I(- is the KL divergence, Go is the latent embedding that follows the distribution of the input atom-level molecular graph G, and Gr is the noised data of G. Note that the diffusion process on G does not guarantee Go x G in our problem setting because the entire model is finally trained to maximize log p(y, s, G) instead of log p(s, G), shown in Eq. (2).
The main difference between the vanilla diffusion models and our proposed diffusion process lies in the assumption on Gr, i.e., vanilla diffusion models assume GT as random noise drawn, whereas DELID defines GT as atom-level decomposed substructures of G. Formally, in DELID, GT (Vr,Ut, AT, Xr, RT) is defined as a graph of graphs, where VT {Fi. F2, Fx} is set of decomposed substructures of G. UT = {E,, E2, ... } is a of edges within each substructure, AT is an adjacency matrix representing Uz, Xy is the atom-feature matrix of the atoms in each substructure, R is the edge-feature matrix of Uy. The diffusion probability on GT is parameterized by GNN to consider the interactions between the substructures and their electron-level information.
Definition 1. Complete graph decomposition. A graph decomposition is complete for G if the decomposed substructures of G, denoted by F2. FK, satisfy F1 FgU..U FK = and Fi F n ... n Fr where K is the number of decomposed substructures.
DELID assumes that Gr is decomposed substructures generated through complete graph decom- position defined by Definition 1. The essential property of the complete graph decomposition is that the nodes in G' are preserved through the overall graph decomposition process, i.e., Xy is preserved through the overall graph decomposition process. Hence, we can consider the node-features of the graphs as constant values in the diffusion process on G. For this reason, the structural differences through the diffusion process are only in the the latent adjacency matrix A, and latent edge-feature matrix R, of the latent graph G,. However, R, is deterministically calculated by A, for the given initial ddge-feature matrix Rg because R, is matrix consisting of the rows of Rg selected by A,- Therefore, if Gy is generated through the complete graph decomposition, the diffusion process on G can be rewritten based on A, E [0, 1] 1 for the given Rg as:
log I(g 2 EgiA log ((a Ra]] - Dxu(((Ar AAo; Ro) (p(AT;Ro)
T - RelA (ga Aa; R [i(|]|(AA... A Ro))].
(5)
DELID assumes Ar,iJ ~ N(uusss olng) for t E 2..,.... T 1}. where Arss is the (i, element of A,, and N (Pr,s.. o..,,) is normal distribution parameterized by Pr,. and oriij- How- ever, since Ag and AT, which are the adjacency matrices of Go and GT respectively, should be in (0, 13111> MV, DELID assumes At...j ~ Bernoullipp.) parameterized by Pe for t E (0. T).
Decomposing G into GT. DELID employs a chemically-infommer molecular decomposition method based on extended functional groups (EFGs) (Lu et al. (222) as an implementation of the complete graph decomposition. The EFG- based method has two benefits: 1) It can generate chemi- cally valid substructures. 2) The size of the substructures is automatically adjusted based on chem- ical knowledge. In chemical science, several other molecular decomposition methods are available for molecular decomposition, such as the junction tree (Jin et al. (008)) and the BRICS decomposi- tion (Liu et al. 2017). However, DELID uses the EFG--based molecular decomposition due to the following two benefits of the EFG-based molecular decomposition: 1) it is an efficient complete graph decomposition that generates chemically-valie subgraphs. and 2) can capture commonly appeared large molecular substructures beyond the traditional small functional groups. Appendix Cempiricall shows the benefits of the EFGG-ased molecular decomposition over other molecular decomposition methods in molecular property prediction.
3.2.2 CONDITIONAL DIFFUSION PROCESS ON ELECTRON LEVEL INFORMATION (p(s)G))
DELID calculates the unknown electron-aware latent representation s through a diffusion process guided by the diffusion process on G. log G) is given by the variational diffusion model as:
log G) log pugr(s]/))
log p(spp11155 (s:).
(6)
5

---

Published as a conference paper at ICLR 2025
where So is the latent embedding representing the ground truth electron-level information and sy noised electron-level information corresponding to Gr. Recall that we define ST as pre-calculated electron-level features (properties) of the decomposed substructures Gr of the input molecule G instead of simple random variable drawn from the standard normal distribution. Detailed descrip- tions to obtain ST will be presented in Section 3.3
We can derive a computable lower bound of the second term in Eq. 6 by marginalizing it with respect to the latent graph Gt-1. The lower bound of the second term is given by:
Σ log 1ss
Σ Eac. 1160 [logp(s-- Gt-1)] t=1
Σ
DKI.
(7)
If GT is generated through the complete graph decomposition, the lower bound of log p(s can also be rewritten based on A₁ and Ro as:
log p(s](( log Ro)
BylAr- 1A Ac-1;
(8)
T Σ Dxa Ro) Ro) Ip(A: Ro)).
The full derivation of the conditional diffusion process on is provided in Appendix B In the conditional diffusion process, DELID assumes that p(s:- 1ss, 11-1; Ro) follows the parameterized normal distributions for t E {2, 3, 3.... 7), follow the parameterized Bernoulli distributions.
while p(So|S1, Ao; Ro) and p(sr AAn; Ro) are assumed to
important to note that the conditional representation of the diffusion process on s described above shows that we can calculate the lower bound of log p(s G) without the ground truth values of s. Furthermore, the conditional representation also demonstrates that we can maximize log p(s[G) by minimizing the KL divergence between q (A,, AA,; Ro) and p(A,-1|Ser Ro) so that the diffusion process conditioned by s, follows the diffusion process G.
3.3 A RETRIEVAL PROCESS FOR OBTAINING ST IN CONDITIONAL DIFFUSION PROCESS
In Section 3.2.2
formulated the conditional diffusion process, which can be performed without the ground truth s. However, we still need the ground truth ST to perform the conditional diffusion process, since DELID defined ST in Eq. decomposed substructures G7 instead of the simple random noise. In this section, we will present a retrieval process of DELID to obtain Sy without expensive quantum mechanical calculations.
as the pre-caleulated electron-level features of the
Formally, ST is defined as embedding vector calculated by a trainable neural network for input matrix Q E R2 xm containing electron-level features about the K decomposed substructures, where the k-th row of Q, denoted by Qg, the calculated m--iimensional electron-level features of the k-th substructure F& Vr- A straightforward way to obtain Q for calculating ST is to execute the quantum mechanical calculations for each FR. However, researchers in physical science have constructed public chemical databases that provide the electron-level features of small molecules via terehhthouuhhutt quantum mechanical calculations (Ramakrishnan et al. 2014 Hoja et al.) 2021 Kim et al. 2019), and the public chemical databases already provide various electron-leved features for most possible small molecules. Hence, DELID leverages the readily accessible databases by taking the pre-caleulated electron-level features of the decomposed substructures F1, FK based on a graph matching method for the public chemical databases.
More precisely, Q2 is determined through the following retrieval process for a given public chemical database D Dym, such QM9 (Ramakrishnan et al., 2014) and PubChemQC (Nakata & Shimazaki 2017) datasets. The retrieval process is given by:
Q Spm,i'
(9)
where S.m..' is the pre-calculated electron-level features electronic energies) of the i- small molecule Gpm,i. in Dam, and an index i is calculated by:
arg max E (1Q.......PPmm.
o(Fx, Gym..)s
(10)
é 6:9 0: + the Tanimoto similarity metric (Bajusz al. 2015) to calculate the similarity between two molecular graphs Fx and Gym, in a graph domain g. In other words, for a decom- posed substructure Ft. assign the pre-calculated electron-level features of molecule in Dym

---

Published as a conference paper at ICLR 2025
whose Tanimoto similarity with Fk is the highest. By performing the above retrieval process for all FR E (F,, :..;..; FKJ. we can get Q for constructing Sr without expensive quantum mechanical calculations. In the implementation of DELID, we used the QM9 dataset as D Dam and transferred 15 energy- and polarity- electron-level features for generating Q2-
4 EXPERIMENTS
We compared the prediction capabilitiis of DELID with those of state-of-the-art methods on various benchmark molecular datasets containing experimentally collected molecules and their properties. We focused on evaluating the prediction capabilities of DELID on the experimentally generated datasets rather than simulated datasets (e.g., the QM9 dataset) due to the following reasons: 1) The simulated datasets are not suitable to evaluate the prediction capabilities of machine learning methods in real-world chemical applications because most molecules in the simulated datasets are too simple and small (Ramakrishhan et al. al.) 2014 [Hoj et al.] (021). 2) Unlike the simulated datasets, heterogeneous and out-of-distribution molecules are common in real-world chemical applications, and we will discuss about this difference in Section 3)) Experimental datasets containing the measurement noises from the uncertainty of the electronic distributions are closer to real-world
nature than the simulated datasets (Wu et al. 2018: Joung et al. 2020).
Although we focused on evaluating the prediction capabilities of DELID on the experimental datasets, DELID also showed the prediction accuracy comparable to state-of-the- of-the-art methods on a large simulated dataset (Appendix H). In addition to the evaluation on the large simulated dataset, we conducted additional experiments to evaluate the prediction performances of DELID for classi- fication tasks (Appendix [ and for different molecular scales of aam (Appendix T).
Datasets. We employed nine benchmark molecular datasets constructed by real-world chemical ex- periments. The benchmark molecular datasets were selected from well-known databases in molec- ular science (Wu et al. 2018, Wu & Weil 2018 Mendez et al., 2019 Joung et al. 2020). For com- prehensive evaluations, we selected the benchmark molecular datasets from four different chemical applications: physicochemistry. toxicity, pharmacokinetics, and optics. The characteristics of the benchmark molecular datasets are summarized in Appendix [
Competitors. We categorized competitor methods into three classes according to commonly used molecular descriptors: molecular fingerprint, 2D molecular graph, and 3D molecular graph. 1) For the molecular fingerprints, we generated three XGBoost (Chen & Guestrin 2016) based ensemble methods called XGB-Mor, XGB-FC, and XGB-MK that predict target molecular properties from input Morgan (Mor) (Rogers & Hahn 2010). functional-class (FC) (Rogers & Hahn 2010). and MACCS Key (MK) (Singh et al.] (2009) fingerprints, respectively. Even though the XaBe ensemble methods with the molecular fingerprints are simple and trivial, they have shown state-of-
the-art prediction accuracy in various chemical applications (Ding et al., 2202 Li et al.) (023b)) 2) For the 2D molecular graph, we employed five GNNs: GIN (Xu et al. 2018), EGCN (Tailor et al. 202T), MPNN (Gilmer et al. 2017). D-MPNN (Yang et al. 2019). UniMP (Shi et al. (021), and AuFP (Xiong et 2019). 3) Although 3D-GNNs are not applicable to the experimental molecular datasets, we calculated the 3D atomic coordinates based on FFSEC and evaluated five 3D-GNNs for the input FFSEC-generated 3D graphs. The following 3D GNNs were employed as competi- tor methods: SchNet (Schütt et al. 2017). DimeNet++ (Gasteiger et al. 2020). PhysChem (Yang et al.
2021 M3GNet (Chen & Ong 2022), FAENet (Duval al. 2023). and ConAN (Nguyen 2024 However, we were not able to exccute or evaluate several 3D BD-GNNs (Schütt et al.
ct al.
2021
Gasteiger al., 2021 Liao & Smidt 2022) due to -fe-of-memory problems or additional requirements on input data. Appendix Elrovidee brief descriptions of the competitor methods.
We used MPNN and GIN for the GNN-based embedding network of G and GT in DELID, respectively. The hyperparameters of DELID and competitor methods were op- timized by grid search on commonly used hyperparameter sets. Also, we followed the orig- inal implementation of the competitor methods to set method specific hyperparameters. The hyperparameter settings of DELID for each benchmark datasets are given in Appendix [F] For the information retrieval of DELID in Section 3.3 we used the QM9 dataset (Ramakrishnan jet al., 20144) generated by a high-throughput quantum mechanical calculation on small organic molecules. Instead of the original QM9 dataset, we used a subset of the QM9 dataset contain- ing the molecules with maximum six atoms as Dym because too large molecules are redundant in matching the decomposed small substructures. The source code of DELID is publicly available at https: // httpg/ttbb...ooeeeeeeeeeeeeeeeee wttp://wo//ggooommmoooo
Implementations.

---

Published as a conference paper at ICLR 2025
Table I: The -to-sc of the competitor methods and DELID on benchmark experimental molec- ular datasets. N/R and N/A mean a negative Re-score indicating failure of regression and an
execution failure related to out of memory or numerical errors, respectively.
Input Tyne
Method
Lipop
ESOL.
ADMET
IGC50
[,350
LC50
L.C.
CHLIX.
CH-AC.
XGB Mor
0.531
0.659
0.717
0.621
0.390
0.497
0.505
N/R
N/R
Molecular
(0/24)
(0.45)
(0.21)
(0.40)
(0.13))
(0.16))
(0018)
Fingerprint
XGB-FC
0.578
(0018)
0.686
(0053)
0.720
(0.00)
0.628
(0023)
0.501
(0022)
0,519
(0,05))
0,503
(0007)
N/R
N/R
XGB-MK
0,542
0.764
0.761
0.680
0,486
0.526
0.471
N/R
N/R
(004))
(0.00)
(0011)
01
(0.02))
(0019)
SchNNet
0.881
0.834
0.765
0.527
D.467
0.456
0.713
(0021)
(0020)
(00122)
(0024)
00.02)
(0.025)
(0624)
(0459)
(0/37))
DimeNet++
N/R
0.878
N/R
0.779
0,541
N/A
0.352
N/A
N/A
(0.025)
(0.119)
(4.45)
(0.000
3D Molecular
PhysChem
0.694
(0024)
(0.23))
0.848
N/A
0.814
(0017)
0,511
N/A
N/A
N/A
N/A
Graph
M3GNet
N/A
0.857
N/A
0.697
0,531
N/A
N/A
N/A
N/A
(0.025)
(0,129)
(0.034)
FAENet
0.670
0.869
0.788
0.708
0.474
D.528
0.437
0.437
(0/25)
(0.033)
10,000
(0015)
(0.023)
(0.094)
(0625)
(0.13))
(OL13))
ConAN
0.738
0.909
0.845
0.819
0,531
0.572
0.466
0.405
0.388
(0016)
(0.15)
(M62))
(0,00)
(1 (4.5
(0,000)
(0/08))
(0.0K)
COLIIS)
GIN
0,808
0.807
0.792
0.545
0.525
N/R
(0.09))
(0007))
(0023)
(0015)
(1015)
(0.00)
(0683))
(00000
EGCN
0,716
0.822
0.814
0.777
0,550
0,503
0.497
0.226
N/R
(0/21)
(0.029)
(0,21)
(0/22))
(1.15))
(0.000)
(0698))
(0
MPNN
0.727
0.810
0.801
0.764
0,502
0.487
0,46
0.385
(0.022)
(0.00)
(0027))
(00022)
(0425))
2D Molecular
D-MPNN
0.726
0.879
0.820
0.787
0,521
0,566
0.494
N/R
N/R
Graph
(0423))
(0.013)
(0.018)
(0.055)
(0.013)
(0.0M))
(D.11)
UniMP
0.810
0.817
0.756
0.512
D.531
0.478
N/R
(0/010)
(0.026)
(0.018)
(0.04))
(0.025)
(0.078)
(0:02)
(0051)
AnFP
0,710
0.909
0
0,807
0,513
0.642
0.456
0.441
0.296
(0/221)
(0.11))
(0025)
(0.013)
(1
(0.079)
(0691)
(0,09))
60.770
DELID
0,782
0.912
0.834
0.844
0.566
0.644
0,532
0.886
0.885
(0.14)
(0.42)
(0,96)
(0,088)
4.1 MOLECULAR PROPERTY PREDICTION ON REAL- WORLD COMPLEX MOLECULES
We measured the - of DELID and the competitor methods on the nine benchmark molecular datasets. In this experiment, focused evaluating the representation capabilities of DELID and the competitor methods on the experimental datasets containing experimentally collected complex and large molecules rather than the simulated datasets. Note that the R R2-score is a normalized metric to measure the regression accuracy. For all datasets, the R2-scores were measured by the 5- fold cross-validation. Table shows the measured R2-scores on the benchmark molecular datasets. Although the 3D-GNNs were sophisticatedly designed to capture the inter-atomic interactions in the 3D geometry and showed accuracy improvements on simulated molecular datasets, they were not executable on many benchmark molecular datasets containing complex and large real-world molecules. However, DELID showed reliable execution performances and achieved state-of-the-art prediction accuracy on most benchmark datasets.
One of the main limitations of the 3D-GNNs is that their generalization capabilities are limited on large molecular graphs due to the impractical time complexities and the easily overfitted embedding schemes (Li et al. 2023a).. In the experiments, the R2-scores of the 3D-GNNs were also lower than those of the 2N- on the Lipop and LMC-H datasets containing many large molecules, even though the 3D-GNNs employ sophisticatedly designed embedding methods with more model parameters. This experimental results directly show the limitations of the 3D-GNNs on real-world complex and large molccules. However, DELID achieved the highest R2-scores on the Lipop and LMC-H datasets. These results show the practical potential of DELID in real-world chemical appli- cations where complex and large molecules commonly appear.
We investigated the reasons for the failure of the competitor methods on the CH-DC and CH-AC datasets. Fig. shows the data distributions of the CH-DC and CH-AC datasets. We also presented the data distributions of the ESOL and IGC50 datasets where most competitor methods success- fully learned the relationships between the molecules and the target properties. Additionally, we plotted the data distribution of the QM9 dataset together with the data distributions of the bench- mark datasets for comparative analysis on the simulated and experimental molecular datasets. The molecules in the datasets were projected to the 2D space through randomly initialized MPNN to preserve the density of the data.
We observed two major results in Fig. 3 biased, whereas the data distribution of the experimental datasets covered larger chemical spaces
First, the data distribution of the simulated dataset was
8

---

Published as a conference paper at ICLR 2025
Benchmark datasets that were easy to regression
Benchmark datasets that were difficut to regression CH-DC
ESDL
IGC50
CHiAC
0.6
025
Q.4
0.4
0.000
dalsser
0.
0.
-3 11055
5MS
-3 050
0.5
-3 075
030
-0.
-1.00
-125
Figure 3: Data Distributions of the benchmark molecular datasets. Black: molecular emebeddings of a simulated dataset called the QM9 dataset. Red: molecular embeddings of an experimentally generated dataset. Note that the data distribution of the QM9 dataset can be plotted differently depending on the scale of the data distribution of the ESOL, IGC50, CH-DC, and CH-AC datasets.
Lipap
ESCL
ADMET
IGC50
LDS0
0.70
1:15
030
6:00
0.52
0.55
880
0.00
0.75
8.70
0.35
0.33
10
45
17
GIN
EGCN
MPNN
UniMp
AMF
DELID
Figure 4: The R2-scores for different sizes of the training data on the Lipop, ESOL, ADMET, IGC50, and LD50 datasets. X-axis: Ratio of the training data. Y-axis: Measured -so-ee
with many outlier molecules. This observation justifies why evaluating machine learning methods on experimentally collected molecular datasets is crucial in validating their practical availability in real-world chemical applications. Second, the CH-DC and CH-AC datasets where most competitor methods failed to predict the target properties covered extremely larger chemical spaces compared to the QM9, ESOL, and IGC50 datasets. Nevertheless, DELID successfully captured the underlying relationships between the molecules and their optical properties on the CH-DC and CH-AC datasets.
4.2 PREDICTION ACCURACY ON VARIOUS SIZES OF TRAINING DATA
Since conducting chemical experiments obtain the experimentally labeled data is time-consuming and labor-intensive, the lack of training data remains one of the main challenges of machine learn- ing in chemical applications (Dou al. 2023a). As described in Section B.3] DELID is flexible in incorporating external electron-level features into molecular representation learning, which is ben- eficial in constructing an accurate prediction model on small training datasets. In this experiment, we compared the Re=scores of DELID and the competitor methods over different sizes of training datasets to demonstrate the effectiveness of DELID on small training datasets.
Fig. 4 shows the - of DELID and the competitor methods for different sizes of training datasets. We measured the Re-scores on the Lipop, ESOL, ADMET, IGC50, and L.550 datasets in which DELID and most competitor methods achieved the R greater than 0.5. However, We did not compare the R2-scores of the XGB- and 3D structure-bast them failed small training datasets. Obviously, we were able to observe that the prediction ac- curacy tends to improved as the size of the training dataset increases for most methods. Among the competitor methods, AtFP showed comparable generalization capabilities to those of DELID on the ESOL and ADMET datasets. However, the accuracy improvements by AulF were marginal compared to GIN, EGCN, MPNN, and UniMP on the Lipop. IGC50, and LDS0 datasets. By con- trast, DELID consistently showed better generalization performances regardless of the benchmark datasets compared to GIN, EGCN, MPNN, and UniMP. Furthermore, DELID outperformed AttFP on the Lipop, IGC50, and LD50 datasets. These experimental results show the practical potential of DELID in real-world chemical applications, which usually suffer from the lack of training data.
methods because most of
4.3 PREDICTION ACCURACY FOR DIFFERENT ELECTRON-LEVE FEATURES
The choice of the electron-level features in the information retrieval can affect the representation capabilities of DELID. To evaluate the prediction capability of DELID for different electron-level features, measured the R2-scoees of DELID for different electron-level features provided in the

---

Published as a conference paper at ICLR 2025
Table 2: The R2 -c-scores of DELID for different Electron-leved features.
Category of Features
Lipop
ESOL.
ADMET
IGC50
1,250
LAC50
I.C--H
CH-DC
CH-AC
Energy-related features
0.776
0.911
0.835
0.842
N/R
0.642
0.538
0.784
0.832
(90055
(0.013)
(0006)
(0.012)
(0,080)
(0029)
(0.149)
(0.09)
Polarity-related features
0.778
0.908
0.755
0.840
0,569
0.632
0.525
0.886
0.877
(0.00)
(U011)
(0 1989
(0.016)
(0120)
(0001)
(BB15)
(0008)
(1
Other features
0.785
0.910
0.711
0.849
0.556
0.632
11,33
0.773
0.861
(0.012)
(0.012)
(0.217)
(0.012)
(0.015)
(0.073)
(00244)
(0.147)
(0.010)
All features (DELID)
0.782
0.912
0.834
0.844
0.566
0.644
015532
0.886 (0.035)
0.8K5
(0.013)
(0.014)
(0,042)
(0.005)
(0.024)
(0.065)
(0.048)
(0.023)
Table 3: The R scores of DELID and its three variants in the Ablation study.
Method
Atom-Level Informazion
ElectronLLevel Information
Information Diffusion
Lipop
ESOL
ADMET
IGC50
LD50
LC30
LMC-H
CH-DD
CH-AC
DELIDa:
X
X
01008)
0.727
(B0-2)
0.810
0.801
00.03)
(8407)
0.764
0.502
(6022)
(8.109)
0.487
(9002)
0.461
0.385 01433)
N/R
DELID,
0,220
0.445
0.537
0.419
0.200
0.226
0.243
0.500
0.548
(0.54)
(0260)
(0024)
(3.54)
(5043)
(8E35)
(U322
DE
X
0.908
(8.06)
DELID
0.782
0.912
0.834
0.844
0.566
01664
0.532
0.886
0.885
(0.83)
(8/044)
(0.44))
(8J86)
(8024)
(8.043)
(8.44)
(4835)
(6823)
QM9 dataset. We categorized the provided electron-level features into the following three classes:
1) energy-related features, 2) polarity-related features, 3) other features. Note that DELID used all electron-level features in representation learning. Table[[show the Re-scores of DELID for differ- ent kinds of the electron-level features. DELID showed significant improvements on the ADMET and CH-DC datasets by using the energy- and polarity-relate clectron-level features respectively, because of the direct relationships between the target molecular properties and these electron-level features (Dong et al. 2018: Joung et al., 2020). However, DELID exploiting all electron-level fea- tures showed the prediction accuracy comparable the best models for all datasets, and this result shows that DELID can select important electron-level features for given target task.
4.4 ABLATION STUDY ON DELID
We conducted an ablation study to evaluate the effectiveness of the electron-level information and self-supervised diffusion of DELID. We generated three variants of DELID for the ablation study. 1) DELID,. model that learns the molecular representations using only the atom-level molecular descripior G, which is the same as MPNN. 2) DELID,, learns the molecular representations using only the fragmented information defined as GT and ST. 3) DELID qm predicts the target molecular property based on Eq. (3) without the self-supervised diffusion on s, i.e., DELID pm predicts the tar- get molecular property by y = f(G)+ (G, Sy; G). Table 3 shows the R2-scores of DELID and its three variants for the ablation study. The R2-scores were measured by the 5-fold cross-validation The RC-scores of DELID,r shows that the incomplete electron-level features is not sufficient to predict the molecular properties of the original molecules. The accuracy improvements were markable DELID gm compared to DELID,, and DELID and this result shows that integrating the atvel and glectron-leved information is crucial for accuracy molecular property prediction. However, we were able observe further improvements by DELID for all benchmark molecular datasets. These results demonstrate the effectiveness of the electron-aware molecular representation leaming based the self- supe-supe diffusion.
5 CONCLUSION
This paper proposed DELID to learn informative molecular representations of real-world complex and large molecules based on the self-supervised diffusion process on the electron-level informa- tion. In this paper, mathematically showed that DELID can learn the electron-aware molecular representations by approximating the diffusion process started from the fragmented electron-level information to the diffusion process started from the decomposed substructures, even though the complete electron-level information about the molecules is not known. By employing the self- supervised diffusion, DELID achieved state-of-the-art prediction accuracy on extensive benchmark datasets containing experimentally collected molecules and their molecular properties. The exper- imental results showed the practical potentials of DELID in real-world chemical applications. As a future work, an efficient method to construct the calculation databases an accurate calculation level for the coarse-graining representation learning of DELID needs to be considered to provide more accurate electron-level features to DELID.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This work was supported by Ministry of Trade, Industry and Energy (No. TS241-10R) and National Research Foundation of Korea (NRF) funded by Ministry Science and ICT (NRF- (22M336A1103022 RS-2024-004069855)
REFERENCES
Dávid Bajusz, Anita Rácz, and Kåroly Héberger. Why tanimoto index appropriate choice for fngerprinttbased similarity calculations? J. Cheminformatiis, 7(1):1-13, 2015.
Camille Bilodeau, Wengong Jin, Tommi Jaakkola, Regina Barzilay, and Klavs F Jensen. Generative models for molecular discovery: Recent advances and challenges. Wiley Interdiscip. Rev. Comput. Mol. Sci., (2(5) 2022.
Ella Bingham and Heikki Mannila. Random projection dimensionality reduction: applications to image and text data. In ACM SIGKDD, pp. 245-250, 2001.
Chenjing Cai, Shiwei Wang, Youjun Xu, Weilin Zhang, Ke Tang, Qi Ouyang, Luhua Lai, and Jian- feng Pei. Transfer learning for drug discovery. Med. Chem., 63(16))88,338694, 2020.
Chi Chen and Shyue Ping Ong. A universal graph deep learning interatomic potential for the periodic table. Nat. Comput. Sci., 2(11):788-7 2022.
Chi Chen, Weike Ye, Yunxing Zuo, Chen Zheng, and Shyue Ping Ong. Graph networks as universal machine learning framework for molecules and crystals. Chem. Mater, 2019.
Dexiong Chen, Leslie O'Bray, and Karsten Borgwardt. Structure-aware transformer for graph rep- resentation learning. In ICML, pp. 3469-3489. PMLR, 2022.
Tianqi Chen and Carlos Guestrin. Xgboost: scalable tree boosting system. In ACM SIGKDD, pp. 785-794, 2016.
William Dawson, Augustin Degomme, Martina Stella, Takahito Nakajima, Laura E Ratcliff, and Luigi Genovese. Density functional theory calculations of large systems: Interplay between frag- ments, observables, and computational complexity. Wiley Interdiscip. Rev. Comput. Mol. Sci., 12 (3)e15774, 2022.
John S. Delaney, Esol: Estimating aqueous solubility directly from molecular structure. J. Chem. Inf. Comput. Sci., 44: 44:1000-1005, 2004.
Yi Ding, Minchun Chen, Chao Guo, Peng Zhang, and Jingwen Wang. Molecular fingerprint-based machine learning assisted qsar model development for prediction of ionic liquid properties. J. Mol. Liq., 326: 15212, 2021.
Jie Dong, Ning-Ning Wang, Zhi-Jiang Yao, Lin Zhang, Yan Cheng, Defang Ouyang, Ai-Ping Lu, and Dong-Sheng Cao. Admetlab: platform for systematic admet evaluation based on a compre- hensively collected admet database. J. Cheminformatics, 10(1):1-11, 2018.
Bozheng Dou, Zailiang Zhu, Ekaterina Merkurjev, Lu Ke, Long Chen, Jian Jiang, Yueying Zhu, Jie Liu, Bengong Zhang, and Guo-Wei Wei. Machine learning methods for small data challenges in molecular science. Chem. Rev., 123(13):8736-877 2023a.
Bozheng Dou, Zailiang Zhu, Ekaterina Merkurjev, Lu Ke, Long Chen, Jian Jiang, Yueying Zhu, Jie Liu, Bengong Zhang, and Guo-Wei Wei. Machine learning methods for small data challenges in molecular science. Chem. Rev., 123(13):8736-87) 2023b.
Alexandre Agm Duval, Victor Schmidt, Alex Hernandez-Garre Garcia, Santiago Miret, Fragkiskos D Malliaros, Yoshua Bengio, and David Rolnick. Faenet: Frame averaging equivariant gnn for materials modeling. In ICML., pp. 9013-9033. PMLR, 2023.
Eberhard Engel and Reiner M Dreizler. Density functional theory. Theor Math. Phys., pp. 351-399, 2011.
11

---

Published as a conference paper at ICLR 2025
Artem Fediai, Patrick Reiser, Jorge Enrique Olivares Peña, Pascal Friederich, and Wolfgang Wenzel. Accurate gw frontier orbital energies of 134 kilo molecules. Sei. Data, 10(1))581, 2023.
Shikun Feng, Yuyan Ni, Yanyan Lan, Zhi-Ming Ma, and Wei-Ying Ma. Fractional denoising for 3d molecular pre-training. In ICML, pp. 9938 9961. PMLR, 2023.
Johannes Gasteiger, Shankari Giri, Johannes T Margraf, and Stephan Günnemann. Fast and uncertainty-aware directional message passing for non-equilibrium molecules. arXiv preprint arXiv:201I.14115 141115 2020.
Johannes Gasteiger, Florian Becker, and Stephan Günnemann. Gemnet: Universal directional graph neural networks for molecules. NeurlPS,
2021.
Justin Gilmer, Samuel Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In ICML, 2017.
Rhys EA Goodall and Alpha Lee. Predicting materials properties without crystal structure: Deep representation learning from stoichiometry. Nat. Commun., 11(1):1-9, 2020.
Mark S Graham, Walter HL Pinaya, Petru-Daniel Tudosiu, Parashkev Nachev, Sebastien Ourselin, and Jorge Cardoso. Denoising diffusion models for diitr--distributionn detection. In CVPR, pp- 2947-2956, 2023.
Piotr S Gromski, Alon B Henson, Jaroslaw M Granda, and Leroy Cronin. How to explore chemical space using algorithms and automation. Nat. Rev. Chem., 3(2):119-13))) 1281 2019.
Jonathan Ho, Chitwan Saharia, William Chan, David Fleet, Mohammad Norouzi, and Tim Sali- mans. Cascaded diffusion models for high fidelity image generation. Mach. Learn. Res., 23(1): 2241-2281, 2022.
Johannes Hoja, Leonardo Medrano Sandonas, Brian G Ernst, Alvaro Vazquez-Mayagoitia. Robert A DiStasio Jr, and Alexandre Tkatchenko. Qm7-x, a comprehensive dataset of quantum--echanee
properties spanning the chemical space of small organic molecules. Sei. data, 8(1):43, 2021.
Scott A Hollingsworth and Ron O Dror. Molecular dynamics simulation for all. Neuron, 99(6): 1129-1143, 2018
Chin-Wei Huang, Jae Hyun Lim, and Aaron C Courville. A variational perspective on diffusion- based generative models and score matching. NeurIPS, 34:22863-22876, 2021.
Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-sofftax arXiv preprint 1111.v11111.11111 2016.
Dipendra Jha, Kamal Choudhary. Francesca Tavazza, Wei-keng Liao, Alok Choudhary, Carelyn Camphell, and Ankit Agrawal. Enhancing materials property prediction by leveraging computa- tional and experimental data using deep transfer learning. Nat. Commun. 10(5316), 2019. doi: 10.1038/441467-0199 13297-w.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Junction tree variational autoencoder for molecular graph generation. In ICML, pp. 2323-2332. PMLR, 2018.
Joonyoung Joung, Minhi Han, Minseok Jeong, and Sungnam Park. Experimental database of optical properties of organic compounds. Sei. Data, 7(1):295, 2020.
Scojin Kim, Jachyun Nam, Junsu Kim, Hankook Lee, Sungsoo Ahn, and Jinwoo Shin. Fragment- based multi-view molecular contrastive learning. Workshop on "Machine Learning for Mate- rials" ICLR, 2023.
Sunghwan Kim, Jie Chen, Tiejun Cheng, Asta Gindulyte, Jia He, Siqian He, Qingliang Li, Ben- jamin A Shoemaker, Paul A Thiessen, Bo Yu, al. Pubchem 2019 update: improved access to chemical data. Nucleic Acids Res., 47(D)) 102--1109, 2019.
Diederik Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. NeurIPS, 34:21696-21777, 2021.
Variational diffusion models.
Diederik Kingma. Auto-encoding variational bayes. arXiv preprint arXiv: 2013.
12

---

Published as a conference paper at ICLR 2025
Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional net- works. ICLR, 2017.
Ondrej L Krivanek, Matthew F Chisholm, Valeria Nicolosi, Timothy Pennycook, George J Corbin, Niklas Dellby, Matthew F Murfitt, Christopher S Own, Zoltan S Szilagyi, Mark P Oxley, et al. Atom-by-atom structural and chemical analysis by annular dark-field electron microscopy. Na- lure, (644444888:557--55777 2010.
Kangming Li, Brian DeCost, Kamal Choudhary, Michael Greenwood, and Jason Hattrick- Simpers. A critical examination robustness and generalizability machine learning prediction of mate- rials properties. Npj Comput. Mater, 9(1):55, 2023a.
Lujun Li. Yiming Zhao, Haibin Yu, Zhuo Wang, Yongjia Zhao, and Mingqi Jiang. An xgboost algorithm based on molecular structure and molecular specificity parameters for predicting gas adsorption. Langmuir, 39(19):6756-676) 2023b.
Yi-Lun Liao and Tess Smidt. Equiformer: Equivariant graph attention transformer for 3d atomistic graphs. arXiv preprint arXiv:2206.11990, 2022.
Tairan Liu, Misagh Naderi, Chris Alvin, Supratik Mukhopadhyay, and Michal Brylinski. Break down in order to build up: Decomposing small molecules for fragment-basee drug design with emolfrag. J. Chem. Inf. Model., 57(4):627-63 2017.
Yunchao Lance Liu, Yu Wang, Oanh Vu, Rocco Moretti, Bobby Bodenheimer, Jens Meiler, and Tyler Derr Interpretable chirality-aware graph neural network for quantitative structure activity relationship modeling in drug discovery. In AAAI, volume 37, pp. 14356-14364 14364, 2023.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. ICLR, 2017.
Jianing Lu, Song Xia, Jieyu Lu, and Yingkai Zhang. Dataset construction to explore chemical space with 3d geometry and deep learning. J. Chem. Inf. Model., 6163):1955-1144, 2021.
Chris Maddison, Andriy Mnih, and Yee Whye Teh. The concrete distribution: A continuous relaxation of discrete random variables. arXiv preprint arXiv:1611.007/3 2016.
David Mendez, Anna Gaulton, A Patrícia Bento, Jon Chambers, Marleen De Veij, Eloy Félix, María Paula Magariños, Juan Mosquera, Prudence Mutowo, Michal Nowotka, et al. Chembl: towards direct deposition of bioassay data. Nucleic Acids Res., +7(D1):D330-D944 2019.
Nico JD Nagelkerke al. A note on general definition of the coefficient of determination. biometrika, 78(3):691-693 1991.
Habib N Najm, Bert J Debusschere, Youssef M Marzouk, Steve Widmer, and OP Le Maître. Uncer- tainty guantification in chemical systems. Int. J. Numer: Methods Eng.. 80(6-7):789-888 814, 2009.
Maho Nakata and Tomomi Shimazaki. Pubchemqe project: large-scale first-principles electronic structure database for data-driven chemistry. J. Chem. Inf. Model., 5000))50600001133300 2017.
Duy Minh Ho Nguyen, Nina Lukashina, Tai Nguyen, An Thai Le, Trungtin Nguyen, Nhat Ho, Jan Peters, Daniel Sonntag, Viktor Zaverkin, and Mathias Niepert. Structure-aware e(3)-invariant molecular conformer aggregation networks. In ICML, pp. 37736-37760, 2024.
Robert G Parr and Weitao Yang. fenntialf theory of the electronic structure of molecules. Annu. Rev. Phys. Chem., 46(1):701-728, 1995.
Raghunathan Ramakrishnan, Pavlo O. Dral, Matthias Rupp, and O. Anatole von Lilienfeld. Quan- tum chemistry structured and properties of 134 kilo molecules, Sci. Data, 1, 2014.
Lucia Reining, The gw approximation: content, successes and limitations. Wiley Interdiscip. Rev. Comput. Mol. Sci., 833):e134, 2018
Sereina Riniker and Gregory Landrum. Better informed distance geometry: using what we know to improve conformation generation. J. Chem. Inf. Model., 55(12):552-22774 2015.
Howard Percy Robertson. The uncertainty principle. Phys. Rev, 34(1):163, 1929.
13

---

Published as a conference paper at ICLR 2025
David Rogers and Mathew Hahn. Extended-connecttvvg fingerprints. J. Chem. Inf. Model., 50(5): 742-754, 2010.
Johannes Schneider and Michail Vlachos. Scalable density- clustering with quality guarantees using random projections. Data Min. Knowl. Discov. 31:972-1005, 2017.
Norbert Schuch and Frank Verstraete. Computational complexity of interacting electrons and fun- damental limitations of density functional theory. Nat. Phys., 5(10):732-735, 2009.
Kristof Schütt, Pieter-Jan Kindermans, Huziel Enoc Sauceda Felix, Stefan Chmiela, Alexandre Tkatchenko, and Klaus-Robert Müller. Schnet: continuous-filte convolutional neural network for modeling quantum interactions. NeurIPS, 30, 2017.
Kristof Schütt,, Oliver Unke, and Michael Gastegger. Equivariant message passing for the prediction of tensorial properties and molecular spectra. In ICML, pp. 9377-9388. PMLR, 2021.
Kristof T Schütt, Huziel Sauceda, P-J Kindermans, Alexandre Tkatchenko, and K-R Müller. Schnet-a deep learning architecture for molecules and materials. Chem. Phys., 148((4))241722, 2018.
Yuning Shen, Julia E Borowski, Melissa A Hardy, Richmond Sarpong, Abigail G Doyle, and Tim Cernak. Automation and computer-assisted planning for chemical synthesis. Nat. Rev. Dis. Primers, 1(1):1-23, 2021.
Yunsheng Shi, Zhengjie Huang, Shikun Feng, Hui Zhong, Wenjing Wang, and Yu Sun. Masked label prediction: Unified message passing model for semi-supervised classification. In IJCAI, 2021.
Eunji Sim, Suhwan Song, and Kieron Burke. Quantifying density errors in dft. J. Phys. Chem. Lett., 9(22):6385 0322663566662,,, 2018.
Narender Singh, Rajarshi Guha, Mare A Giulianotti, Clemencia Pinilla, Richard A Houghten, and Jose L. Medina-France Chemoinformatic analysis of combinatorial libraries, drugs, natural prod- ucts, and molecular libraries small molecule repository. J. Chem. Inf. Model., 2009.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020.
Kazu Suenaga and Masanori Koshino. Atom-by-atom spectroscopy at graphene edge. Nature, 468 (7327):1088-1090 2010.
Shyam A Tailor, Felix Opolka, Pietro Lio, and Nicholas Donald Lane. Do we need anisotropic graph neural networks? In ICLR, 2021.
Yusuke Tashiro, Jiaming Song, Yang Song, and Stefano Ermon. Csdi: Conditional score-based diffusion models for probabilistic time series imputation. NeurIPS, 34:24804-2433 2021.
Clement Vignac, Igor Krawczuk, Antoine Siraudin, Bohan Wang, Volkan Cevher, and Pas- cal Frossard. Digress: Discrete denoising diffusion for graph generation. arXiv preprint arXiv:2209. 14734, 2022.
Hongwei Wang, Weijiang Li, Xiaomeng Jin, Kyunghyun Cho, Heng Ji, Jiawei Han, and Martin D Burke.
molecule representation learning. ICLR, 2022.
Yusong Wang, Tong Wang, Shaoning Li, Xinheng He, Mingyu Li, Zun Wang, Nanning Zheng, Bin Shao, and Tie- Yan Liu. Enhancing geometric representations for molecules with equivariant vector-scalar interactive message passing. Nat. Commun. 15(1):313, 2024.
Oliver Wieder, Stefan Kohlbacher, Mélaine Kuenemann, Arthur Garon, Pierre Ducrot, Thomas Sei- del, and Thierry Langer. compact review of molecular property prediction with graph neural networks. Drug Discovery Today: Technologies, 37:1-12, 2020.
Daniel S Wigh, Jonathan M Goodman, and Alexei A Lapkin. review of molecular representation in the age of machine learning. Wiley Interdiscip. Rev. Comput. Mol. Sci., 12(5)ee 163,, 2022.
14

---

Published as a conference paper at ICLR 2025
Fang Wu and Stan Z Li. Diffimd: geometric diffusion model for molecular dynamics simulations. In AAAI, volume 37, pp. 5321-5329, 2023.
Kedi Wu and Guo- Wei Wei. Quantitative toxicity prediction using topology based multitask deep neural networks. J. Chem. Inf. Model., 58(2):520 Model,, -3,1 2018.
Zhengin Wu, Bharath Ramsundar, Evan N Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh S Pappu, Karl Leswing, and Vijay Pande. Moleculenet: benchmark for molecular machine learn- ing. Chem. Sci., 922):11--330 2018.
Zhaoping Xiong, Dingyan Wang, Xiaohong Liu, Feisheng Zhong, Xiaozhe Wan, Xutong Li, Zhao- jun Li, Xiaomin Luo, Kaixian Chen, Hualiang Jiang, al. Pushing the boundaries of molecular representation for drug discovery with the graph attention mechanism. J. Med. Chem., 63(16); 8749-8760, 2019.
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful graph neural networks? ICLR, 2018.
Riqiang Yan and Robert Vassar. Targeting the 3 secretase bacel for alzheimer's 's disease therapy. Lancet Neurol. 13(3):319-322) 2014.
Kevin Yang, Kyle Swanson, Wengong Jin, Connor Coley, Philipp Eiden, Hua Gao, Angel Guzman- Perez, Timothy Hopper, Brian Kelley, Miriam Mathea, et al. Analyzing learned molecular repre- sentations for property prediction. J. Chem. Inf. Model., 59(8):3370-3388 2019.
Shuwen Yang, Ziyao Li, Guojie Song, and Lingsheng Cai. Deep molecular representation learning 2021.
via fusing physical and chemical information. NeurIPS, 34:
Zhaoning Yu and Hongyang Gao. Molecular representation learning via heterogeneous motif graph neural networks. In ICML, pp. 25581425594, PMLR, 2022.
Ye Yuan, Jiaming Song, Umar Iqbal, Arash Vahdat, and Jan Kautz. Physdiff: Physics-guided human motion diffusion model. In CVPR, pp. 16010-16021, 2023.
Xuan Zang, Xianbing Zhao, and Buzhou Tang. Hierarchical molecular graph self-supervised learn- ing for property prediction. Commun. Chem., 6(1):34, 2023.
Viktor Zaverkin, David Holzmüller, Luca Bonfirraro, and Johannes Kästner. Transfer learning for chemically accurate interatomic neural network potentials. Phys. Chem. Chem. Phys., 25(7): 5383-5396, 2023.
Lukas Zbinden, Lars Doorenbos, Theodoros Pissas, Adrian Thomas Huber, Raphael Sznitman, and Pablo Márquez-Neila. Stochastic segmentation with conditional categorical diffusion models. In CVPR, pp. 1119-1129, 2023.
Claudio Zeni, Robert Pinsler, Daniel Zügner, Andrew Fowler, Matthew Horton, Xiang Fu, Sasha Shysheya, Jonathan Crabbé, Lixin Sun, Jake Smith, et al. Mattergen: a generative model for inorganic materials design. arXiv preprint (xXiv:2312.03687, 2023.
Lvmin Zhang, Anyi Rao, and Mancesh Agrawala. Adding conditional control text-to-image diffusion models. In CVPR, pp. 3836-3847, 2023.
Zaixi Zhang, Qi Liu, Hao Wang, Chengqiang Lu, and Chee-Kong Lee. Motif-based graph self- supervised learning for molecular property prediction. NeurIPS, 15888, 2021.
15

---

Published as a conference paper at ICLR 2025
A SAMPLING METHOD FOR PARAMETERIZED BERNOULLI DISTRIBUTION
Unlike the conventional diffusion models, DELID assumes the parameterized Bemoulli distribution on Ag and Ar because each element of them should be in the binary domain {0. }. Hence, we need generate sample from reparameterized Bernoulli distribution. By the existing work on categorical reparameterization (Jang et al.) 2016; Maddison et al. 2016), we can sample a random variable following the Bernoulli distribution under the reparameterized distribution as:
(log e log(1 e) + log ((x) - log(1 ((x))),
(11)
where a is the sigmoid function, ø E (0, 1) is trainable shape parameter of Bernoulli distribution, and UO, 1) is random number from an uniform distribution (0, 1).
B DERIVATION OF THE CONDITIONAL DIFFUSION PROCESS ON S
In this section, we present a detailed derivation of the conditional diffusion process on s. Since the second term D')[G) in Eq. (2) is not directly computable, we derive the lower bound of log p(|G() based on a diffusion process started from ST as follows.
log G)) = logp(st(G) G)
log p(s)-1|s:) Σ
(12)
t=1
We marginalize the second term in Eq.
12 for G and calculate its lower bound as follows.
p(8:-1 (Ja_
log p(
Σ log
G (10---/
{1 (a-n
(s, 1r Gr g(G--IlG))dGG-
log
p(s:-1 1t, G. Gr-1)p(G--1155
log
((Gr-I/G)dd.a (c)
Gr-1
Σ 19( poog 18s,, C2-1)) Dally(Ga=IGG))l/CGG-ass/) (13)
Finally, the lower bound of log log p(s[G) is given by:
logp(s|() 2 log (ogg(sT|G)
Σ (a.) [logp(s-- St, Gr-1)l
T
Σ
DKL Dee(g(GG--1Ghhll(/G==a))))))
(14)
If Gr is generated through the complete graph decomposition, we also rewrite the lower bound of log G(s as follows.
logr(s|() 2 logp(ss/ Ao; Ro) +
F Fy Po A:-:: Ro)]
(15)
C ALGORITHMIC DESCRIPTION OF DELID
Algorithm shows an algorithmic description of the forward and training processes DELID.
D STATISTICS OF THE BENCHMARK EXPERIMENTAL DATASETS
4shos the statistics and target molecular properties of the nine benchmark molecular datasets used for the experimental evaluations.
16

---

Published as a conference paper at ICLR 2025
Algorithm Training Process of DELID
Input:
Deraint training dataset;
Dami a dataset for the information retrieval;
uy an initial learning rate of optimizer;
Output: 0% optimized model parameters: repeat
for (G. y) E Derain do
#/ Decomposition of an atom noom-level molecular graph. F2..... Fx <- RFGDeconpost t on(G) I/ Information retrieval to obtain sr-
for k 1; K do
arg ma Pomll ((Fk, Ggm.s
& Sqm,1*
end for
// Model parameter optimization.
Calculate Jg log G) by Eqs. 4 7 Jo
end for until is converged
Table 4: Statistics and target molecular properties of the benchmark molecular datasets that contain the atom-level molecular structures and their experimentally observed target properties.
Application Category
Averuge Number
Dataset
Target Molecular Property
of Molecules
of Atoms
Lipop Mendez ct al 2019
Lipophilicity
4,200
48.51
Plysicochemistry
ESOL
2009
Aqueous solubility
1.128
25.64
ADMET [Dong et a
2018
Aqueous solubility
4,801
26.76
IGC50 Wu & Weil 2018
Tetrahymenapyiffrnit toxicity
1.791
19.29
Toxicity
1.C50 (il &
Fathead minnow toxicity
822
22.32
LD50 Wa & Wei
2018
Oral rat toxicity
7,412
31.30
Pharmacokinetiis
LMC-H Mendez
2019
microsomal clearance in human
5,347
54.53
Optics
CH-DC
Joung
2020
Absorption in Dichloromethane
2,429
28.95
CH-AC
Toung et al.
2020
Absorption max in Acctonitrile
1,781
29.71
E COMPETITOR METHODS
In the experiments, we compared the prediction capabilities of DELID with a baseline tree method and ten state-of-the-art GNNs, which have been widely used in chemical applications. The competi- tor methods are briefly described as:
XGB-Mor: XGBoost (XGB) (Chen & Guestrin 2016) is tree-based gradient boosting
model, and showed state-of-the-arr performances in various seientific applications. For the experimental evaluations, we generated XGB-Mor that predicts the target molecular properties for the Morgan (Mor) fingerprints of the atom-level molecular structures (Rogers) & Hahn 2010).
We generated XGB-FC by combining XGB with the functional-class (FC) fin- gerprints of the input molecules (Rogers & Hahn 2010). The FC fingerprint represents the atom-level molecular structures based on their functional substructures and atoms.
XGB-FC:
XGB-MK:
We also generated XGB-MK based on the MACCS Key (MK) fingerprint (Singh et al., 2009), which is one of the most commonly used molecular representations. MACCS key encodes the atom-level molecular structures based on 166-bits binary patterns.
GIN (Xu et al. 2018): Graph isomorphism network (GIN) is an effective framework for graph representation learning based on graph isomorphism test.
EGCN (Tailor et al. 2021 Efficient graph convolution (EGC) is an isotropic GNN based on adaptive filters and aggregation fusion in the node aggregation phase. EGC out- performed common anisotropic GNNs, such as graph attention networks, on benchmark datasets.
17

---

Published as a conference paper at ICLR 2025
MPNN Gilmer et al. 2017): Message passing neural network is unified framework of node and edge convolution methods for learning molecular representations on quantum chemistry.
D-MPNN (Yang et al. 2 Directed MPNN (D-MPNN) is extension of the origi- nal MPNN for the directed molecular graphs. It employs a message passing scheme via directed edges (bonds).
UniMP Shi al. 2021): Unified message passing (UniMP) a transformer- based GNN. UniMP showed state-of-the-art prediction capabilities by incorporating feature and label propagation both training and inference time based on the transformer architecture.
AttFP (Xiong et al. 2019): AttFP is a network that uses a graph self-attention mechanism to learn molecular representations for drug discovery. AuFP was designed to learn non- local intra-molecular interactions to extract informative molecular representations.
SchNet (Schütt et al. 2017): is a convolutional neural network for learning molecular
representations based quantum interactions in molecules. It has been widely used baseline model various chemical applications (Schüit et al. 2017 2018).
DimeNet++ (Gasteiger et al. 2020): DimeNet aims to learn molecular representations based on the directional embedding that extracts inter-atomic 3D geometry. DimNet++ is an advanced version of DimeNet to learn the molecular representatios based on the uncertainty-aware directional embedding.
PhysChem (Yang et al. 202T): PhysChem is a neural architecture that learns molecular representations via fusing the information about the inter-atomic geometry and message passing through chemical bonds. PhysChem showed state-of-the-art prediction accuracy on various simulated molecular datasets.
M3GNet (Chen & Ong y022; Graph neural networks with three-body interactions (M3GNet) neural network learn molecular representations based on the three-body inter-atee atomic interactions. Although M3GNet requires large computational costs to calcu- late the three-body interactions, showed state-of-the-art prediction accuracy on various molecular and materials datasets.
FAENet (Duval et al. 2023): Frame averaging equivariant GNN (FAENet) is simple and fast GNN optimized for stochastie frame-averaging. FAENet can learn molecular repre- sentations by processing atom-relative positions with full flexibility without symmetry- prescrving requirements.
ConAN (Nguyen et al.) 2024]: Conformer aggregation network (ConAN) is an E(3)- invariant molecular conformer aggregation network to learn molecular representations based on ensembles methods on molecular conformers. showed state-of-the-art predic- tion accuracy on several molecular datasets by employing 2D-3D aggregation mechanism based on a differentiable solver for the Fused Gromov-Wasserstein Barycenter problem.
F IMPLEMENTATION DETAILS AND HYPERPARAMETER SETTINGS
We followed common graph-based descriptor convert atom-level molecular structure into attributed graph = (V,tl, A, X, R), where is set of nodes (i.c., atoms), t/ is a set of edges (i.e., chemical bonds), A {0, 1].1 [V] an adjacency matrix, X E R |V|xd a ddddimensional node-feature matrix, and R E g xr is an r-dimensional edge-feature matrix (Wieder et al. 2020). We used the pre-defined 200-dimensional atomic embeddings (Goodall & Lee (0200 to construct the node-feature matrix X. We defined the edge features as an one-hot encoding of 22 bond types
(Wieder et al. 2020 Chen et al. 2019). The pre-defined bond types were provided in RDKi which a popular Cheminformatics library computational chemistry.
The model parameters of DELID were optimized by the AdamW optimizer (Loshchilov & Hutter 2017) for all experiments in this paper. The initial learning rate and La regularization coefficients were fixed to 5e-4 and 5e-6 for benchmark datasets, respectively. Batch size is also fixed to 64 for all benchmark datasets. The GNN- based embedding networks were constructed by two node
htpps//www.nkkii.org
18

---

Published as a conference paper at ICLR 2025
Table 5: Electron-level features used for the retrieval process of DELID the QM9 dataset.
Category
Feature Name
Unit
HOMO
eV
LUMO
eV
HOMO-LUMO gap
eV
Zero point vibrational energy
eV
Energy-rehe chlate-related feature
Internal energy at 0 K
eV
Internal energy at 298. 1.5 K
eV
Enthalpy 298. 18.15522 K
eV
Free energy at 298. 15 K
eV
Hcat capacity at 298. 15 K
cV
Dipole moment
Debyc
Polarity-related feature
Isotropic polarizability
Bohr
Electronic spatial extent
Bohr²
Roiational constant A
Gllz
Other features
Rotational constant B
GHz
Rotational constant C
GHZ
aggregation layers and one dense layer with 64 output channels. DELID and experiment scripts were implemented with PyTorch 2.0.0+cul 17 and PyTorch Geometric 2.3.1 under Python 3.9.
In the implementation of the information retrieval makrishhan et al., 2014) containing the molecules of maximum six atoms as an external quantum mechanics dataset 2 pm We used 15 clectron-level features in the QM9 dataset to construct the fea- ture matrix of the fragmented electron-level information The selected 15 electron-level features of the QM9 dataset are shown in Table 5
used subset of the QM9 dataset (Ra-
G PREDICTION ACCURACY FOR DIFFERENT DECOMPOSITION METHODS
DELID employs the EFG-based decomposition method for generating the decomposed substruc- tures GT from the input molecule G. Table 6 shows the R -corees of two variants of DELID that use two well-known molecular decomposition methods: BRICS (Liu al. 2017) decomposition and junction tree method (Jin et al. (0188.. Although the RR=-scorss of DELID were not signifi- cantly changed for the implementations of the molecular decomposition methods, DELID with the EFG-based decomposition showed higher R2-scores for most benchmark experimental datasets.
Table 6: The R--scoree of DELID for different molecular decomposition methods.
Decomposition Method
Lipop
ESOL
ADMET
IGC50
LD50
LC50
LMC-H
CH-DC
CH-AC
BRICS (Liu al. 2017
0.763
0.824
0.516
0.635
0.515
0.865
0.863
(0.018)
(0.00)
(0.05))
(0000)
(003))
(0058)
(B040)
(0.47)
(0:011)
Junction Tree Jin et al. 2018
0.770
0.905
0.808
0,823
0,518
0.622
0,518
0.866
0.867
(0012)
(0.013)
(0103)
(0.05)
(0023)
(0057)
(0025)
(0.053)
(0.0011
EFG Lu ct
2021
0.782
0.912
0.834
0.844
0.566
0.644
0.532
0.886
0.885
(0.023)
H PREDICTION ACCURACY ON LARGE SIMULATED DATASETS
reliable in real- world complex molecules, we conducted an experiment of predicting molecular properties on large simulated dataset to evaluate the prediction capabilities of DELID large molecular datasets. For the evaluation, we used the QM-GW dataset (Fediai et al. 2023) containing the GW-level HOMO- LUMO gaps of 133,885 molecules. The GW method (Reining 2018) is an approximation method for the density functional theory calculations. The GW method computationally expensive but accurate in calculating the molecular properties related to the electronic energies. Thus, the machine learning methods should be able learn the GW-level calculations huge number of molecules in order to build an accurate prediction model on the QM-GW dataset.
Although the calculated physical and chemical properties of the molecules
"httpss//pytorch.org http:://pyoorchhccmmmerrrrrraaae hew.eeeeemmmmrccrttteeeesciooooi
19

---

Published as a conference paper at ICLR 2025
Table 7: The -ccoro of DELID and the competitor 2D GNNs on the QM-GW dataset containing 13k molecules and their calculated properties.
GIN
EGCN
MPNN
D-MPNN
UniMP
DELID (0.006)
AttFP
0.863
0.880
0.880
0.879
0.878
0.862
0.885
(0.008)
(0.004)
(0.005)
(0.003)
(0.002)
(0,003)
Table 7 shows the Re-scores of DELID and the competitor methods on the QM-GW dataset (Fediai et al. aaal]]222 (023) containing the GW-level HOMO-LUMO gaps of 133,885 molecules. In this experiment, DELID and all competitor 2D GNNs easily achieved the -c-scores greater than 0.85 because the simulated datasets are generated by the simple and consistent methods. In particular, although some the experimental datasets, they also achieved the Ro-scores greater than 0.85 on the large simulated dataset. This result demonstrates our main argument that the prediction capabilities of the machine learning models on the simulated datasets do not tell us the actual prediction capabilities of the machine learning models in real-world chemical applications.
GNNs failed to predict the molecular properties
I PREDICTION ACCURACY IN CLASSIFICATION TASKS
In the experiments, we focused on evaluating the prediction performances of the machine learning methods in regression problems due the following two reasons: 1) The regression problems is generalized problem of the classification problem, i.c., the classification problem is a specific case of the regression problem, where the number of possible classes in the target variable is fixed to countable natural number. 2) Most classification problems in physical and chemical applications are fundamentally the downstream tasks of the regression problem.
Table 8: The Fi-scores of DELID and the competitor 2D GNNs in the classification tasks of the BACE and BBBP datasets.
Dataset
GIN
EGCN
MPNN
D-MPNN
CGCNN
UniMP
AuFP
DELID
BACE
0.777
0.765
0.771
0.769
0.770
0.773
0.773
0.805
(0.019)
(0.026)
(0.027)
(0,027)
(0.024)
(0.017)
(0.013)
(0.014)
BBBP
0.908
0.911
0.897
0.894
0.894
0.912
0.905
0.924
(0.012)
(0.009)
(0.020)
(0.014)
(0.009)
(0.010)
(0.008)
(0.004)
There is no implementation issue of DELID in the classification tasks. Eq. (2) is generally applicable to both regression and classification tasks. In this experiment, measured the Fi-scores of DELID and the competitor 2D-GNNs in the classification tasks. We used two experimentally collected molecular datasets called BACE (Yan & Vassar 2014) and BBBP (Wu et al. 2018). The BACE and BBBP dataset contain experimentally measured biological activities 1,513 and 2,050 molecules, respectively. Table still showed the highest prediction accuracy.
& shows the measured Fl-scores on the BACE and BBBP datasets, and DELID
J PREDICTION ACCURACY FOR DIFFERENT MOLECULAR SCALES OF EXTERNAL CALCULATION DATASETS
We measured the R2-scores of DELID for different Dym containing different sizes of small molecules. We generated D Dym,c for (4, 5, 7), where is the maximum number of atoms. For example, Dgm,c is constructed from the QM9 dataset by collecting small molecules containing the atoms less than or equal to e.
the measured Re-scores of DELID for different values of c. DELID showed consistent R² -coores for different sizes of the molecules in D 9m because the input complex and large molecules are already decomposed into the small substructures by the EFGG-based decompsotion. This result shows that DELID is robust to the volume of D Dam required for the information retrieval.
20

---

Published as a conference paper at ICLR 2025
Table 9: The R -cores of DELID on the benchmark molecular datasets for different Dym containing different sizes of small molecules.
Max. Num.
Num. Molecules
Atoms (- d) in pm
Lipop
ESOL
ADMET
IGC50
LD50
LC50
LMC-H
CH-DC
CH-AC
c=
45
0.780 (0010)
0.921
0.843
0.863
(0018)
0.565
0.636
(0.003)
0.533
(0120)
0.876
0,841
(0
175
0.776
0.912
0.680
0.848
0.586
0.641
0.538
0.873
0,880
(0.03))
(0.910)
(0:010)
(0022)
(0.071)
(0.041)
(0.055
(0.04)
0,782
0.912
0.834
0.844
0,566
0.644
0,532
0,886
0.885
(0.03)
(0.014)
(0.042)
(0.000)
(00024)
(0.063)
(0.048)
(0.155)
(0023)
c= 7
3,990
0.781
0.916
0.843
0.844
0,574
0.658
0.530
0.872
0.882
(0.030)
(0.912)
(0/023)
(0.015)
(0,000)
(0.063)
(0.033)
(0/122)
(0012)
K EXECUTION TIME OF THE TRAINING AND INFERENCE PROCESSES OF DELID
Compared to the conventional GNNs, DELID requires additional computation execute the infor- mation retrieval and the self-supervised diffusion. In this experiment, we compared the execution time of DELID with those of the competitor GNNs. We separated the entire execution process of machine learning methods into data pre-processing. training, and inference steps. We measured the entire execution time of the data pre-processing and inference processes, whereas we measured the execution time of one epoch for the training process. We compared the execution time of GIN, MPNN, UniMP, SchNet, PhysChem, and DELID, as shown in Table [0] The execution time was measured in machine with Intel 12900K CPU, 128G memory, and NVIDIA GeForce RTX 3090 Ti GPU.
Table 10:
Execution time of DELID and the competitor methods on the Lipop dataset. The execution
time measured in seconds.
Category
Method
Data Pre- rrepprocsssigg
Training
Inference
Total
SchNet
119.937
1,109.350
0.172
1,229.459
3D-GNN
PhysChem
59.453
109,271.543
10.289
109,341.285
GIN
3.109
152.504
0.031
155.644
2D-GNN
MPNN
3.109
429.524
0.062
432,695
AUFP
3.109
125.273
0.031
128.413
DELID
50.875
562,549
0.124
613.548
The 2D-GNNs were the most efficient among the competitor methods and DELID because they do use additional molecular descriptors and complex node aggregation mechanisms. In contrast, the 3D-GNNs required the most execution time. In particular, the total execution time of PhysmChem was 109,34 seconds, which is 178 times greater than the total execution time of DELID. Although DELID requires more execution time in data pre-processing and the conditional diffusion pro- cess, it showed comparable execution time with vanilla MPNN, which was employed to implement the atom- embedding network of DELID.
L EMBEDDING RESULTS AND MOLECULAR WEIGHT
We visualized the electron-awart representation for the molecular weight, which can be one of the underlying variable determining several target molecular properties. We plotted for the molecular weight on the Lipop, ESOL, and ADMET datasets, where the target molecular properties are related the molecular weight. As shown Fig. [ even though the molecular weight was not provided for training DELID, DELID generated that roughly describes the underlying molecular weight.
M MEAN ABSOLUTE ERRORS IN MOLECULAR PROPERTY PREDICTION
Table shows maen absolute error (MAE) of the competitor methods and DELID in predicting molecular properties on the experimental molecular datasets. The evaluation results of DELID were consistent with the evaluation reulsts in Talbe
21

---

Published as a conference paper at ICLR 2025
Lipop
ESOL
ADMET
10
-10
-40
-20
20
10
30
20
10
-20
-30
30
50
Low molecular weight
High molecular weight
Figure 5: Visualization of the electron-aware representation for the underlying molecular weight.
Table 11: MAEs of the competitor methods and DELID in predicting molecular properties.
Method
Lipop
ESOL
ADMET
IGC50
LD550
LC:50
LMC-H
CH-DC
CH-AC
GIN
0.456
0.397
0.628
0.330
0.443
0.717
0.349
37.268
33.125
(0.010)
(0.02))
(0.026)
(0.020)
(0.067)
(0.029)
(0.06)
(2.818)
(2.835)
EGCN
0.447
0,594
0.602
0.331
0.446
0.722
0.337
33.469
34.578
(0.015)
(0.02))
(0.021)
(0.050)
(0.008)
(1.462)
(2876)
MPNN
0.434
0.603
0.649
0.351
0,460
0.704
0.342
31.257
32.765
(0.012)
(0.04))
(0.122)
(0.014)
(0.007)
(0.173)
(0,00)
(1.570)
(2,70))
D-MPNN
0.422
0,507
0.636
0.327
0.472
0.645
3.332
40,549
33,574
(0.011)
(0.031)
(0.019)
(0.018)
(0.010)
(0.026)
(0,009)
(2.524)
(3,042)
UniMP
0.453
0.609
0.619
0.335
0,450
0.707
0.352
38,546
35,896
(..20)
(.019)
(0.020)
(0.051)
(0,010)
(3:204)
AttFP
0.466
0,441
0.577
0.315
0,472
0.635
0.335
54.234
46,211
(0.005)
(0.034)
(0.024)
(0.011)
(0.004)
(0.062)
(0,006
(3.334)
(3:300)
DELID
0.395 (0.011)
0,425
0.562
0.279
0.443
0,592
0.310
20,430
19,503
(0.027)
(0.028)
(0.004)
(8.007)
(0.1137)
(0,007)
(2,2033
(1,66))
N DIFFUSION PROCESS ON S IN UNSUPERVISED SETTINGS
In our problem setting in Eq. (2. the entire model parameters of DELID is optimized to maximize log ply,s, s, G). For this reason, although the diffusion model on s is trained by the self-supervised scheme guided by the diffusion model on G, the embedding results on are finally affected by the target molecular property y. To investigate the representation learning capabilities of DELID in unsupervised settings, we re-implemented DELID to maximize log p(s. G) by removing the predic- tion layers in DELID and measured the Wasserstein distance between the data distributions of the original molecular graph G and the diffusion output So on the ESOL dataset. G was projected into the vector space through randomly initialized GNNs to preserve the data distribution the original molecular graphs Schneider & Vlachos (2017) Bingham & Mannila 2001).
Wosserstein distance
0.5
0113
0.13
145
200
130
420
Epach
Figure 6: Wasserstein distance between the data distributions of G and so on the ESOL dataset.
As shown Fig. 6 the Wasserstein distance between G and Sn consistently decreased as the diffusion model on s was optimized. This result shows that the diffusion models of DELID worked well in the unsupervised setting. Furthermore, this result worth noting because the diffusion model on s successfully learned the data distribution of G with only the fragmented information about decomposed substructures and their electron-level features.
22

---

Published as a conference paper at ICLR 2025
Table 12: The R2 -coors of the competitor methods, DELID, and NN-DELID on benchmark exper- imental molecular datasets.
Input Type
Method
Lipop
ESOL
ADMET
IGC50
LD50
LC50
LMC-H
CH-DC
CH-AC
XGB-Mor
0,531
0.659
0.717
0.621
0.390
0.497
0.505
N/R
N/R
Fingerprint
Molecular
XGB PC
00024)
0.686
(1045)
0.720
(0.001)
0.628
01800)
0.501
(0113))
(0.06]
0,519
0.503
(0018)
N/R
N/R
(0009)
(0607)
XGB-MK
0,542
0.764
0.761
0.680
0.486
0,526
0.471
N/R
N/R
(064))
(0947)
(0.000)
(8031)
(0.112)
(0.21)
(3019)
SchNet
0.881
0.834
0.765
0.527
0.467
0.456
0.713
0.702
DimeNett+
N/R
0.878
N/R
0.779
0.541
N/A
0.352
N/A
N/A
(0025)
(0019))
(0.45)
(0.00)
3D Molecular
PhysChem
0.694
0.848
N/A
0.814
N/A
N/A
N/A
N/A
Graph
M3GNet
N/A
0.857
N/A
0.697
0.531
N/A
NAA
N/A
N/A
(8025)
(8029))
(18344)
FAENet
0.670
0.869
0.788
0.708
0.528
0.437
0.437
0.310
((0M))
ConAN
0.738
0.909
0.845
0.819
0,531
0,572
0.466
Q.40S
0.388
(U015)
(0007)
(8848)
(0,00)
(D628)
(0.10))
(0.115)
GIN
0.709
0.808
0,807
0.792
0.545
0.525
0.472
0.242
N/R
EGCN
0.716
0.822
0,814
0.777
0.550
0,503
0.497
0.226
N/R
(0X21)
(0.29))
(0.211
(0A22)
(0.15))
(0.000
(D008)
(0086)
MPNN
0.727
0.810
0.801
0.764
0.502
0,487
0.46
0.385
N/R
2D Molecular
D-MPNN
0.726
0.879
0,820
0.787
0.521
0,566
0.494
N/R
N/R
Graph
(0437)
(0:033)
(0,058)
(0008)
(1.01))
(0L09)
(0011)
UniMP
0.810
0,817
0.756
0.512
0.531
0.478
0.166
N/R
AnFP
0,710 (0/21)
0,909
0,851
0.807
0.513
0.642
0.456
0.441
0.296
(8018)
(L.22)
(0013)
(0.055)
(0,77))
(D600))
(0409))
(0.3700
NN. DEI JID
0.908
0.825
DELID
0.782
0.912
0,834
0.844
0.566
0.644
0.532
0.886
0.885
(0.025)
O REPRESENTATION LEARNING CAPABILITIES OF DELID IN UNSUPERVISED SETTINGS
DELID is designed to learn electron-aware molecular representations for given molecular structures G and target molecular properties y by maximizing log p(y, S, G). as described in Eq. (2). However, the diffusion models on G and can be trained without y by maximizing log p(s G) instead of log ply, s, G). In other words, we can train the diffusion models with a fully unsupervised setting and build prediction model by transferring the diffusion models into the downstream prediction task. In this experiment, we measured the RC-scores of fully connected neural network (FCNN) that employs the molecular representation of the unsupervised DELID as its input data molecular property prediction. We denote the FCNN following DELID by NN-DELID.
Table 12 shows the Re-scores the competitor methods and NN-DELID. In this experiment, NN- DELID also outperformed the competitor methods for most benchmark datasets. However, the R2-scoree of DELID were usually higher than those of NN-DELID.
P LIMITATIONS AND FUTURE WORK
Since the self-supervised diffusion processes of DELID start from the decomposed molecular sub- structures and their electron- features, the performances of DELID are basically dependent on the molecular decomposition methods and the external calculation datasets. In particular, as shown in Table 2 the input electron-level features can directly affect the prediction accuracy of the final prediction models. However, we did not develop a molecular decomposition method and calculation dataset specialized in the self-diffusion processes of DELID. For this reason, the improvements by DLIED are essentially limited to the performances of the EFG-based decomposition method and the QM9 dataset. Therefore, molecular decomposition and calculation dataset specialized in DELID need to be considered as future work.
23


Reviews:
[{"review_id": "uiiartfshZ", "reviewer": "Reviewer_Kwdq", "summary": "The paper proposes to enhance molecular representation learning by incorporating electron-level information via diffussion process, which eliminates the need of expensive quantum calculations. The proposed methods are evaluated on real-world experimental property prediction datasets to show the applicability and effectiveness, and also on simulation dataset such as QM9 for further discussion. Several analysis are conducted to assist the findings.", "questions": "See weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The paper proposes a new perspective on enhancing molecular representation learning by leveraging the diffusion process. \n- The proposed method save the computational costs on quantum calculations.", "weaknesses": "- The decomposed substructures serve as the noise, what is the motivation and reason of such implementation? Why not insert noise to the molecules? The authors have defined such, but there are not sufficient explanations of why doing so. \n\n- The authors mention that FFSEC has low calculation accuracy, then why still use it to calculate coordinates? Why not use commonly used Rdkit to generate? \n\n- How to define the substructure size? The paper mentions that EFG-based method is used for decomposition, however they also remove the large molecules in QM9 due to the redundancy? \n\n- Why only $R^2$-scores and F1 scores are reported? The commonly used evaluation metrics like MAE/RMSE and AUC/accuracy are not reported. The reported results are also not very significant compared with baselines, such as ESOL, ADMET, LC50.\n\n- How is the 5-fold leave-one-out cross-validation implemented? Is it leave one fold out? Why not follow the standard evaluation method? What about the split type? Is it scaffold or random split?\n\n- Have the authors compared with methods that utilize both 2D and 3D graphs, like 3d infomax [1] and GraphMVP [2]?\n\n- The proposed method seems not universal as it is mentioned that only under certain circumstances the method would work well, like relatively large molecules. I am also confused about the term of electron, it seems that the proposed method is based on substructures, why is it called electron-aware? \n\n- The paper needs proofreading, i.e., some tense is not consistent. Some contents show conflict. For example, in the related work SchNet is introduced under the 2D-GNNs, while it is actually 3D GNN, where the authors also mention such in the Competitor Methods part.\n\n\n[1] Stärk, Hannes, Dominique Beaini, Gabriele Corso, Prudencio Tossou, Christian Dallago, Stephan Günnemann, and Pietro Liò. \"3d infomax improves gnns for molecular property prediction.\" In International Conference on Machine Learning, pp. 20479-20502. PMLR, 2022.\n\n[2] Liu, Shengchao, Hanchen Wang, Weiyang Liu, Joan Lasenby, Hongyu Guo, and Jian Tang. \"Pre-training Molecular Graph Representation with 3D Geometry.\" In International Conference on Learning Representations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2BkAsptPjG", "reviewer": "Reviewer_3yqb", "summary": "This paper presents a new approach for electron-aware molecular representation learning. The key motivation is to introduce electron-level information into the learning process. The framework is formulated as a self-supervised diffusion process maximizing $log p(y,s,G)$, which is implemented as follows: (1) the input 2D molecular graph is decomposed based on extended functional groups (EFGs) to obtain a noised adjacency matrix $G_T$; (2) for each of the components, quantum-chemical properties are retrieved from a support set (a subset of QM9 in the experiments) as noised electron-level information $s_T$; (3) the model is then required to denoise the adjacency matrix and the electron-level information and predict the final property y simultaneously.\n\nThe proposed method is evaluated on several experimental molecular property prediction datasets, showing higher prediction accuracies than existing methods. The proposed method is also shown to have better generalization capabilities by an experiment simulating data-scarce scenarios. An ablation study on model components is provided. The authors also provide experimental results for different electron-level features, different external support datasets, execution time, etc.\n\nI feel that the paper is of good quality overall, but there are still some points that need clarifications and discussions.", "questions": "- Given the points mentioned in Weaknesses, it is then unlikely the improvements of the proposed method come from a more physical design. Could you provide an intuitive explanation about *why* do the improvements occur?\n- A few important implementation details seem to be missing from the current version of paper. For example, what exact value does $T$ have?\n- Is $y$ required for the model to learn good molecular representations? If it is required, do models trained on different datasets predict similar $s_0$?\n- In section 4.4, the ablation study on DELID, could you also list the `#params` of the variants? What happens if the `#params` of the ablated variants match that of the complete DELID?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The paper is well motivated. I agree with the authors that a molecular representation model should be electron-aware. The proposed method implements this without requiring ground-truth labels for whole molecules, which is novel and interesting.\n* The paper is also easy to understand in general. I was able to follow the paper with most of my questions answered in the text.\n* Probably thanks to the fast training/inference time, the evaluation is very thorough and comprehensive. The proposed method, DELID, not only shows higher prediction accuracies than existing methods, but also exhibits lower computation consumption and better generalization capability. The comparative analysis on different feature selections, different support datasets and different decomposing methods makes one believe the current design is the optimal.\n* The provided code is clean and easy to follow, with dataset files directly included. This gives a good impression that the work can be easily reproduced using the provided code.", "weaknesses": "- Interpreting $s_T$ as electron-level information could be confusing. $s_T$ of shape $N_{comp} \\times m$, which are pre-calculated properties for the decomposed substructures, actually seems more like substructure-level information rather than electron-level information. First, the only reason why the features can be vaguely regarded as containing information about electrons is that they are computed using quantum chemistry methods like DFT. However, these methods run on conformations, not on 2D graphs. The retrieval process for $s_T$ actually links a 2D substructure to a 3D conformation, and the latter could even be of a similar but different molecule. Second, the used features are mostly molecular-level observables aggregated from electron density. These two reasons make $s_T$ less physical and less suitable to be regarded as electron-level information.\n- $s_t (t = 0, 1, ..., T - 1)$ is not only unknown, but also hardly defined. I think it can only be interpreted as latents, not as electron-level or substructure-level information. For example, the meaning of $s_0$, an $N_{comp} \\times m$ tensor, is not clear given that the components are already connected in its corresponding graph $G_0$.\n- The method is called self-supervised diffusion, while the current implementation seems to rely on the availability of $y$ .", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KPB67AMKwT", "reviewer": "Reviewer_QeAj", "summary": "The paper introduces a method called DELID (Decomposition-supervised Electron-level Information Diffusion) for molecular representation learning. DELID utilizes molecular decomposition to generate substructures and matches them with a subset of the QM9 dataset to retrieve electron-level information. The method comprises two diffusion models: one estimates the original molecular graph from the decomposed subgraphs, and the other estimates the electron-level information from the electron-level features of these substructures. Finally, DELID employs these estimates to predict molecular properties. This approach allows DELID to learn electron-aware molecular representations and predict properties of complex molecules effectively, leveraging both structural and electronic information.", "questions": "1. Diid the authors use only the middle layer embedding of the diffusion model in their predictions or did they also use the final estimated electronic information?\n2. The difference between DELIDqm and DELID in table3 makes me wonder if the electron level of the diffusion model is well constructed for electronic information of complex molecules. Can the authors provide the difference between the final estimated electronic information and the information obtained from the dft calculations?\n3. What is the difference between the molecules generated by the diffusion model and the original molecules?\n4. Can the embedding vectors generated by the diffusion model  distinguish different molecules? For example using t-sne to visualize the embedding of a certain dataset and distinguish different categories of molecules with molecular weights or whether they are aromatic molecules?\n5. How to construct the molecular graph and how to combine several subgraphs to generate a graph? How to represent the electron information from the QM9?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper introduces a self-supervised diffusion models method to represent molecules by estimating electron-level information in complex molecules. The proposed method is efficient as it does not require expensive quantum mechanical calculations to estimate electron-level information. DELID achieves state-of-the-art prediction accuracy on extensive real-world molecular datasets.", "weaknesses": "1. The performance of DELID is dependent on the external quantum mechanical calculation datasets, such as QM9. The distribution and the quality of such datasets may influence the ability of the models. \n2. After breaking down the molecule into substructures, the electronic information of each part is calculated, and thus the interactions between the parts can be neglected. For example, the electronic shift of two neighboring functional groups. It is doubtful that the diffusion model is able to construct such interactions well.\n3. The decomposition from complex to simple molecules is unique, but the process of denoising from simple molecules to generate the original molecule is not unique, e.g. isomers. This is not well avoided by the electronic information generated using the self-supervised method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wSnlqXgsIO", "reviewer": "Reviewer_KLby", "summary": "The authors propose a method for learning molecular representations using self-supervised diffusion processes. This method can learn an \"electron-aware\" representation without requiring ground truth electron properties of the complete molecule. This is achieved by leveraging the synergy between the diffusion process at the graph level and the electron-level information. Additionally, by decomposing the complete graph into subgraphs (molecular fragments), the reverse diffusion process can utilize subgraph information readily available in chemical databases. Lastly, the method showed superior performance compared to state-of-the-art methods in various downstream tasks.", "questions": "- Is there a typo in Equation 1? Should it be t=1 or t-1 in the denominator?\n- Regarding the construction of s_T: Some properties, such as energy, are extensive properties, so the quantity depends on molecule size. Retrieval using Tanimoto similarity does not consider molecular size. If the retrieved fragment and the actual fragment have different sizes, the property could differ significantly. Will this negatively impact model performance? Additionally, is there a list of the 19 features used to construct Q_k? Is the final $s$ just the 19 features for the complete molecule?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "- Originality: The idea of learning electron-aware molecular representation without needing ground truth electron properties is novel and significant. The method is also original in how it leverages the synergy between the diffusion process and electron-level information.\n\n- Quality: The method is well designed, and the experiments are thorough.\n\n- Clarity: The paper is well written and easy to follow.\n\n- Significance: The method showed superior performance on various tasks and can inspire future research in the field.", "weaknesses": "- The authors provide details on the diffusion process, but the actual process of predicting the properties is not described in enough detail. How are the learned representations used to make final property predictions?\n- In Table 1, the benchmark methods are mostly GNNs and XGB. It would be better to compare with more diverse methods, such as Fingerprint + MLP/SVR, or other methods representing diverse paradigms.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yeEFVgYc6g", "reviewer": "Reviewer_bQLK", "summary": "This paper proposes a method that uses two diffusion models to learn the representations of molecules and electron information, then the representations are used for property predictions. The novelty of this work is to estimate electron information from existing public database, without expensive ground truth labeling, which is achieved by a self supervised training from the molecular diffusion model. \n\nThe authors performed comprehensive experiments with relevant baseline models, and demonstrate the state of the art performance of the proposed model on the experimental dataset that contains large molecules and experimental measurements.", "questions": "- It is not so clear to me what parts are built with a separate neural network model. For example, my understanding is $\\log p(A_{t-1}|A_t)$, $\\log p(s_{t-1}|s_t, A_{t-1})$, $\\log p(A_{t-1}|s_t)$ are parametrized as separate models. It will be good if the authors can clarify and improve the notations.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- Originality: This paper proposes a novel approach that utilizes the accessible electron information of subgraphs (or building blocks) of a molecule from public database, to enhance the representation and property prediction. The self-supervised learning also shows a good theoretical and algorithmic ground, so as to solve the difficulties of the need for expensive calculations for ground truth electronic structure.\n- Quality: This paper has demonstrated solid theoretical derivations, and has performed comprehensive numerical experiments on multiple tasks, comparisons with other models and ablation studies.\n- Clarity: The figures and formulations are clear and illustrative. The text has good clarity.\n- Significance: This work shows a potential way for addressing small scale molecular dataset.", "weaknesses": "- Even though it is a good use of publicly available electronic structure for subgraphs/fragments, the final electronic information $s_0$ is only an estimation of the ground truth. And in fact, it is not guaranteed to be a good estimation, as the electronic structure of a molecule with different fragments covalently bonded can be very different from mixing the electronic structures of different individual isolated fragments. Therefore, physically, this self supervised learning, which is essentially trying to mix the electronic information of fragments in the same way as mixing the structural information, is not very meaningful.\n- Based on the point above, I would expect that the information mostly gained from the electronic representation will come directly from $s_T$ which are the electronic features of the fragments, instead of from $s_0$. And this can be verified from Table 3, the performance difference between \"DELID_qm\" and \"DELID\" is actually very small and generally within the error range.\n- The benchmark dataset used here is a small dataset, as it is an experimental dataset and also includes large molecules. But for some other competitor models, it makes sense that they don't perform that well if they are trained from scratch for this small dataset. That does not mean that the method proposed here is really SOTA, as I would expect those competitor method can potentially perform very well with a pretrain-finetune workflow (i.e. pretrained on a large scale simulated datasets, and then finetuned for the small experimental dataset).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)