Molecular Motif Learning as a pretraining objective for molecular property prediction

Domain: physics
Published: 2025-11-27
Editorial outcome: minor_revision

Abstract
Molecular property prediction is crucial for drug discovery in biopharmaceuticals since it helps identify promising compounds, optimizing the efficacy of developing new therapies. Despite its importance, existing deep learning-based methods for this task are often incongruous with fundamental chemical properties. Here we show that an unsupervised pretraining approach, Molecular Motif Learning (MotiL), learns molecular representations that preserve both whole-molecule structure and motif-level information directly from native molecular graphs. MotiL produces representations that group small molecules sharing a common core structure (i.e., scaffold) and proteins with related three-dimensional structures and functions. We evaluated MotiL on at least 16 molecule benchmarks, and uncovered that it captures analogous graph representations not only for small molecules with the same scaffold but also for protein macromolecules with similar structures and overlapping chemical functions such as tRNA binding. These informative representations empower MotiL to surpass the accuracy of state-of-the-art contrastive or predictive methods in the prediction of molecular properties like blood brain barrier permeability.

Introduction
Molecular property prediction is a pivotal area of research in computational chemistry and biophysics, encompassing the property prediction of both small molecules1,2 and protein macromolecules3,4. Precise prediction of molecular properties can significantly enhance the understanding of chemical and biological systems, leading to advancements in drug discovery5,6, materials science4,7, and molecular biology8,9.

Small molecules, i.e., molecules of low molecular weight, play crucial roles in various biological processes and therapeutic applications10–12. Their properties, such as toxicity13,14, blood brain barrier permeability15, and adverse drug reactions16, are pivotal for drug efficacy and safety17,18. Traditional wet-lab experimental methods to determine these properties are time-consuming and costly19,20. It has led to the necessity of developing reliable computational models, particularly machine learning approaches21,22. In addition, proteins play critical roles in many cellular functions, and their properties like folding and stability are essential for understanding biological mechanisms and designing therapeutic interventions23,24. Computational methods, including molecular dynamics simulations and machine learning, have been employed to predict protein properties by bridging the gap between amino acid sequences and biological functions25,26. Understanding these properties allows researchers to anticipate how proteins behave in different environments, which in turn facilitates the rational design of proteins with desired characteristics.

Existing deep learning-based molecular representation learning methods can be divided into two categories: contrastive learning and predictive learning. Contrastive learning methods11,12 typically produce two augmented versions of a molecular graph through techniques like subgraph removal or perturbation, and then use them to form positive and negative sample pairs. While these methods can distinguish chemically different molecules, they may inadvertently disrupt chemically meaningful substructures (e.g., functional groups or residues), thereby weakening the preservation of critical motif information. Predictive learning methods27–30, in contrast, aim to learn molecular representations by predicting masked subgraphs27,28, contexts29, or attributes30. Beyond these graph-based methods, recent methods based on SMILES or sequences have also shown promise by leveraging larger unlabeled corpora (e.g., ChemBERTa31, SMILES-BERT32, ESM33). While these methods benefit from scalability, they may sacrifice finer chemical granularity, such as stereochemistry or bonding topology, which graph-based methods aim to preserve.

It is crucial for molecular representation learning to produce representations that accurately reflect the underlying chemical structures and functions. However, existing methods struggle to effectively capture the representations of motifs, such as functional groups in small molecules or amino acid residues in proteins, due to their limited ability to model localized chemical semantics or hierarchical substructures. For instance, prior contrastive learning methods11,12,27 often rely on random subgraph sampling or global-level supervision, which may overlook pharmacologically or structurally meaningful motifs. Similarly, predictive molecular representation learning, which involves masking subgraphs, contexts, or attributes, also compromises the integrity of motif information in the original molecule, further hindering the preservation of critical chemical properties.

To address the above limitations, we propose a pretraining representation learning method for molecules, i.e., Molecular Motif Learning (MotiL). MotiL includes three phases totally: diffusion priming, bi-scaled training, and task-specific fine-tuning. The three phases achieve the goals of global structure perception, local feature integration, and real-world label alignment, respectively. In the first phase, we design a Diffusion Molecular Model (DiffMoM) to warm up the parameters of GNN by altering and correcting chemical bonds. In the second phase, MotiL learns molecular representations directly from the original molecule graph by the GNN encoder in an unsupervised manner. Both the first and second phases enable MotiL to effectively learn chemically meaningful graph representations and motif representations across various molecular contexts. At the graph level, MotiL produces similar graph representations for small molecules sharing the same scaffold (e.g., naphthalene and barbituric acid) or for proteins exhibiting similar three-dimensional structures and overlapping chemical functions (e.g., protein kinase activity and tRNA binding). At the motif level, MotiL discerns functional groups with identical chemical bonds (e.g., the carbon-oxygen double bond in carboalkoxy, carbonyl, and carboxyl groups) and assigns them similar motif representations. Moreover, MotiL identifies structural isomers among a wide range of amino acids and assigns them similar representations, while the amino acids with disparate structures (e.g., aromatic rings vs. linear chains) are assigned dissimilar representations. In the final phase, the pretrained GNN parameters are further fine-tuned using backpropagated gradients calculated from the classification or regression loss for each molecule. This phase enables MotiL to accurately predict the micromolecular properties like toxicity and blood brain barrier permeability, as well as the macromolecular properties like adenosine triphosphate (ATP) binding. After evaluation on public benchmarks, MotiL consistently outperforms the state-of-the-art methods, e.g., MotiL surpasses the runner-up by 10.1% on the QM7 dataset in the quantum mechanics domain. Additional results further highlight MotiL’s effectiveness across domains such as physiology, biophysics, and physical chemistry.

In this work, we systematically formulate molecular motif representation learning as a core pretraining objective, rather than using motifs merely as auxiliary tools for molecular learning. Moreover, we successfully applied MotiL to both micro- and macromolecular property prediction, demonstrating its versatility across molecules of different scales.

Results
Overview of MotiL
We propose diffusion priming as the first phase of MotiL to perceive the global structure of a molecule and thus enhance the robustness of GNN to structural noise perturbations in the molecule graph G (Fig. 1a). DiffMoM consists of two processes: noise addition and noise reduction. In the noise addition process, Gaussian noise is introduced directly into the adjacency matrix of the molecular graph G, producing a perturbed adjacency matrix A at each time step t. Then in the noise reduction process, DiffMoM aims to recover the original bonds in G. It uses the perturbed molecule graph G at the T-th time step as inputs of the parameterized GNN encoder Gθ (where θ denotes the trainable parameter group) to produce the reconstructed graph G^. During the optimization of DiffMoM, θ is adjusted to minimize the reconstruction error between G and G^, ensuring that Gθ accurately captures the molecular structure of G.

In the second phase of MotiL, we integrate the local features in a molecule via two GNN encoders where network pruning is applied to Gθ and then results in a new GNN encoder Gθ0 with fewer weight connections (Fig. 1b). For a given molecule, Gθ and Gθ0 produce two different graph representations. Both representations are designed to be as similar as possible for the same molecule, and as dissimilar as possible for different molecules. With this design, we preserve the complete information of graphs and motifs in model training, ensuring that Gθ learns accurate molecular representations and functional group’s representations.

Moreover, considering the representation’s stability of the same motif (functional group or amino acid) across different molecule graphs, we design the bi-scaled learning mechanism in MotiL, which operates at both the graph and motif scales. First, MotiL aligns the two graph representations derived from the same molecule graph, ensuring the invariance of the overall molecular semantics under different encoders. Second, MotiL identifies key motifs like functional groups and ensures that the representations from the same motif are consistent across different molecules. In the example of Fig. 1b, for the input molecule aspirin, its graph representations H1 and H2 should be close to each other, while the graph representations H2 and H3 (benzamide’s representation) from different molecules should be pushed apart. Meanwhile, the representations of the same functional group (e.g., both S1 and S2 for benzene ring) should be similar but the representations of different functional groups (e.g., S1 for benzene ring and S5 for carboxyl group) should be dissimilar. The bi-scaled learning mechanism ensures that MotiL not only captures the overall molecular semantics but also maintains the semantic stability of each functional group.

In the third phase of MotiL, Gθ is fine-tuned to align with real-world labels (Fig. 1c). Gθ first extracts the graph representation H5 from a previously unseen molecule paracetamol. Then H5 is fed into a multilayer perceptron (MLP) classifier to obtain a logit value. The logit value is compared with the true labels, e.g., hepatotoxicity, to calculate the classification loss. During this phase, the optimizer computes the backpropagated gradients based on the classification loss to further update θ. The task-specific fine-tuning phase ensures that Gθ is adjusted to improve its performance on the specific molecular property prediction task.

MotiL boosts the performance of molecular property prediction
The range of molecular properties is vast, covering scales from large-scale impacts on human physiology to minute electronic characteristics4,7,8. These properties include effects such as drug metabolic stability, target selectivity, toxicity, and the inhibition potential against HIV. To thoroughly evaluate MotiL’s effectiveness, we test its performance across diverse small micromolecule datasets categorized into four main areas: physiology, biophysics, physical chemistry, and quantum mechanics. To illustrate MotiL’s capability in dealing with complex biological entities, we also include experiments involving protein macromolecules. The details of experimental datasets can be found in Supplementary Section B.

Tables 1 and 2 show the experimental performance of state-of-the-art methods and MotiL on classification and regression tasks related to small molecular property prediction (see Supplementary Tables 3 and 4 for complete results, including exact p-values). Similar to baselines, we use the area under the receiver operating characteristic curve (ROC-AUC) as the metric for classification tasks. For regression tasks, we use root mean square error (RMSE) for ESOL, FreeSolv, and Lipophilicity (abbreviated as Lip.), and mean absolute error (MAE) for QM7, QM8, and QM9. The calculation method for evaluation metrics is provided in Supplementary Section C. From Tables 1 and 2, we can see that MotiL outperforms all baselines on each dataset, achieving an average relative improvement of 3.0% over the runner-up. For instance, MotiL surpasses the runner-up method KANO with a relative improvement of 1.6% in ROC-AUC on MUV and 10.1% in MAE on QM7. Although KANO has demonstrated high performance in Tables 1 and 2, it fails to accurately predict the blood-brain barrier permeability of certain chemotherapeutic drugs or antibiotics, as well as the hepatotoxicity of some molecules (Supplementary Fig. 9a). MotiL corrects KANO’s erroneous predictions in specific cases and surpasses KANO in overall prediction accuracy (Supplementary Fig. 9b). The augmentation-based methods like MolCLR, CMPNN, and KANO outperform supervised and predictive self-supervised baselines, but they often overlook the negative impact of graph augmentation on motif integrity. In contrast, MotiL avoids aggressive augmentation, integrates a generative diffusion model to enhance robustness, and achieves consistently superior performance across benchmarks. These results highlight the effectiveness of motif-preserving representation learning.

Although augmentation-based methods like MolCLR and CMPNN KANO outperform supervised methods like GEM and predictive self-supervised methods like GROVER, they consistently overlook the negative impact of augmentation, thus falling behind MotiL. Our experiments on absorption, distribution, metabolism, and excretion (ADME) prediction tasks (Supplementary Table 6) show that MotiL consistently delivers robust and competitive performance, achieving strong predictive accuracy and lower variance compared to baseline models. These results highlight the potential of MotiL for pharmacologically relevant applications. Our sensitivity analysis (Supplementary Fig. 1) reveals that a retaining probability of p = 0.7 in network pruning achieves the best trade-off between representation divergence and convergence stability, leading to consistently strong performance across datasets. Our ablation study (Supplementary Fig. 2) shows that each component of MotiL (DiffMoM, bi-scaled training, and network pruning) contributes to performance gains, with DiffMoM playing the most critical role in enhancing robustness and generalization across diverse molecular tasks.

Next, we evaluate the Fmax performance of different methods on two protein datasets: gene ontology (GO) term prediction and enzyme commission (EC) number prediction. The results are shown in Table 3 (see Supplementary Table 5 for complete results, including exact p-values). As in previous work, the GO term prediction task is divided into three sub-tasks: biological process (BP), molecular function (MF), and cellular component (CC)24,34. We can observe that MotiL consistently outperforms baselines, including state-of-the-art graph-based methods like GearNet and CDConv, highlighting its potential for broader applications in computational biology. On GO-BP, GO-MF, and GO-CC, MotiL achieves the highest Fmax performance and outperforms the runner-up CDConv by 2.2%, 2.2%, and 6.9%, respectively. Also, on EC, MotiL achieves the most significant improvement, reaching an Fmax of 87.9%, surpassing the runner-up CDConv by 1.8%. These consistent gains across diverse tasks suggest that the generative diffusion model DiffMoM allow the GNN encoder in MotiL to capture intricate structural information from the original protein graphs, which is crucial for accurate protein property prediction.

MotiL learns micromolecular representations consistent with molecular structures
We use t-distributed stochastic neighbor embedding (t-SNE)35 to project the micromolecular representations learned by MotiL into a 2D space (Fig. 2a), where each point represents a molecule, and the color indicates the molecule’s scaffold. A scaffold represents the basic structure of a molecule, and compounds sharing the same scaffold typically exhibit similar chemical and biological properties. For example, we select four chemically diverse scaffolds from ESOL. The selected scaffolds, such as Naphthalene, are frequent and structurally distinct within ESOL, making them suitable for evaluating the model’s ability to differentiate solubility-relevant chemotypes. A lower Davies-Bouldin (DB) index36 indicates tighter grouping within clusters and greater separation between them. We compare MotiL with three methods: No pretraining (i.e., not using diffusion priming and bi-scaled training), Molecular contrastive learning (MCL) with augmentation (i.e., MolCLR11), and MCL without augmentation (removing the diffusion priming phase from MotiL). The following conclusions can be drawn: 1) No pretraining yields the worst results, with disorganized and poorly separated clusters on QM8 (DB index: 5.42), highlighting the necessity of pretraining representation learning for effective molecular representation. 2) The performance of MCL with augmentation and MCL without augmentation is comparable. On one hand, proper molecule graph augmentation helps the GNN identify similar and dissimilar negative pairs based on the original and pruned networks. Additionally, we design the bi-scaled representation alignment mechanism to ensure consistency between representations from the same graph (i.e., molecule) or motif (i.e., functional group). Since the molecular representation (e.g., Aspirin’s representation H1) or functional group’s representation (e.g., Benzene ring’s representation S1) is a vector composed of multiple values, the lines on the molecular representations in panels c and d are used to separate different values. 3) MotiL achieves clear clustering results on the four datasets. For instance, on ESOL, the molecules with four different scaffolds, such as those containing naphthalene (colored yellow) and barbituric acid (colored red), are clearly separated in the representation space. Also, Supplementary Fig. 8 indicates that the diffusion priming phase used in MotiL effectively mitigates the sensitivity of our method to structural noise perturbations.

Next, we plot the t-SNE representation visualization results (Fig. 2b, Supplementary Fig. 3) of all molecules on multiple datasets. For a pair of molecules, we measure their structural similarity by calculating the RDKit fingerprint (RDKFP) similarity11 based on the Tanimoto coefficient37 between their fingerprints. From Fig. 2b, we can see that molecular pairs with high RDKFP similarity (above 90%) tend to be closely positioned in the representation space on the BBBP dataset. In addition, we insert an augmented molecule to verify the effect of molecule graph augmentation. The augmented molecule is obtained from the molecule with the compound identifier (CID) of 6196 through subgraph removal and is highlighted with the brown dashed box. Since the removed amide group is a key functional group of the original molecule, the RDKFP similarity between the augmented molecule and the original one is less than 75%. Correspondingly, the augmented molecule is projected farther from the original one in the representation space than another molecule with a similarity of 0.951. Interestingly, the polarity of the amide group, its ability to form hydrogen bonds, and its potential to increase molecular rigidity generally reduce a molecule’s permeability through the blood-brain barrier15, which is the molecular property predicted in BBBP. Such results demonstrate that MotiL captures the structural and functional properties of small molecules, and can distinguish between easily confusable molecule pairs constructed via augmentation. More visualization results and analysis can be found in Supplementary Section G.

Finally, we analyze the relationships among 10 common functional group representations by calculating their cosine similarity (Fig. 2c). The results are presented in the form of heatmap where the x-axis and y-axis correspond to the different functional group representations from Gθ and Gθ0, respectively. We observe that the representations of the same functional group remain similar although they are from different encoders, while those of different functional groups are mostly dissimilar. Such results reflect the different roles that different functional groups play in forming molecule graph representations. Also, some functional groups exhibit higher similarity due to overlapping atoms and bonds in their chemical structures. For example, carboalkoxy, carbonyl, and carboxyl groups all contain carbon-oxygen double bonds, resulting in pairwise representation similarities of no less than 80%, forming a 3×3 block (framed in gray) in the heatmap. Similarly, the carboxamide, amidine, and secondary ketimine groups, which share carbon-nitrogen single or double bonds, exhibit representation similarities of no less than 70% (framed in black). The above analysis demonstrates that the bi-scaled learning mechanism in MotiL enables the learned motif representations to align with the structural properties of functional groups.

MotiL learns macromolecular representations consistent with protein structures and chemical properties
For visualization, we use t-SNE for small molecule embeddings due to its strength in preserving local structure, which aligns with our goal of examining scaffold-level clustering. For protein embeddings, we employ UMAP, which better maintains both local and global relationships in high-dimensional data. This choice improves interpretability when analyzing functional similarity in multi-label classification tasks such as GO and EC prediction. Specifically, we use a non-linear dimension reduction technique named uniform manifold approximation and projection (UMAP)38 to project the ATP binding protein representations learned by MotiL onto a 2D coordinate space (Fig. 3a). These representations are obtained from the last convolution layer of Gθ in MotiL. We apply the density-based spatial clustering of applications with noise (DBSCAN)39 clustering algorithm to identify and visualize the cluster IDs of individual proteins. For example, when we focus on Cluster 6, where the shape indicates whether the ATP binding of a protein is predicted correctly, it is evident that proteins 1WLE-A and 3LSS-A are grouped together, and all proteins in this cluster are predicted correctly.

Next, we plot the heatmap of Jaccard similarity for the GO terms related to the molecular function within the clusters containing 10 to 40 proteins, specifically Clusters 1, 6, 7, 9, 10, and 11 (Fig. 3b). We observe that the similarity is generally high within each cluster and relatively low between clusters (Fig. 3b, Supplementary Fig. 4), indicating that the learned representation of each protein is consistent with its molecular functions. We then focus on specific cases, examining the three-dimensional (3D) structures of proteins (Fig. 3c, Supplementary Fig. 5). Figure 3c illustrates that within Cluster 3, proteins 4AU8-A, 1G3N-A, 2BPM-A, 5XVU-A, and 3VN9-A display significant structural similarity to each other. Additionally, these proteins in the same cluster are linked by shared GO terms, such as protein kinase activity. Moreover, we perform GO term analysis for multiple clusters (Supplementary Fig. 6) and find that different clusters are associated with distinct molecular functions. Finally, we calculate the similarity scores among the representations of 20 amino acids and present them in the similarity heatmap (Fig. 3d, Supplementary Fig. 7) where the x-axis and y-axis correspond to the different amino acid representations from Gθ and Gθ0, respectively. We can see that the representations of the same amino acid across different encoders are highly similar, whereas different amino acids show lower similarity. Also, MotiL accurately identifies molecular structures with similar aromatic rings, structural isomers, and different side chain structures as highly similar, similar, and dissimilar, respectively (Supplementary Fig. 7).

MotiL corrects the erroneous prediction of the strongest baseline
We present a comprehensive evaluation of prediction disagreements between MotiL and the strongest baseline KANO on two toxicity-related benchmarks: blood-brain barrier permeability (BBBP) and drug-induced liver injury (SIDER). Figure 4a displays quantitative statistics of disagreement cases between MotiL and KANO on the complete datasets. On the BBBP dataset, the two models produce conflicting predictions on 57 out of 203 test samples (28.1%). Among these, MotiL correctly predicts 40 instances (70.2%), while KANO is correct in only 17 (29.8%). On the SIDER dataset, the models disagree on 63 of 295 samples (21.4%), with MotiL accurately predicting 43 samples (68.3%) and KANO only 20 (31.7%). Figure 4b, Supplementary Fig. 10, and Supplementary Fig. 11 present the results of the activity cliff experiment, designed to evaluate model robustness on challenging molecular pairs with high structural similarity but divergent labels. We identified activity cliff samples from BBBP and SIDER by selecting molecular pairs with a Tanimoto similarity above 0.85 and differing binary labels. Among the disagreement cases on these activity cliffs, MotiL achieves higher accuracy than KANO: on BBBP, MotiL correctly predicts 10 out of 15 samples (66.7%) compared to KANO’s 5 (33.3%); on SIDER, MotiL is correct on 11 of 19 samples (57.9%), while KANO predicts 8 correctly (42.1%). The results in Fig. 4a and Fig. 4b demonstrate that MotiL outperforms KANO in handling ambiguous or challenging samples. Figure 4c and Supplementary Fig. 9a illustrate molecular structures from selected cases in the BBBP dataset where MotiL outperforms KANO. Streptomycin, a large and highly polar aminoglycoside antibiotic, has poor BBB permeability due to its molecular weight (581 Da) and multiple hydrophilic groups. While KANO misclassifies it as permeable, MotiL correctly predicts its non-permeability. Mexazolam, a benzodiazepine derivative, exhibits high lipophilicity and low molecular weight, making it BBB-permeable. MotiL correctly identifies this molecule’s property, while KANO fails. Figure 4d and Supplementary Fig. 9b show representative cases from the SIDER dataset. Erlotinib, a tyrosine kinase inhibitor, contains aromatic rings and is known to form reactive metabolites, contributing to hepatotoxicity. MotiL successfully predicts this risk. Vincristine, with its complex alkaloid structure and nitrogenous rings, has documented hepatotoxic potential due to metabolism-induced stress. Again, MotiL correctly classifies this molecule, while KANO does not. These qualitative results complement our statistical findings and provide insight into MotiL’s decision-making process, especially in chemically intricate or borderline cases.

Discussion
In this work, we present MotiL, a pretraining representation learning method for molecules. MotiL operates in three phases: diffusion priming, bi-scaled training, and task-specific fine-tuning. We integrate the generative diffusion model, network pruning, and bi-scaled training loss into MotiL to enable molecular motif learning. Experimental results demonstrate that MotiL achieves superior performance on 14 small molecule benchmarks and 2 protein macromolecule benchmarks. More importantly, the representations of small molecules, proteins, functional groups, and amino acids learned by MotiL align well with their structural and chemical properties. Our results advance molecular representation learning by yielding embeddings that more faithfully capture underlying chemical structure, motif-level information, and molecular function. This improvement supports more accurate property prediction across small molecules and macromolecules, with potential utility in pharmaceutical research and bioinformatics. There are many promising directions to explore in future work. For instance, exploring more advanced generative models for the diffusion priming phase could further enhance the quality of molecular representations. Also, exploring the hybrid use of generative diffusion models and representation learning for new molecule design can accelerate drug discovery, enhance targeted therapies, and advance personalized medicine. The capability to design molecules may result in more effective treatments, a deeper understanding of biological processes, and new opportunities in synthetic biology and diseased diagnosis.

Methods
In a small molecule graph G, the atom set is represented by V and the chemical bond set by E. In this context, an atom u ∈ V is treated as a node of the graph, while a chemical bond (u, v) ∈ E is considered an edge connecting atoms u and v. The initial node feature matrix is denoted by X and the initial features of a node u are denoted by xu. We denote the adjacency matrix of G as A, where each entry is either 0 or 1: 1 if there is an edge between two nodes, otherwise 0. Therefore, we can represent G=(A,X). For a protein graph, the nodes correspond to amino acid residues, and the edges correspond to peptide bonds. In the following method description, the operations on small molecule graphs are elaborated, while similar operations on protein graphs are not detailed for the sake of brevity.

Diffusion priming
Let the perturbed adjacency matrix and perturbed graph at the t-th time step be At and Gt, respectively. In the diffusion priming phase, the perturbed molecule graph Gt=(At,X) at the T-th time step is produced in the noise addition process, and then fed into Gθ in the noise reduction process (Fig. 1a). The details are as follows:
1. Noise addition process: The generative diffusion model DiffMoM progressively perturbs the adjacency matrix A of the molecular graph G by adding Gaussian noise, where the level of noise is controlled through a predefined diffusion schedule. At each time step t (t ∈ {1, 2, ..., T}), the graph Gt-1 from the previous step is perturbed to produce Gt as follows:
Gt=(A,X),
At = sqrt(αt) * At-1 + sqrt(1 - αt) * st,
where αt is a noise scheduling coefficient and st ~ N(0,I) represents Gaussian noise. The noise scheduling coefficient αt is typically defined as a predefined sequence using the following formula:
αt = 1 - (t/T) * (βmax - βmin) + βmin,
where βmin and βmax represent the minimum amount of noise introduced in the early time step (t = 0) and the maximum amount of noise introduced in the final time step (t = T), respectively.
2. Noise reduction process: In the noise reduction process, the parameterized GNN encoder Gθ is trained to reverse noise perturbations and reconstruct the original graph G from GT. Specifically, at the T-th time step:
s0 = Mϕ(Gθ(AT,X)):
We use the reparameterization trick to avoid redundant computations40,41, i.e., the predicted noise s0 is directly used to calculate the reconstructed adjacency matrix A^ of G^ in one step:
A^ = AT - sqrt(1 - αT) * s0 / sqrt(αT),
3. Training objective: The overall objective is to denoise GT, eventually reconstructing the original molecule graph from pure noise. Therefore, the loss function LDiffMoM of DiffMoM is designed to measure how well Gθ can recover A:
LDiffMoM = ||A - A^||2:

Bi-scaled training
The phase of bi-scaled training includes two main parts: bi-scaled network encoding and bi-scaled representation optimization.
Bi-scaled network encoding. In Gθ, the representation of a node is updated through the message passing mechanism as follows:
h(l+1)v = σ(W(l) * sum(h(l)u) + b(l)),
l ∈ {0, 1, ..., L-1},
where h(l)u is the representation of node u at the l-th layer (h(0)u = xu), N(v) is the neighbor set of node v, W(l) is the weight matrix at the l-th layer, b(l) is the bias vector at the l-th layer, and σ is an activation function like the rectified linear unit (ReLU). For an L-layer GNN encoder Gθ, we define the parameter group θ = {W(1), W(2), ..., W(L)}.
To prune GNN, we apply random masking to its weight matrix W(l). Given the retaining probability p, the weight matrix W(l)prune after masking is defined as follows:
W(l)prune = W(l) ⊙ Mp,
where ⊙ denotes element-wise Hadamard product. Mp is a binary masking matrix with the same shape as W(l). Each entry in Mp is 1 with probability p (indicating that the corresponding weight is retained) and 0 with probability 1-p (indicating that the corresponding weight is pruned). Similarly, in a GNN encoder Gθ0, the update formula for node representation is:
h~(l+1)v = σ(W(l)prune * sum(h~(l)u) + b(l)),
l ∈ {0, 1, ..., L-1},
where h~(0)u = xu. Also, we define the parameter group θ0 = {W(1)prune, W(2)prune, ..., W(L)prune}. Finally, the output node representation of two encoders are defined as hv = h(L)v and h~v = h~(L)v.

Bi-scaled representation optimization. After obtaining node representations, given the graph G and motif Gmotif, we can further derive the graph representation H and motif representation S using a summing-average graph readout function:
H = 1/|V| * sum(hv), S = 1/|Vmotif| * sum(hv),
k ∈ V, k ∈ Vmotif
Another encoder, G0θ, produces graph and motif representations in the same manner as described in Eq. (9). For brevity, the details are omitted here.
Then the loss Lbi-scale during the bi-scaled training phase is defined as:
Lbi-scale = LG + α * LGmotif,
where α is a balance coefficient and LG and LGmotif are the graph-level and motif-level training loss. LG and LGmotif are defined based on the Barlow Twins loss42,43:
LG = sum(1 - Ck,k)^2 + sum(sum(C^2i,j)),
LGmotif = sum(1 - C0k,k)^2 + sum(sum(C0^2i,j)),
where D is the representation dimension. Let the number of graphs and motifs be Ngraph and Nmotif, respectively, C and C0 are calculated as follows:
C = 1/Ngraph * sum(Hnorm * Hnorm),
C0 = 1/Nmotif * sum(Snorm * Snorm),
where Hnorm and Snorm represent the normalized representations of graphs and motifs, respectively. We denote the normalized representations of the i-th graph and i-th motif as Hi,norm and Si,norm in Hnorm and Snorm, respectively. They are derived from the i-th graph representation Hi and i-th motif representation Si (without loss of generality, Hi and Si are defined similarly to Eq. (9)):
Hi,norm = (Hi - μi) / (σi + ϵ), Si,norm = (Si - μ0i) / (σ0i + ϵ),
where μ, μ0 are mean values, σ, σ0 are standard deviation values, and ϵ is a small positive constant added to prevent overflow due to division by zero.

Task-specific fine-tuning
In the third phase, we apply the pretrained Gθ to the molecular property prediction task to fine-tune its parameter group θ.
For the classification task, given a new molecule graph G0=(A0,X0), we first obtain its logit vector y^ = [y^1, y^2, ..., y^C] (C is the number of classes) by using Gθ in combination with an MLP-based classifier Mψ:
y^ = Softmax(Mψ(Gθ(A0,X0))):
Then we minimize the cross-entropy loss LCE between y^ and the true label’s one-hot vector y=[y1, y2, ..., yC]:
LCE = - sum(yc * log(y^c)):
For the regression task, given a new molecule graph G0=(A0,X0), we first obtain its predicted value y^ by using Gθ in combination with an MLP-based classifier Mω:
y^ = Mω(Gθ(A0,X0)):
Then we minimize the mean square error loss LMSE between y^ and the true score y of G0:
LMSE = (y^ - y)^2:

Datasets
1. Small molecule datasets: For MotiL pretraining, which includes diffusion priming and bi-scaled training, we sample 250,000 unlabeled molecules from ZINC1521, a publicly accessible database of purchasable drug-like compounds. For task-specific fine-tuning, we utilize 14 benchmark datasets from MoleculeNet22, encompassing 678 binary classification tasks and 19 regression tasks across domains such as drugs, biology, physics, and chemistry. Following the methods in previous work11,12, we conduct three independent runs using three random-seeded scaffold splits for all datasets, except QM9, with a training/validation/testing ratio of 8:1:1. Scaffold splitting44 is a more challenging task that partitions molecules based on their scaffolds, offering a better assessment of model generalization on out-of-distribution data. QM9 follows the random splitting strategy commonly used in related works11,12,30 for comparison.
2. Protein macromolecule datasets: The task in the Gene Ontology (GO) dataset focuses on predicting protein functions across multiple GO terms. Following ref. 34, proteins are divided into hierarchically related functional classes within three ontologies: biological process (BP) with 1943 classes, molecular function (MF) with 489 classes, and cellular component (CC) with 320 classes. The GO dataset consists of 29,898 proteins for training, 3322 for validation, and 3,415 for testing. The task in the Enzyme Commission (EC) dataset focuses on predicting three-level and four-level EC numbers, following the splits in ref. 34, with 15,550 proteins for training, 1729 for validation, and 1919 for testing. EC number prediction is also a multi-label classification task. We use 1D sequential orders and 3D geometric coordinates of amino acids from the protein datasets for MotiL pretraining, and incorporate the label information of GO terms or EC numbers during the task-specific fine-tuning phase. Further details about the experimental datasets are provided in Supplementary Section B.

Baselines
For the small molecular property prediction task, we compare MotiL with 6 supervised learning methods and 10 self-supervised learning methods. The supervised learning methods include two widely used GNNs (GCN45 and GIN46), one geometry-based GNN (GEM10), and three message passing neural networks (MPNN47, DMPNN48, and CMPNN49). The self-supervised learning methods include four predictive self-supervised learning methods (N-GRAM30, Hu et al.29, MGSSL28, and GROVER27) and five contrastive self-supervised learning methods (GraphMVP50, MolCLR11, MolCLR CMPNN 11, and KANO12). For the protein macromolecular property prediction task, we compare MotiL with 4 sequence-based methods, 3 graph-based methods, and 8 hybrid methods. Specifically, sequence-based methods include CNN25, ResNet26, LSTM26, and Transformer26; graph-based methods include GCN45, GAT51, and 3D CNN52; hybrid methods include GraphQA53, GVP54, IEConv residue level 55, GearNet56, GearNet IEConv 56, GearNet Edge 56, GearNet Edge-IEConv 56, and CDConv24. Further details about baselines and related work are provided in Supplementary Section A.

Training details
We use CMPNN49 and CDConv24 as the specific GNN encoders in MotiL for micro- and macromolecule tasks, respectively. CMPNN enhances graph representations by strengthening the message interaction between atoms and bonds. CDConv applies discrete and continuous convolutions to model regular and irregular structures, respectively. In the small molecule dataset, the initial features of atoms are determined by eight attributes (e.g., chirality, hybridization, and atomic mass), while bonds are represented by four attributes (e.g., bond type and conjugation). For the protein macromolecule dataset, we use the sequence and geometric structure information of proteins as inputs to CDConv.
During the diffusion priming phase, we optimize the loss LDiffMoM using the Adam optimizer57 with a learning rate of 3e-5. The iteration number is set to T=1000 and the MLP encoder Mϕ is a 3-layer network with hidden sizes of 300, 300, and 50. In the bi-scaled training phase, the Adam optimizer with a learning rate of 3e-5 is used to optimize the bi-scaled training loss Lbi-scale. Gθ is trained with a batch size of 1024 over 50 and 5 epochs for micro- and macromolecules, respectively. The balance coefficient α in Eq. (10) is set to 1.0. In the task-specific fine-tuning phase, we add a two-layer MLP on top of the pretrained Gθ for molecular property prediction. We fine-tune Gθ for Ne epochs (Ne ∈ [10, 200]) during each independent run. The Adam optimizer is used to update Gθ with a learning rate ranging from 1e-6 to 1e-3 for all datasets. Fine-tuning is performed only on the training set, with hyperparameter tuning on the validation set to achieve optimal results. MotiL was implemented using PyTorch (version 1.12.1) and other open-source libraries including PyTorch Geometric (version 2.3.0), RDKit (version 2022.9.1), and scikit-learn (version 0.24.2). All experiments were conducted on an Ubuntu server equipped with NVIDIA GeForce RTX 3090 GPUs.

Peer review rounds
[{"round": 1, "reviewer_comments": {"reviewer_1": "### Summary\nThe authors present a framework, MotiL, for molecular representation learning. The goal of the proposed method is to ensure\nhigher-fidelity modelling of local structures (i.e. motifs) in molecules as well as richer global representations. To achieve this,\nMotiL consists of three training stages: warmup through employing an edge-based based denoising diffusion model, bi-\nscaled contrastive training, and finetuning on supervised downstream tasks. The method shows consistent (but marginal)\nimprovements over existing approaches and the relevancy to both the domains of small molecules and macromolecules is a\nnice aspect of the study. The authors perform subsequent interrogations of the model to evaluate the extent to which the\nlearned representations capture biological/chemical semantics.\n### Major\nFor the MoleculeNet-based benchmark datasets, do the authors perform any further processing or standardisation? At this\npoint, it is well known that these datasets contain erroneous labelling and duplicate examples as well as non-standardised\nstructures [1]. I think benchmarking on these types of datasets is fine (if not super meaningful) for a ML conference-type\npublication but hinders the case for publication in a journal with a broader audience. Furthermore, these benchmarks are in\ngeneral quite uninformative for real-world applications. Have the authors considered benchmarking on more application-\nfocussed datasets such as [2]?\nOverall, I believe the impact is limited. While the proposed method does consistently improve on previous methods, there is\nonly a marginal improvement over KANO in the small molecule tasks. And the benefit on the macromolecular tasks is less\nclear when compared to pre-trained structural encoders. Taken together with the weakly motivated benchmarks I think this\nincremental performance improvement does not warrant publication in a journal with such broader readership. As it stands, I\nthink there is little for non-ML researchers to take from this paper and so targeting a venue with a more ML-focussed\nreadership would be more suitable for this particular work.\n### Minor:\n* L45 (+L89) I'd rephrase the use of the word \"generate\" here to avoid conflation with generative modelling.\n* The authors state current methods struggle to effectively capture representations of local motifs. Is there evidence to\nsupport this claim?\n* Contrastive learning is described twice in the introduction. I would recommend tightening the prose here a little.\n* The introduction implicitly assumes structure-based GNN encoders are the only line of research here by not considering\nalternate representation such as sequence or SMILES that do not require GNNs. These methods also often enable access\nto much larger quantities of data for training and pre-training which is a trade-off that should be mentioned.\n* L136 \"Gaussian noise into G in a controlled manner.\" This is a shallow description. The authors should describe that the\nadjacency matrix is noised.\n* L148. Grammar. \"the local feature\" -> the local features.\n* L149. Typo \"encodes\" -> encoders\n* L245-250 this is quite repetitive of what has been previously stated.\n* In Table 2 the authors are comparing to the variants of GearNet without pre-training. This seems like an unfair comparison.\nAdditionally, the authors should denote the source of the results for the baseline methods if they did not run them\nthemselves.\n* L282 EC prediction is a classification task.\n* L309. Rephrase messy clustering.\n* The authors make frequent to reference to their proposed methods robustness to noise but do not perform any experiments\nto demonstrate this. Nor do they the describe the source of the noise they are referring to. Surely, given that SE(3) invariance\nis built into the model, the effect of structural noise is likely to be minimal? As far as I can tell from the description,\ncoordinates are used in to model.\n* What is the justification for using t-SNE for the small molecule embeddings and UMAP for the protein embeddings?\n* In figure 3a it is a little odd to start the cluster IDs from -1.\n* What do the green points along the X axis in the inset panel in Figure 3a indicate?\n* I actually disagree with the authors claim that the similarity of the representations of the amino acids reflect their similarity\noutside of the authors cherry-picked examples. For example, Serine and Tryptophan and the low similarity of Alanine and\nGlycine.\n* L458 \"Fig 4.\" is repeated.\n* Figure 4 is a little hard to gain significant insight from. Whilst it is nice to see specific examples where the models disagree,\ncherry picking favourable cases is second-best to a more quantitative summarisation of the disagreements. I believe such an\nanalysis should replace this figure, and figure 4 could be an inset panel or moved to the supplementary.\n* The authors should specify a license for the source code.\n* Supplementary L68: I think some evidence (or elaboration) needs to be provided for the claim that \"the negative effects of\nmolecular graph augmentation have hindered the advancement of MCL\".\n* \"L120. To the best of our knowledge, we are the first to propose the concept of molecular motif learning.\". Could the authors\nplease elaborate on this and clarify what exactly is being claimed? I could find a few works that purport to address this to\nsome degree:\n[1] https://practicalcheminformatics.blogspot.com/2023/08/we-need-better-benchmarks-for-machine.html\n[2] [https://doi.org/10.1021/acs.jcim.3c00160]https://doi.org/10.1021/acs.jcim.3c00160\n[3] https://arxiv.org/abs/2012.12533\n[4] https://arxiv.org/abs/2012.12533", "reviewer_2": "The work developed by Ziyang Liu and co-workers focuses on a new representation learning method, MotiL, which\nleverages graphs and motifs for representation learning. I agree with the authors that property prediction using ML can\nsignificantly advance drug discovery by reducing the need for experimentally demanding wet-lab methods.\nThe article is well-written, and the description of the algorithm is clear and insightful. The authors provide the necessary\ninformation for a general audience. I find it particularly important that both small-molecule and biomolecular benchmarking\npredictions were included, demonstrating the model’s flexibility. However, I struggled to fully agree that this new method\nstands out in terms of predictive performance compared to existing methods. Nonetheless, I believe this article makes a new\nand valuable contribution to the field by introducing motif learning. Below are some points for consideration:\n1. It would be helpful to clarify why the authors chose MoleculeNet.\n2. It was unclear from the main text that the metrics obtained for other methods in Tables 1 and 2 were not calculated by the\nauthors but rather sourced from previous studies ([12], [27], [24]). Since the metrics for other models were taken from past\narticles, how can the authors guarantee that the comparison is fair and not influenced by differences in training and\nevaluation methods? Is the cross-validation method identical? Additionally, this approach limits further comparison. Also, it\nmight be harder to answer why RMSE was used to evaluate physical chemistry properties and MAE for quantum properties\nand why more metrics for regression and classification weren’t explored?\n3. In the SI, it is mentioned that the standard deviations (std) were calculated from 3 independent runs. Could the authors\nclarify what this means? Given the std values, is it possible to perform statistical analyses? Since the paper claims that MotiL\nsurpasses other models, is this improvement statistically significant?\n4. The study in SI section F, which evaluates the different components of MotiL, is interesting and important. I agree that the\nresults suggest that the full MotiL architecture performs better, especially in regression tasks. It would be beneficial, if\npossible, to add a sentence to the main text summarizing the key conclusions from these ablation studies.\n5. I find Figure 2a very insightful, as it clearly shows that MotiL can distinguish different molecular scaffolds. However, I\nsuggest using more contrastive colors, particularly in the BBBP plots, where some shades (e.g., pinks) are too similar.\nAdditionally, what was the rationale behind choosing the specific molecules, scaffolds, and database for Figure 2a?\n6. In Figure 2b, and especially in Figure 3 in the SI, some dashed lines are difficult to see.\n7. In not sure if I fully understand the data behind Figure 3a. UMAP and DBSCAN were used to project the representations\nlearned by MotiL for ATP-binding proteins. Why was this specific class chosen? Also, does the shape in the figure indicate\nwhether a prediction is correct in identifying an ATP-binding protein?\n8. Figure 3c could be better organized. I prefer the layout used in Figure 5 SI, and I think more GO terms should be written to\nhighlight similarities. For example, in ID 3, the common GO terms, protein kinase activity, Kinase activity, and Protein\ntyrosine kinase activity, could be included.\n9. The authors also make a more in-depth comparison between the 2nd best algorithm (KANO) and MotiL, but I struggle to\nunderstand the significance of this section. In fact, improvements between KANO and MotiL appear to be marginal. While I\nacknowledge that MotiL correctly predicts BBB permeability and toxicity for key molecules while KANO does not, I find this\nsection unnecessarily long and less relevant. I believe a more challenging comparison could be beneficial. For example,\nevaluating performance on activity cliffs.\n10. I appreciate the author’s commitment to improving the model in future work, with already some defined future steps that\ncould further accelerate drug discovery.\n11. Finally, I recommend revising the references. For example, reference [1], Diaz, N. et al. Discovery of potent small\nmolecule inhibitors of lipoprotein (a) formation. Nature 1–6 (2024), it’s written on the journal’s website as Diaz, N. et al.\nDiscovery of potent small-molecule inhibitors of lipoprotein(a) formation. Nature 629, 945–950 (2024). Reference [7] Wan, Z.\net al. Unconventional superconductivity in chiral molecule-tas2 hybrid superlattices. Nature 1–6 (2024) it’s written on the\njournal’s website as Wan, Z, et al. Unconventional superconductivity in chiral molecule–TaS2 hybrid superlattices. Nature\n632, 69–74 (2024).\nGiven the increasing importance of ML in advancing drug discovery and the innovative methodology presented in this work,\nI recommend this article be accepted for revision."}, "author_response": "Dear Reviewers,\nWe sincerely thank the reviewers for the thorough evaluation of our manuscript and for recognizing the significance\nof our work. We are also grateful for the reviewers’ constructive comments and insightful suggestions, which have\nhelped us identify areas for improvement and significantly enhanced the quality of our manuscript. In the revised\nmanuscript, we have carefully addressed all comments and highlighted the corresponding changes in blue font for\nclarity. In this response letter, we provide a point-by-point reply to each of the comments from Reviewer #1 (pages\n1 to 24) and Reviewer #2 (pages 25 to 45), respectively.\nResponse to Reviewer #1:\nComment 1: For the MoleculeNet-based benchmark datasets, do the authors perform any further processing or\nstandardization? At this point, it is well known that these datasets contain erroneous labelling and duplicate examples\nas well as non-standardized structures [1]. I think benchmarking on these types of datasets is fine (if not super\nmeaningful) for a ML conference-type publication but hinders the case for publication in a journal with a broader\naudience. Furthermore, these benchmarks are in general quite uninformative for real-world applications. Have the\nauthors considered benchmarking on more application-focused datasets such as [2]? Overall, I believe the impact is\nlimited. While the proposed method does consistently improve on previous methods, there is only a marginal\nimprovement over KANO in the small molecule tasks. And the benefit on the macromolecular tasks is less clear\nwhen compared to pre-trained structural encoders. Taken together with the weakly motivated benchmarks I think\nthis incremental performance improvement does not warrant publication in a journal with such broader readership.\nAs it stands, I think there is little for non-ML researchers to take from this paper and so targeting a venue with amore\nML-focused readership would be more suitable for this particular work.\n[1] https://practicalcheminformatics.blogspot.com/2023/08/we-need-better-benchmarks-for-machine.html\n[2] https://doi.org/10.1021/acs.jcim.3c00160\nReply:\nWe sincerely thank the reviewer for this constructive and detailed feedback. We have carefully addressed each aspect\nof the comment and have made significant clarifications and additions in the revised manuscript, summarized below:\n1. On the use of MoleculeNet-based benchmark datasets: We acknowledge the reviewer’s concern regarding\nthe known limitations of MoleculeNet datasets, such as erroneous labels, duplicated entries, and lack of\nmolecular standardization [1]. As with many previous studies (e.g., GROVER, MolCLR, KANO), our intention\nin using MoleculeNet is to ensure fair and directly comparable benchmarking under widely recognized settings.\nThese benchmarks, while imperfect, continue to serve as a de facto standard in the field of molecular machine\nlearning, offering controlled comparisons across methods. We emphasize that we did not apply any additional\nprocessing or standardization to these datasets beyond the standard splits provided, in order to strictly follow\nthe established evaluation protocols used by prior work. This design choice facilitates meaningful comparisons\nwith strong baselines and highlights the incremental benefits offered by our method within this common\nframework.\n2. On evaluating with more application-oriented datasets: We appreciate the reviewer’s excellent suggestion\nto evaluate MotiL on more realistic, application-focused datasets. To this end, we have conducted new\nexperiments on a benchmark introduced by [2], which focuses on absorption, distribution, metabolism, and\nexcretion (ADME) properties that are central to real-world drug development and pharmacokinetics. There are\nsix ADME-related prediction tasks in this dataset: HLM clearance, human plasma protein binding (hPPB),\nMDR1-MDCK efflux ratio, rat liver microsome (RLM) stability, rat plasma protein binding (rPPB), and\naqueous solubility. These properties are clinically relevant and span diverse pharmacological mechanisms. We\ncompare our method against strong baselines (CMPNN, MGSSL, MolCLR, KANO), and the RMSE results\nunder three independent runs using three random-seeded are summarized in Table 1 (lower RMSE value\nindicates better performance):\nTable 1: Prediction result comparison of different methods on ADME datasets.\nThe results demonstrate that MotiL achieves consistently stable and competitive performance across all\nsix ADME-related prediction tasks. On most tasks, MotiL not only achieves strong predictive performance but\nalso maintains low standard deviations (e.g., ±0.0054 for HLM and ±0.0047 for Solubility), indicating its\nrobustness and reproducibility. In Table 1, we use bold font to denote the best performance on each task and\nitalic font to indicate the second-best. Statistical analysis further confirms that MotiL significantly outperforms\nthe second-best method (e.g., KANO on the task of hPPB) on each task (p < 0.05), demonstrating its clear\nsuperiority. Overall, MotiL consistently ranks among the best-performing methods with minimal fluctuations\nacross diverse pharmacokinetic properties, highlighting its strong generalization ability. Moreover, MotiL\nhandles these tasks within a unified framework without the need for task-specific tuning, demonstrating its\npractical utility for real-world drug discovery applications.\n3. On the perceived marginal improvement and general impact:\n(1) Statistically significant improvements over KANO: While the numerical differences between MotiL and\nKANO on small-molecule tasks may seem modest in some cases, we emphasize that these improvements are\nstatistically significant and consistently reproducible. To rigorously verify it, we conducted paired two-tailed t-\ntests across all 14 MoleculeNet benchmark datasets comparing MotiL with the strongest baseline, KANO. The\nresults, presented in Table 2 and Table 3 (also included as Supplementary Tables 3 and 4), show that all p-\nvalues are below 0.05, confirming that the improvements are statistically significant and unlikely to result\nfrom random fluctuations. Moreover, the standard deviations across multiple runs are consistently low\n(typically <1.5%), indicating that the performance is stable across different random seeds and initializations.\nTherefore, the observed gains, though sometimes small in absolute value, are consistent, statistically valid, and\nreproducible, reflecting the robustness of our approach.\n(2) Addressing both micro- and macromolecular property prediction: Beyond numerical gains, the key\ncontribution of MotiL is its unified framework that supports both small-molecule and macromolecule\nproperty prediction tasks, which most prior methods cannot do. For example, models such as KANO and\nMolCLR are designed specifically for small-molecule graphs and do not generalize to macromolecular graphs\n(e.g., peptides, nucleotides). Some macromolecular methods rely on pre-trained protein language models or\n3D structural encoders. However, these methods require expensive and curated protein structures; cannot\nhandle small-molecule graphs effectively; lack a unified representation space for both molecular types. In\ncontrast, MotiL jointly pre-trains on small-molecule and macromolecular graphs using both 1D sequences and\n3D geometric information, enabling it to generalize across diverse chemical spaces without task-specific tuning.\nThis versatility meets real-world drug discovery needs, supporting both small-molecule ADME property\nprediction and macromolecular interaction modeling, even when structural data are incomplete.\nTable 2: ROC-AUC (%) performance comparison on small molecules. The testing results are presented as the\nmean and standard deviation, calculated from three independent runs. The best result for each dataset is\nhighlighted in bold, and the runner-up result is italicized. All baseline results are directly taken from Ref. [23],\nwhich follows the same evaluation protocol as this study: three random-seeded scaffold splits for small molecules.\nTable 3: Performance comparison on small molecules. We use the metric of RMSE or MAE where a lower value\nindicates better performance. The testing results are presented as the mean and standard deviation, calculated from\nthree independent runs. The best result for each dataset is highlighted in bold, and the runner-up result is italicized.\nAll baseline results are directly taken from Ref. [23], which follows the same evaluation protocol as this study:\nthree random-seeded scaffold splits for small molecules.\n[1] https://practicalcheminformatics.blogspot.com/2023/08/we-need-better-benchmarks-for-machine.html\n[2] https://doi.org/10.1021/acs.jcim.3c00160\nTo better reflect these distinctions, we have revised the manuscript (Supplementary Section G) to include:\n(1) a more explicit discussion of the limitations of MoleculeNet and motivation for ADME evaluation; (2) an\nexpanded section comparing real-world ADME performance; (3) additional clarity in the scope, positioning,\nand impact of our work across application contexts.\nWe have also incorporated the above content into Supplementary Section I (Expanded discussion on\nbenchmark limitations and real-world ADME evaluation) as follows:\nSection I.1 (Limitations of MoleculeNet benchmarks): “While MoleculeNet remains the most widely\nadopted benchmark suite for evaluating molecular property prediction models, its use is not without challenges.\nPrevious studies and domain experts have highlighted issues such as label noise, duplicate molecules, and\nstructural inconsistencies (e.g., protonation states, tautomers, stereochemistry) within its datasets [72]. These\nlimitations can affect the reliability of comparative evaluation, especially for tasks requiring high-fidelity\nchemical understanding. Nevertheless, due to its established baselines and broad community adoption,\nMoleculeNet provides a useful foundation for measuring progress in molecular machine learning under\nstandardized settings. To maintain comparability with prior work such as GROVER, MolCLR, and KANO, we\nused the unmodified MoleculeNet datasets, adhering to the same data splits and evaluation protocols. This\nenables fair head-to-head comparisons while acknowledging the benchmark’s inherent constraints.”\nSection I.2 (Motivation for evaluating on ADME datasets): “To move beyond synthetic benchmarks and\ntoward more application-driven evaluation, we conducted additional experiments on six ADME-related\ndatasets [73]. These datasets reflect core pharmacokinetic properties relevant to drug development, including\nmetabolism (HLM, RLM), plasma protein binding (hPPB, rPPB), membrane permeability (MDR1-MDCK ER),\nand aqueous solubility. Unlike MoleculeNet tasks, these ADME endpoints are measured in realistic laboratory\nor clinical settings, often featuring complex structure-activity relationships and significant data heterogeneity.\nThey serve as practical testbeds for evaluating a model's ability to generalize to pharmacologically meaningful\nand commercially relevant tasks.”\nSection I.3 (Comparative analysis and discussion): “We report the performance of MotiL and four baseline\nmethods (CMPNN, MGSSL, MolCLR, and KANO) on the six ADME prediction tasks, with results summarized\nin Table 5. All metrics are averaged over three independent runs, and standard deviations are reported to assess\nstability. All baseline models are implemented using publicly available code with their default hyperparameter\nconfigurations. The results demonstrate that MotiL achieves highly stable and competitive performance. While\nKANO achieves relatively good results on certain tasks, its results often come with larger variances (e.g.,\n±0.0225 for HLM and ±0.0588 for Solubility), suggesting potential inconsistency. In contrast, MotiL exhibits\nlow standard deviations across most tasks (e.g., ±0.0054 for HLM and ±0.0047 for Solubility), indicating\nsuperior robustness. Although the absolute performance gap between MotiL and top-scoring models may\nappear marginal on some tasks, MotiL consistently ranks among the best-performing methods with minimal\nfluctuation, highlighting its strong generalization ability. Statistical analysis further confirms that MotiL\nsignificantly outperforms the second-best method (e.g., KANO on the task of hPPB) on each task (p < 0.05),\ndemonstrating its clear superiority. Moreover, MotiL’s ability to handle diverse pharmacokinetic properties\nwithin a unified framework, without the need for task-specific tuning, emphasizes its practical utility for real-\nworld applications in drug discovery.”\nSection I.4 (Broader impact and positioning): “By including the evaluation on real-world datasets and\nexplicitly acknowledging the known limitations of earlier benchmarks, we aim to position MotiL as a bridge\nbetween foundational research in molecular representation learning and its applications in drug discovery\npipelines. Our framework supports both small molecule and macromolecular tasks within a unified architecture\nand demonstrates improvements in consistency, interpretability, and practical relevance. We hope these\nadditional experiments and clarifications provide greater context for our design choices and reinforce the\nbroader significance of our contributions.”\nWe have also added the following experimental conclusion into the Section of “MotiL boosts the\nperformance of molecular property prediction” in the main text:\n“Our experiments on absorption, distribution, metabolism, and excretion (ADME) prediction tasks\n(Supplementary Table 5) show that MotiL consistently delivers robust and competitive performance, achieving\nstrong predictive accuracy and lower variance compared to baseline models. These results highlight the\npotential of MotiL for pharmacologically relevant applications.”\nThe corresponding statistical analysis on the MoleculeNet benchmark dataset has been included in the\nrevised manuscript under Supplementary Section D (Complete performance comparison on small molecules),\nas presented below:\n“To assess whether the observed performance gains of MotiL over the strongest baseline (KANO) are\nstatistically significant, we performed paired two-tailed t-tests across all MoleculeNet benchmark datasets\nusing the results from three independent runs. The corresponding p-values are summarized in Table 3 and Table\n4. All values are below the 0.05 threshold, confirming that MotiL’s improvements are statistically significant\nand robust across different datasets.”\nComment 2: L45 (+L89) I'd rephrase the use of the word \"generate\" here to avoid conflation with generative\nmodelling.\nReply:\nWe thank the reviewer for highlighting the potential ambiguity caused by our use of the word “generate” in the\ncontext of molecular and protein representation learning (Lines 45 and 89). We agree that the term may inadvertently\nsuggest a connection to generative modeling, which is not the intended meaning in our work.\nTo avoid this conflation and to align with standard terminology in the field, we have revised the manuscript to\nreplace “generate” with “produce” where appropriate. Specifically, we made the following changes:\n1. Line 45: “Contrastive learning methods [11, 12] typically produce two augmented versions of a molecular\ngraph through techniques like subgraph removal or perturbation, and then use them to form positive and\nnegative sample pairs.”\n2. Line 89: “At the graph level, MotiL produces similar graph representations for small molecules sharing the\nsame scaffold (e.g., naphthalene and barbituric acid) or for proteins exhibiting similar three-dimensional\nstructures and overlapping chemical functions (e.g., protein kinase activity and tRNA binding).”\nIn addition, we proactively revised several other instances of the term “generate” to “produce” to ensure\nconsistent and unambiguous language throughout the manuscript. Some examples are as follows:\n1. Line 49: “It is crucial for molecular representation learning to produce representations that accurately reflect\nthe underlying chemical structures and functions.”\n2. Caption of Figure 1: “The pretrained GNN encoders produce representations for unseen molecules, which are\nthen fed into a multilayer perceptron classifier to learn the relationship between these graph representations\nand the true molecular property labels.”\nComment 3: The authors state current methods struggle to effectively capture representations of local motifs. Is\nthere evidence to support this claim?\nReply:\nWe thank the reviewer for raising this important question regarding our claim that current methods struggle to\neffectively capture representations of local motifs. Our motivation for this claim is based on both observations from\nprior studies and experimental evidence in our work:\n1. Prior work acknowledges limitations in capturing local motifs: Several recent studies have pointed out that\nmany GNN-based molecular representation models, particularly those relying on global pooling or contrastive\nstrategies over entire graphs, tend to underemphasize fine-grained local substructures. For instance:\n(1) GROVER [1] primarily uses contrastive objectives over whole-molecule views, without explicit\nmodeling of localized functional groups or motifs.\n(2) MolCLR [2] applies contrastive learning on augmented molecular views but does not explicitly model or\npreserve chemically meaningful local motifs.\n(3) KANO [3], while improving representation through domain knowledge, still does not include an explicit\nmechanism for local motif identification or encoding.\nThese approaches often rely on the model to implicitly learn local patterns, which may be insufficient for\ntasks where specific substructures (e.g., pharmacophores, binding pockets) are strongly tied to functional\noutcomes. To address the reviewer’s concern, we have revised the manuscript to more carefully justify this\nstatement (the current methods struggle to effectively capture representations of local motifs) and added\nnecessary citations:\n“However, existing methods struggle to effectively capture the representations of motifs - such as\nfunctional groups in small molecules or amino acid residues in proteins - due to their limited ability to model\nlocalized chemical semantics or hierarchical substructures. For instance, prior contrastive learning methods\n[11, 12, 27] often rely on random subgraph sampling or global-level supervision, which may overlook\npharmacologically or structurally meaningful motifs.”\n2. Evidence from our experiments: To provide empirical evidence supporting our claim that current methods\nstruggle to effectively capture representations of local motifs, we conducted activity cliff experiments as part\nof our evaluation. Activity cliffs refer to pairs of molecules that are highly similar in overall structure (global\nscaffold), yet exhibit large differences in biological activity due to small variations in local motifs. This\nexperimental setup provides a direct way to test whether a model can capture the impact of subtle local changes\non molecular properties. We designed a dedicated activity cliff evaluation protocol as follows:\n(1) We first identified molecular pairs from the BBBP and SIDER datasets that exhibit high structural\nsimilarity (Tanimoto similarity > 0.85) but substantial differences in biological activity labels.\n(2) We then extracted the individual molecules from these pairs to construct the activity cliff test sets,\nfocusing on cases where subtle local differences drive large activity changes.\n(3) Finally, for each dataset, we analyzed the prediction disagreements between KANO and MotiL on these\nchallenging examples to assess their ability to capture local motif variations.\nAs shown in Figure 1, we found that KANO, the best-performing baseline in our main results, exhibited\nsubstantial performance degradation on activity cliff molecule pairs. It indicates that KANO fails to capture\nthe subtle yet critical differences arising from local motifs. These experimental results provide concrete\nevidence that current methods, exemplified by KANO, have limitations in learning local motif representations.\nOn the BBBP dataset, among 15 activity cliff disagreement cases, MotiL correctly predicted 10 pairs (66.7%),\nwhile KANO only predicted 5 correctly (33.3%). On the SIDER dataset, among 19 disagreement cases, MotiL\ncorrectly predicted 11 pairs (57.9%), compared to 8 correct predictions (42.1%) by KANO.\nMoreover, Figures 2 and 3 provide concrete examples of activity cliff pairs. These examples include pairs\nwith Tanimoto similarity between 0.85 and 1.0, where only minor differences in functional groups exist. In\nthese examples, KANO fails to distinguish the structurally similar molecules with distinct activities,\nwhereas MotiL successfully captures subtle motif-level differences in most cases. In the BBBP dataset (Figure\n2), we observe that in the first three molecular pairs, MotiL successfully predicts the correct labels for both\nmolecules in each pair, while KANO misclassifies one molecule per pair. These cases highlight MotiL’s\nstronger sensitivity to local functional group variations, such as substitutions between carboxyl and ester groups,\nor subtle stereochemical differences on steroid-like scaffolds (e.g., a pair with a Tanimoto similarity of 0.851).\nIn contrast, in the last two pairs, both models incorrectly classify one molecule per pair, indicating that certain\nfine-grained structural changes still present significant challenges, even for MotiL, despite its overall superior\nperformance on activity cliff prediction. In the SIDER dataset (Figure 3), several activity cliff pairs with high\nstructural similarity (e.g., 0.956 and 1.0) but opposite toxicity labels are presented. In the first pair (similarity\n= 0.956), MotiL correctly distinguishes between the two molecules, while KANO produces identical predictions,\nfailing to capture the subtle discriminative features. However, in the remaining three pairs, both models exhibit\nmixed outcomes: for each pair, one molecule is misclassified by KANO and the other by MotiL. These\nexamples suggest that both models still face difficulties in fully resolving the subtle chemical cues that\ndefine the activity cliff boundary in complex toxicity scenarios. These results highlight that MotiL is more\nrobust to subtle substructural changes than KANO, which are often the root cause of activity cliffs. We\nattribute this improvement to MotiL’s motif-level representation learning framework and the diffusion-based\ndenoising mechanism, which enhance the model’s ability to focus on local structural variations that drive\nbiological activity.\nPrediction statistics on activity cliff samples\nFigure 1: Statistics on disagreements between KANO and MotiL on activity cliff samples.\nTanimoto\nsmilarity\nLabel: 1 Label: 0\n1.0 KANO: ✓ KANO: ✗\nMotiL: ✓ MotiL: ✓\nC1=CC=CC(=C1C(O)=O)OC(C)=O CC(=O)Oc1ccccc1C(O)=O\nLabel: 1 Label: 0\n0.851 KANO: ✓ KANO: ✗\nMotiL: ✓ MotiL: ✓\nCCC(=O)C1(O)CCC2C3CCC4=CC(=O)CCC4(C)C3C(O)CC21C C[C@]12CCC(=O)C=C1CC[C@H]3[C@@H]4CC[C@]\n(O)(C(=O)CO)[C@@]4(C)C[C@H](O)[C@H]23\nLabel: 1 Label: 0\n1.0 KANO: ✓ KANO: ✗\nMotiL: ✓ MotiL: ✓\n[C@H](CC1=CC(=C(C=C1)O)O)(C(O)=O)N N[C@@H](Cc1ccc(O)c(O)c1)C(O)=O\nLabel: 1 Label: 0\n1.0 KANO: ✓ KANO: ✗\nMotiL: ✓ MotiL: ✗\nCN1C2CCC1CC(C2)OC(=O)[C@H](CO)c3ccccc3 CN1C2CCC1CC(C2)OC(=O)C(CO)c3ccccc3\nLabel: 1 Label: 0\n0.972 KANO: ✓ KANO: ✗\nMotiL: ✗ MotiL: ✓\n[Cl].CC(C)NCC(O)COc1cccc2ccccc12 CC(C)NCC(O)COc1cccc2ccccc12\nFigure 2: Prediction results for activity cliff samples on BBBP.\nTanimoto\nsmilarity\nLabel: 1 Label: 0\n0.956 KANO: ✓ KANO: ✗\nMotiL: ✓ MotiL: ✓\nCC(C)C[C@@H](C(=O)N[C@@H](CCCN=C(N)N)C(= CC(C)CC(C(=O)NC(CCCN=C(N)N)C(=O)N1CCCC1C(=\nO)N1CCC[C@H]1C(=O)NCC(=O)N)NC(=O)[C@@H]( O)NCC(=O)N)NC(=O)CNC(=O)C(CC2=CC=C(C=C2)O)\nCC2=CNC3=CC=CC=C32)NC(=O)[C@H](CC4=CC=C(C NC(=O)C(CO)NC(=O)C(CC3=CNC4=CC=CC=C43)NC(=\n=C4)O)NC(=O)[C@H](CO)NC(=O)[C@H](CC5=CNC6 O)C(CC5=CN=CN5)NC(=O)C6CCC(=O)N6\n=CC=CC=C65)NC(=O)[C@H](CC7=CN=CN7)NC(=O)[\nC@@H]8CCC(=O)N8\nLabel: 1 Label: 0\n1.0 KANO: ✓ KANO: ✗\nC(CNCCNCCNCCN)N MotiL: ✗ C(CNCCNCCN)N MotiL: ✓\nLabel: 1 Label: 0\n0.860 KANO: ✓ KANO: ✗\nMotiL: ✗ MotiL: ✓\nCCCN1CCCCC1C(=O)NC2=C(C=CC=C2C)C CCCCN1CCCCC1C(=O)NC2=C(C=CC=C2C)C\nLabel: 1 Label: 0\n0.883 KANO: ✓ KANO: ✗\nMotiL: ✗ MotiL: ✓\nCC(=O)N(CC(CO)O)C1=C(C(=C(C(=C1I)C(=O)NCC(CO CC(=O)N(CC(CO)O)C1=C(C(=C(C(=C1I)C(=O)NCC(CO\n)O)I)C(=O)NCCO)I )O)I)C(=O)NCC(CO)O)I\nFigure 3: Prediction results for activity cliff samples on SIDER.\n[1] Rong, Y. et al. Self-supervised graph transformer on large-scale molecular data. Advances in Neural Information\nProcessing Systems 33, 12559–12571 (2020).\n[2] Wang, Y., Wang, J., Cao, Z. & Farimani, A. B. Molecular contrastive learning of representations via graph neural\nnetworks. Nature Machine Intelligence 4, 279–287 (2022).\n[3] Fang, Y. et al. Knowledge graph-enhanced molecular contrastive learning with functional prompt. Nature\nMachine Intelligence 5, 542–553 (2023).\nComment 4: Contrastive learning is described twice in the introduction. I would recommend tightening the prose\nhere a little.\nReply:\nWe thank the reviewer for pointing out the redundancy in our introduction regarding the description of contrastive\nlearning. We agree that the prose can be made more concise and have accordingly revised the relevant section to\nimprove clarity and reduce repetition.\nOriginally, contrastive learning was described both when introducing the main categories of molecular\nrepresentation learning methods and again when discussing its impact on motif representation. To improve clarity\nand reduce redundancy, we have refined the exposition. The third paragraph in the Section of Introduction now\nconcisely introduces the contrastive learning mechanism, while the subsequent discussion focuses exclusively on its\nlimitations, specifically its challenges in capturing the representations of chemically meaningful motifs, without\nreiterating the methodology. This restructuring helps streamline the introduction and emphasizes the motivation\nbehind our proposed approach.\nThe revised portion in the Section of Introduction of the main text now reads:\n“Existing deep learning-based molecular representation learning methods can be divided into two categories:\ncontrastive learning and predictive learning. Contrastive learning methods [11, 12] typically produce two\naugmented versions of a molecular graph through techniques like subgraph removal or perturbation, and then use\nthem to form positive and negative sample pairs. While these methods can distinguish chemically different molecules,\nthey may inadvertently disrupt chemically meaningful substructures (e.g., functional groups or residues), thereby\nweakening the preservation of critical motif information. Predictive learning methods [27–30], in contrast, aim to\nlearn molecular representations by predicting masked subgraphs [27, 28], contexts [29], or attributes [30].\nBeyond these graph-based methods, recent methods based on SMILES or sequences have also shown promise\nby leveraging larger unlabeled corpora (e.g., ChemBERTa [31], SMILES-BERT [32], ESM [33]). While these\nmethods benefit from scalability, they may sacrifice finer chemical granularity, such as\nstereochemistry or bonding topology, which graph-based methods aim to preserve.\nIt is crucial for molecular representation learning to produce representations that accurately reflect the\nunderlying chemical structures and functions. However, existing methods struggle to effectively capture the\nrepresentations of motifs, such as functional groups in small molecules or amino acid residues in proteins, due to\ntheir limited ability to model localized chemical semantics or hierarchical substructures. For instance, prior\ncontrastive learning methods [11, 12, 27] often rely on random subgraph sampling or global-level supervision,\nwhich may overlook pharmacologically or structurally meaningful motifs. Similarly, predictive molecular\nrepresentation learning, which involves masking subgraphs, contexts, or attributes, also compromises the integrity\nof motif information in the original molecule, further hindering the preservation of critical chemical properties.”\nComment 5: The introduction implicitly assumes structure-based GNN encoders are the only line of research here\nby not considering alternate representation such as sequence or SMILES that do not require GNNs. These methods\nalso often enable access to much larger quantities of data for training and pre-training which is a trade-off that should\nbe mentioned.\nReply:\nWe thank the reviewer for this insightful comment regarding alternative molecular representations such as SMILES\nstrings and protein sequences. We agree that these representations constitute an important line of research and offer\npractical advantages, especially in terms of data availability and scalability. In response, we would like to make the\nfollowing clarifications:\n1. On the choice of molecular graphs vs. sequences or SMILES: We acknowledge that SMILES- and sequence-\nbased approaches (e.g., ChemBERTa [1], SMILES-BERT [2], ESM [3]) enable access to massive corpora of\nmolecular or protein data and have shown promising performance in various domains. However, our focus on\ngraph-based representations is motivated by their closer alignment with the underlying physical and chemical\nstructure of molecules. Molecular graphs preserve explicit atom- and bond-level information, spatial topology,\nand connectivity constraints, which are properties often distorted or omitted in linear representations. In this\nsense, graph representations can be viewed as a more complete and chemically faithful form of sequence or\nSMILES data, which makes them particularly suitable for tasks that depend on local substructure integrity, such\nas motif learning or binding affinity prediction.\n2. Trade-offs between data accessibility and structural fidelity: We agree with the reviewer that there exists a\nfundamental trade-off between data scale and structural richness. SMILES and sequence formats are often\neasier to collect in large quantities from public databases, and thus allow for training models at scale. Graph-\nbased models, while more expressive in encoding chemistry, often suffer from more limited data resources due\nto the need for preprocessed structural annotations. We have now explicitly acknowledged this trade-off in the\nrevised introduction to present a more balanced view of current representation learning paradigms. While\nsequence- and SMILES-based models benefit from greater data availability and computational efficiency,\ngraph-based approaches retain richer chemical semantics by preserving the topological and bonding structures\nof molecules. In this work, we focus on graph representations as they offer a more direct and faithful encoding\nof molecular motifs, which are often lost in linearized formats.\n3. Addition of discussion in both the main text and supplementary material: To better address the reviewer’s\npoint and improve the accessibility of our paper for broader audiences, we have made two specific additions:\nIn the section of Introduction, we now briefly mention representative works based on SMILES or sequences\n(e.g., ChemBERTa [1], SMILES-BERT [2], ESM [3]), highlighting their scalability and complementary role\nin molecular representation learning:\n“Beyond these graph-based methods, recent methods based on SMILES or sequences have also shown\npromise by leveraging larger unlabeled corpora (e.g., ChemBERTa [31], SMILES-BERT [32], ESM [33]).\nWhile these methods benefit from scalability, they may sacrifice finer chemical granularity, such as\nstereochemistry or bonding topology, which graph-based methods aim to preserve.”\nFurthermore, we have added a dedicated subsection in the appendix (i.e., Supplementary Section A.4,\nSequence-based and SMILES-based molecular representation learning) that summarizes prominent SMILES-\nand sequence-based approaches. This section discusses their relative advantages, limitations, and how they\ncompare to graph-based models in terms of expressiveness, data availability, and applicability to motif-level\nlearning:\n“In addition to graph-based representations, molecular property prediction has also benefited from linear\nrepresentations such as SMILES strings and amino acid sequences. These formats treat molecules and proteins\nas sequences, enabling the application of natural language processing (NLP) techniques to molecular modeling.\nFor instance, models like ChemBERTa [46] and SMILES-BERT [47] leverage transformer-based architectures\nto learn contextual embeddings from large-scale unlabeled SMILES corpora. Similarly, the models such as\nESM [48] utilize masked language modeling on millions of amino acid sequences to capture biological\nsemantics. A key advantage of sequence-based or SMILES -based approaches is their scalability: the simplicity\nof SMILES and FASTA formats allows access to massive datasets and efficient pretraining pipelines. However,\nthis scalability comes with trade-offs. Unlike molecular graphs, which naturally encode topological and\nstereochemical information, SMILES strings or amino acid sequences often linearize or obscure such structural\nnuances. Therefore, while sequence models offer practical benefits in terms of data volume and pretraining\nefficiency, they may struggle to preserve fine-grained spatial or functional substructures critical to downstream\nmolecular tasks. Our work focuses on graph-based models as they offer a more faithful structural\nrepresentation, particularly advantageous for capturing and aligning motifs in molecular graphs.”\nTogether, these additions clarify the landscape of molecular representation learning and provide context\nfor our graph-based choice, while acknowledging the strengths of alternative input modalities.\n[1] Chithrananda, S., Grand, G. & Ramsundar, B. ChemBERTa: Large-Scale Self-Supervised Pretraining for\nMolecular Property Prediction. NeurIPS Workshop on Machine Learning for Molecules (2020).\n[2] Wang, S., Guo, Y., Wang, Y., Sun, H. & Huang, J. SMILES-BERT: Large Scale Unsupervised Pre-Training for\nMolecular Property Prediction. Proceedings of the 10th ACM International Conference on Bioinformatics,\nComputational Biology and Health Informatics 429–436 (2019).\n[3] Rives, A. et al. Biological structure and function emerge from scaling unsupervised learning to 250 million\nprotein sequences. Proceedings of the National Academy of Sciences 118, e2016239118 (2021).\nComment 6: L136 \"Gaussian noise into 𝒢 in a controlled manner.\" This is a shallow description. The authors\nshould describe that the adjacency matrix is noised.\nReply:\nWe thank the reviewer for this valuable suggestion. Based on your suggestions, we have revised the text in the\nSection of Methods to explicitly state that the noise is added to the adjacency matrix of the molecular graph. The\nrevised sentence now reads:\n“In the noise addition process, Gaussian noise is introduced directly into the adjacency matrix of the molecular\ngraph 𝒢, producing a perturbed adjacency matrix 𝑨 at each time step t.”\n!\nMeanwhile, we have revised the content in the Section of Methods as follows:\n“The generative diffusion model DiffMoM progressively perturbs the adjacency matrix 𝑨 of the molecular\n!\ngraph 𝒢 by adding Gaussian noise, where the level of noise is controlled through a predefined diffusion schedule.”\nThis clarification more accurately reflects the operation performed in the diffusion process and improves the\ntechnical precision of our description.\nComment 7: L148. Grammar. \"the local feature\" -> the local features.\nReply:\nWe thank the reviewer for their attentive reading and constructive feedback regarding the grammatical issue in Line\n148. We have revised the phrase “we integrate the local feature in a molecule...” to “we integrate the local features\nin a molecule...” to correct the singular/plural mismatch. In addition, we carefully checked the manuscript for other\ninaccurate expressions and made further refinements to improve clarity. For example, since Figure 2 presents\nvisualization results of multiple methods, not only MotiL, we have revised the caption of Figure 2a from “T-SNE\nvisualization of the molecular representations learned by MotiL.” to “T-SNE visualization of the molecular\nrepresentations learned by MotiL and other methods.”.\nComment 8: L149. Typo \"encodes\" -> encoders\nReply:\nWe appreciate the reviewer’s careful attention to detail. We have corrected “two GNN encodes” to “two GNN\nencoders” in the revised manuscript. In addition, we have thoroughly reviewed the entire manuscript to identify and\ncorrect any typographical errors, ensuring consistency and clarity throughout the paper. In addition, we carefully re-\nexamined the main text and the supplementary material to identify and correct other typographical and formatting\nissues, ensuring consistency and clarity throughout the paper. For example, in the supplementary material (Line 119)\nof the originally submitted manuscript, we corrected a missing space in the sentence: “Lipophilicity[59] is extracted”\n-> “Lipophilicity [59] is extracted.”\nComment 9: L245-250 this is quite repetitive of what has been previously stated.\nReply:\nWe thank the reviewer for pointing out the redundancy in Lines 245–250. Upon review, we agree that this portion\nof the manuscript reiterates points already discussed in the Introduction and Methodology sections, particularly the\nlimitations of augmentation-based contrastive learning and the motivation behind our motif-preserving strategy.\nTo address this concern, we have revised the paragraph to reduce repetition while preserving its function as a\ntransition to the performance evaluation. The updated version now concisely summarizes the contrast between\nbaseline approaches and our method, emphasizing key insights without duplicating previous content. The revised\ntext reads:\n“The augmentation-based methods like MolCLR, CMPNN, and KANO outperform supervised and predictive\nself-supervised baselines, but they often overlook the negative impact of graph augmentation on motif integrity. In\ncontrast, MotiL avoids aggressive augmentation, integrates a generative diffusion model to enhance robustness, and\nachieves consistently superior performance across benchmarks. These results highlight the effectiveness of motif-\npreserving representation learning.”\nComment 10: In Table 2 the authors are comparing to the variants of GearNet without pre-training. This seems like\nan unfair comparison. Additionally, the authors should denote the source of the results for the baseline methods if\nthey did not run them themselves.\nReply:\nWe thank the reviewer for raising this important concern regarding the fairness and clarity of the comparisons\npresented in Table 2.\n1. On comparing MotiL with non-pretrained GearNet variants: We agree with the reviewer that differences\nin pretraining can influence the validity of performance comparisons. In our case, although MotiL employs a\npretraining phase, it is important to clarify that no external protein data were introduced during this phase. As\ndescribed in the Section of Datasets in the main text, “We use 1D sequential orders and 3D geometric\ncoordinates of amino acids from the protein datasets for MotiL pretraining, and incorporate the label\ninformation of GO terms or EC numbers during the task-specific fine-tuning phase.” That is, all pretraining for\nMotiL was performed strictly using the proteins in the benchmark datasets themselves, without incorporating\nadditional protein sequences or structures from external sources. In contrast, GearNet’s pretraining variant\nexplicitly incorporates a large external protein structure corpus for pretraining. As stated in Section 5.1 of the\nGearNet paper [1]: “We use the AlphaFold protein structure database (CC-BY 4.0 License) (Varadi et al., 2021)\nfor pretraining.” This use of an extensive external protein structure database distinguishes GearNet’s\npretraining setup from ours in both scope and data resources. In Table 2 of our manuscript, we chose to report\nthe non-pretrained versions of GearNet and its variants (e.g., GearNet-Edge, GearNet-IEConv, GearNet-Edge-\nIEConv), following their original evaluation protocols without pretraining. These results were drawn directly\nfrom Ref. [24], and as the original paper does not report performance for the same variants with pretraining, we\ndid not include pretrained counterparts to avoid inconsistencies. We now explicitly clarify this setting in the\nrevised manuscript and have added a note to the Table 2 caption indicating this difference: “The pretraining\nphase of MotiL does not incorporate extra protein data, which is consistent with baselines such as GearNet.”\n2. On denoting the source of baseline results: We also appreciate the reviewer’s suggestion to make the\nprovenance of baseline results more transparent. As indicated in the Section of Methods and now explicitly\nclarified in the caption of Table 2, the results for all baseline models (including GearNet and CDConv) are\ndirectly reported from Ref. [24]. The updated caption now reads: “All baseline results in this table are directly\nreported from Ref. [24], which follows the same evaluation protocol as this study: 10-fold cross-validation\nsplits for proteins.”\nTherefore, the updated caption for Table 2 is as follows:\n“Performance comparison on protein macromolecules. All baseline results in this table are directly reported\nfrom [24], which follows the same evaluation protocol as this study: 10-fold cross-validation splits for proteins. The\npretraining phase of MotiL does not incorporate extra protein data, which is consistent with baselines such as\nGearNet.”\n[1] Zhang, Z. et al. Protein representation learning by geometric structure pretraining. The 11th International\nConference on Learning Representations, ICLR 2023 (2023).\nComment 11: L282 EC prediction is a classification task.\nReply:\nWe sincerely thank the reviewer for pointing out this important clarification regarding the nature of the EC number\nprediction task in Line 282. You are correct that EC number prediction is a multi-label classification task, not a\nregression task. Each protein can be associated with multiple enzyme commission (EC) numbers, corresponding to\nhierarchical functional annotations based on the type of chemical reactions catalyzed. Thus, the task is best modeled\nas a classification problem, where the goal is to predict one or more correct EC labels for a given protein.\nIn our original manuscript, the sentence in Line 282 was imprecise and may have inadvertently implied that\nEC prediction was a regression task. We have corrected the sentence in the revised manuscript to more accurately\nreflect the task type. The updated version now reads:\n“It suggests that MotiL has a strong advantage in classification tasks like EC number prediction.”\nWe have also reviewed the surrounding text in Section 4.4 to ensure that other task types (e.g., ATP binding\nclassification and GO prediction) are correctly described and consistently referred to using appropriate terminology\n(i.e., classification vs. regression).\nComment 12: L309. Rephrase messy clustering.\nReply:\nWe sincerely thank the reviewer for pointing out the use of the informal phrase “messy clustering” in Line 309. To\nimprove clarity and rigor, we have revised the sentence to use more appropriate technical terminology that better\nreflects the clustering behavior observed in the representation space.\nSpecifically, we have modified the original sentence from: “No pretraining yields the worst results, with messy\nclustering on QM8 (DB index: 4.93), highlighting the necessity of pretraining representation learning for effective\nmolecular representation.” to the following revised version: “No pretraining yields the worst results, with\ndisorganized and poorly separated clusters on QM8 (DB index: 4.93), highlighting the necessity of pretraining\nrepresentation learning for effective molecular representation.”\nThis updated wording provides a clearer and more objective description of the low-quality clustering results\nobserved when pretraining is absent. We also ensure that the interpretation remains directly supported by the\nquantitative DB index value reported.\nComment 13: The authors make frequent to reference to their proposed methods robustness to noise but do not\nperform any experiments to demonstrate this. Nor do they describe the source of the noise they are referring to.\nSurely, given that SE(3) invariance is built into the model, the effect of structural noise is likely to be minimal? As\nfar as I can tell from the description, coordinates are used in to model.\nReply:\nWe sincerely thank the reviewer for this insightful and constructive comment. We understand the importance of\nproviding empirical evidence to support our claim of noise robustness and clarifying what we mean by \"noise\" in\nour study.\n1. Clarification of noise definition: In the context of our work, the “noise” refers specifically to structural\nperturbations in molecular graphs, particularly edge-level noise such as missing chemical bonds, which may\narise due to experimental uncertainty in graph construction or preprocessing artifacts [1-4]. Although our model\nleverages 3D geometric features and is designed to be SE(3)-invariant, graph-based representations (especially\nthe topological structure of the molecular graph) remain central to the encoder. Perturbations to this topology\ncan still significantly impact learned representations in most models.\n2. New experiments to quantify noise robustness: To address the reviewer’s concern, we have added a new\ncontrolled experiment. In this experiment, we simulate graph-level noise by randomly deleting 10%, 20%, and\n30% of edges from each molecular graph in the BBBP (from MoleculeNet) and Solubility (from ADME)\ndatasets. We evaluate the ROC-AUC or RMSE performance of three competitive baselines (MolCLR, MGSSL,\nKANO), our model (MotiL) and an ablation version of MotiL without diffusion priming under each noise level.\nAs shown in Figure 4, MotiL exhibits significantly smaller decreases in ROC-AUC or smaller increases in\nRMSE as noise levels rise, demonstrating its superior robustness to graph structural perturbations. This\nrobustness is attributed to the diffusion priming phase, which learns denoising-aware representations by\nmodeling and reversing a noise corruption process during pretraining.\n(a) BBBP (b) Solubility\nFigure 4: Performance of different methods under increasing graph structural noise.\nTo reflect this improvement, we have revised the manuscript by:\n(1) Clarifying our definition of noise as structural perturbations in molecular graphs.\n(2) Adding the new experiment and detailed analysis in Supplementary Section G (Robustness evaluation to\ngraph structural noise);\nThe revised sentence in the main text now reads:\n“We propose diffusion priming as the first phase of MotiL to perceive the global structure of a molecule\nand thus enhance the robustness of GNNs to structural noise perturbations in the molecular graph 𝒢 (Fig.\n1a).”\n“On the other hand, MCL without augmentation removes the reliance on molecule graph augmentation,\nbut meanwhile ignores the GNN encoder's robustness to structural noise perturbations in molecule graphs\nand thus does not yield better results than MCL with augmentation.”\n“Also, Supplementary Fig. 8 indicates that the diffusion priming phase used in MotiL effectively mitigates\nthe sensitivity of our method to structural noise perturbations.”\nThe additional content in Supplementary Section G (Robustness evaluation to graph structural noise) is\npresented as follows:\n“To evaluate the robustness of molecular representation models under structural noise, we introduce\nvarying levels of edge deletion to the molecular graphs in the BBBP and Solubility datasets. Fig. 8 shows the\nROC-AUC or RMSE results for MotiL, MotiL w/o diffusion priming, and three representative baselines\n(MolCLR, MGSSL, and KANO). MotiL exhibits significantly better resistance to noise perturbations, with a\nflatter error curve and superior RMSE across all noise levels. It demonstrates the effectiveness of the diffusion\npriming phase in enhancing the stability of learned representations under noisy graph conditions.”\n[1] Heid E, McGill C J, Vermeire F H, et al. Characterizing uncertainty in machine learning for chemistry[J]. Journal\nof Chemical Information and Modeling, 2023, 63(13): 4012-4029.\n[2] Ryu S, Kwon Y, Kim W Y. A Bayesian graph convolutional network for reliable prediction of molecular\nproperties with uncertainty quantification[J]. Chemical science, 2019, 10(36): 8438-8446.\n[3] Crusius D, Cipcigan F, Biggin P C. Are we fitting data or noise? analysing the predictive power of commonly\nused datasets in drug-, materials-, and molecular-discovery[J]. Faraday Discussions, 2025, 256: 304-321.\n[4] Jiang S, Qin S, Van Lehn R C, et al. Uncertainty quantification for molecular property predictions with graph\nneural architecture search[J]. Digital Discovery, 2024, 3(8): 1534-1553.\nComment 14: What is the justification for using t-SNE for the small molecule embeddings and UMAP for the\nprotein embeddings?\nReply:\nWe sincerely thank the reviewer for this insightful question regarding our choice of visualization methods for the\nlearned embeddings. Our decision to use t-SNE for small molecules and UMAP for proteins was guided by both the\nnature of the underlying data and empirical observations of visualization quality.\n1. For small molecules, we employed t-SNE because our goal was to inspect whether molecules with similar\nchemical scaffolds (e.g., naphthalene, barbituric acid) are positioned closely in the embedding space. t-SNE is\nwell-suited for highlighting local neighborhood structures, and is commonly used in molecular representation\nlearning literature for this purpose [1, 2]. Since small molecule graphs tend to be lower-dimensional and\nstructurally simpler, t-SNE is effective in capturing the fine-grained local similarities needed for scaffold-level\ninspection.\n2. For proteins, we instead used UMAP due to the higher complexity and dimensionality of protein\nrepresentations, which encode both 1D sequential and 3D structural features. UMAP has been shown to better\npreserve both local and global manifold structures, which is critical for interpreting functional similarity across\nproteins [3, 4]. In our case, UMAP yielded more cohesive and interpretable clusters when visualizing\nembeddings related to GO terms or EC numbers, which are often multi-label and hierarchically organized.\nIn summary, our choice reflects a data-driven and task-sensitive strategy: t-SNE was preferred for visualizing\nlocal motif-based clustering in small molecules, while UMAP was more suitable for global functional embedding\nstructures in proteins.\nWe have updated the manuscript to clarify this point. The corresponding revision in the main text reads (added\nin Section “MotiL learns macromolecular representations consistent with protein structures and chemical\nproperties”):\n“For visualization, we use t-SNE for small molecule embeddings due to its strength in preserving local\nstructure, which aligns with our goal of examining scaffold-level clustering. For protein embeddings, we employ\nUMAP, which better maintains both local and global relationships in high-dimensional data. This choice improves\ninterpretability when analyzing functional similarity in multi-label classification tasks such as GO and EC prediction.\nSpecifically, we use a non-linear dimension reduction technique named uniform manifold approximation and\nprojection (UMAP) [50] to project the ATP binding protein representations learned by MotiL onto a 2D coordinate\nspace (Fig. 3a).”.\n[1] Fang, Y. et al. Knowledge graph-enhanced molecular contrastive learning with functional prompt. Nature\nMachine Intelligence 5, 542–553 (2023).\n[2] Wang, Y., Wang, J., Cao, Z. & Farimani, A. B. Molecular contrastive learning of representations via graph neural\nnetworks. Nature Machine Intelligence 4, 279–287 (2022).\n[3] Wang J, Zhang S, Qiao H, et al. UMAP-DBP: an improved DNA-binding proteins prediction method based on\nuniform manifold approximation and projection[J]. The Protein Journal, 2021, 40: 562-575.\n[4] Wang Z, Combs S A, Brand R, et al. Lm-gvp: an extensible sequence and structure informed deep learning\nframework for protein property prediction[J]. Scientific reports, 2022, 12(1): 6832.\nComment 15: In figure 3a it is a little odd to start the cluster IDs from -1.\nReply:\nWe thank the reviewer for this helpful observation. In Figure 3a, we used the DBSCAN algorithm to perform\nunsupervised clustering on the t-SNE embeddings of molecular representations. According to the standard DBSCAN\nimplementation, data points that are identified as outliers or noise (i.e., those not assigned to any cluster) are\nautomatically labeled with cluster ID “-1.” We acknowledge that this notation may not be immediately intuitive to\nall readers. To address this concern and improve clarity, we have revised the manuscript as follows:\nWe updated the caption of Figure 3a to explicitly clarify the meaning of “-1” as follows:\n“UMAP visualization of the representations for ATP binding proteins (Cluster ID -1 represents noise points\nas defined by DBSCAN, i.e., points not assigned to any cluster).”\nComment 16: What do the green points along the X axis in the inset panel in Figure 3a indicate?\nReply:\nWe thank the reviewer for this valuable question. The green points shown along the X axis in the inset panel of\nFigure 3a correspond to data samples from cluster ID = 9, which is represented using green in the main t-SNE\nvisualization. These points are valid members of the cluster and are not outliers. However, we acknowledge that\ntheir placement along the horizontal axis in the inset panel may have visually suggested they were outliers or\nunclustered points, which could lead to confusion. This was an unintended artifact of the specific layout and scaling\nused in the inset. To resolve this and enhance clarity, we have revised Figure 3a in the updated manuscript by\nreplacing the inset panel with a clearer version that eliminates overlap and better preserves spatial relationships.\nThe updated figure is shown below:\na b\nid:1 id:6 id:7 id:9 id:10id:11\nid: 6\n1WLE-A\n3LSS-A\nFigure 5: UMAP visualization of the representations for ATP binding proteins (Cluster ID -1 represents noise\npoints as defined by DBSCAN, i.e., points not assigned to any cluster).\nc d\n4AU8-A\nComment 17: I actu(iadl:l y3 )disagree with the authors claim that the similarity of the representations of the amino acids\nCommon GO terms:\nreflect their similarity outside of the authors cherry-picked examples. For example, Serine and Tryptophan and the\nprotein kinase activity, …\nlow similarity of A2laBnPiMne-A and Glycine.\n(id: 3)\nReply:\nWe sincerely thank the reviewer for this insightful comment. We acknowledge the concern that our initial\npresentation of ami1nWo LaEc-iAd similarity was based on a few illustrative examples, which could be perceived as cherry-\n(id: 6)\npicking. We agree that such limited qualitative examples may not be sufficient to support broader claims regarding\nthe fidelity of the learned representations across the full set of amino acids. To address this concern, we would like\nto clarify the following points:\n1. Systematic analysis provided in submission: In the originally submitted manuscript, we included a complete\npairwise similarity analysis of all 20 standard amino acids based on their learned representations in the\npretraining stage. This analysis is visualized as a 20×20 cosine similarity heatmap in Supplementary Figure 7\n(Appendix Section G), titled “Heatmap of the amino acid’s representation similarity”. The matrix reflects the\nglobal structure of the learned embedding space and provides a holistic view of how the model organizes amino\nacids.\n2. Interpretation of learned similarities: We emphasize that the amino acid representations learned by MotiL\nare context-sensitive, i.e., they are shaped by the protein sequence and 3D structural environments encountered\nduring pretraining, rather than fixed based solely on biochemical or physicochemical properties. Therefore,\ncertain similarities or dissimilarities (e.g., Serine–Tryptophan being closer than Alanine–Glycine) arise from\ndata-driven co-occurrence and structural proximity, not necessarily canonical chemical similarity.\n3. Specific explanation of the pair of Serine and Tryptophan: We acknowledge that pairwise examples such\nas Serine–Tryptophan or Alanine–Glycine may not fully align with conventional biochemical similarity\ngroupings. As the reviewer rightly notes, Serine (a small polar residue) and Tryptophan (a large nonpolar\naromatic residue) differ substantially in terms of size, polarity, and side-chain chemistry. Our intent was not to\nsuggest that MotiL's learned representations strictly reproduce canonical biochemical classifications. Instead,\nthe goal was to qualitatively illustrate that the learned motif representations are context-sensitive, i.e., shaped\nby the local sequence and structural environments encountered during pretraining. For example, Serine and\nTryptophan may be encoded similarly if they frequently appear in analogous structural or functional contexts\n(e.g., surface-exposed residues near active sites or domain boundaries) across the training dataset. This context-\ndependent behavior is a natural consequence of the self-supervised pretraining framework used in MotiL, where\nrepresentations are influenced by co-occurrence and neighborhood semantics rather than static chemical\nproperties.\n4. Specific explanation of the pair of Alanine and Glycine: While Alanine and Glycine have small, nonpolar\nside chains and are both weakly hydrophobic, their learned embeddings in MotiL are notably dissimilar. This\ndivergence can be attributed to their distinct structural roles: Glycine, due to its minimal side chain, contributes\nto conformational flexibility and often resides in protein turns or flexible loops, whereas Alanine favors α-\nhelical regions and imparts structural rigidity. Consequently, in context-aware representation learning, such as\nin MotiL, their embeddings reflect these functional and structural contextual differences rather than their\nsuperficial chemical similarity.\n5. Revised wording and figure in the manuscript: We have revised the relevant descriptions in the manuscript\nto more carefully reflect this point. Specifically, we now clarify that the examples of amino acid similarities are\nintended to illustrate localized representation patterns learned by the model rather than to draw broad\nbiochemical conclusions. We explicitly acknowledge that certain pairs, such as Serine and Tryptophan or\nAlanine and Glycine, may deviate from canonical biochemical expectations due to the context-sensitive nature\nof the self-supervised learning process. The representations learned by MotiL are influenced by the structural\nand sequential environments encountered during pretraining, rather than solely by chemical similarity. To\nprovide a more comprehensive and objective analysis, we have revised Supplementary Section J (Detailed\nresults and analysis of protein representations) accordingly and updated Figure 7 (the amino acid similarity\nheatmap, now shown in Figure 6 below) to enhance visual clarity and interpretability. The updated text in\nSupplementary Section J “Detailed results and analysis of protein representations” reads:\n“Example 4: The similarity between serine and tryptophan is moderate, despite their apparent\nbiochemical differences. Serine is a small polar residue with a hydroxyl-containing side chain, while tryptophan\nis a large, nonpolar aromatic amino acid. Although they differ substantially in physicochemical properties,\nMotiL assigns them a moderately high similarity. It can be explained by their co-occurrence in analogous\nstructural or functional contexts, such as surface-exposed sites near active centers or domain boundaries,\nacross the training dataset. Such context-dependent embedding behavior is characteristic of MotiL’s self-\nsupervised learning process, which encodes local neighborhood semantics rather than relying solely on static\nbiochemical similarity.\nExample 5: The similarity between alanine and glycine is relatively low, despite their similar chemical\nnature. Both alanine and glycine have small, nonpolar side chains and are both weakly hydrophobic, thus they\nare often grouped together in conventional biochemical classifications. However, their learned similarity in\nMotiL is significantly lower, which reflects their distinct structural roles in proteins. glycine, being the smallest\namino acid with a hydrogen side chain, is frequently located in flexible regions like loops and turns. In contrast,\nalanine’s methyl side chain favors 𝛼-helical structures and imparts rigidity. MotiL effectively captures these\ndifferences in structural context, leading to diverging embeddings even for chemically similar amino acids.”\nStructural isomers Aromatic rings\nIsoleucine Leucine Histidine Tyrosine\nAlanine Glycine Histidine Glutamine Serine Tryptophan\nDistinct structural roles Different side chain structures Functionally co-located residues\nFigure 6: Heatmap of the amino acid’s representation similarity.\nComment 18: L458 \"Fig 4.\" is repeated.\nReply:\nWe thank the reviewer for pointing out the repeated reference to \"Fig. 4\" in Line 458. We have revised the sentence\nto eliminate the redundancy and improve fluency. The updated version in Section “MotiL corrects the erroneous\npredictions of the strongest baseline” of the main text now reads:\n“We present a comprehensive evaluation of prediction disagreements between MotiL and the strongest\nbaseline KANO on two toxicity-related benchmarks: BBBP (blood-brain barrier permeability) and SIDER (drug-\ninduced liver injury). Fig. 4a displays quantitative statistics of disagreement cases between MotiL and KANO on the\ncomplete datasets.”\n“Fig. 4b, Supplementary Fig. 10, and Supplementary Fig. 11 present the results of the activity cliff experiment,\ndesigned to evaluate model robustness on challenging molecular pairs with high structural similarity but divergent\nlabels.”\nComment 19: Figure 4 is a little hard to gain significant insight from. Whilst it is nice to see specific examples\nwhere the models disagree, cherry picking favorable cases is second-best to a more quantitative summarization of\nthe disagreements. I believe such an analysis should replace this figure, and figure 4 could be an inset panel or moved\nto the supplementary.\nReply:\nWe thank the reviewer for this constructive suggestion regarding Figure 4. The intent behind Figure 4 was to provide\nreaders with a concrete, visual comparison between MotiL and KANO by highlighting specific test samples where\nthe two models diverged in their predictions. We agree that although such qualitative examples can be insightful,\nthey are inherently limited in generalizability and may appear as cherry-picking if not supported by broader statistical\nevidence. In response to the reviewer’s valuable feedback, we have taken the following steps to improve the rigor\nand clarity of this analysis:\n1. Addition of a quantitative disagreement analysis: To complement the qualitative examples, we now include a\nnew quantitative evaluation that systematically analyzes prediction disagreements between MotiL and KANO\nacross the entire test sets for BBBP and SIDER. This analysis measures not just how often the models disagree,\nbut also which model is more frequently correct in those disagreement cases. The results are summarized in\nFigure 7 below. Specifically:\n(1) On the BBBP dataset, the two models produce conflicting predictions on 57 out of 203 test samples\n(28.1%). Among these disagreement cases, MotiL is correct in 40 instances (70.2%), whereas KANO is\ncorrect in only 17 (29.8%).\n(2) On the SIDER dataset, the models disagree on 63 of 295 samples (21.4%). MotiL correctly predicts 43 of\nthese (68.3%), while KANO succeeds on 20 (31.7%).\nThese results provide a clear and quantitative indication that MotiL’s superiority is not confined to isolated\nexamples, but holds across a sizable portion of challenging cases. It supports the conclusion that MotiL is more\nrobust when facing ambiguous or borderline molecular instances, particularly in the context of real-world\nbiomedical classification.\n2. Relocation of Figure 4 to the supplementary materials: In line with the reviewer’s suggestion, we have moved\nthe original Figure 4, which presents specific visualized disagreement cases, to the supplementary materials,\nnow labeled as Supplementary Figure 9. We agree that this figure is better suited as a supplemental illustration\nrather than a main result. In the revised main text, we now reference this figure as a complement to the newly\nadded quantitative analysis, making it clear that it serves only to support, not replace, the broader evaluation.\nWe have revised the content of Section “MotiL corrects the erroneous predictions of the strongest baseline” in\nthe main text as follows:\n“We present a comprehensive evaluation of prediction disagreements between MotiL and the strongest\nbaseline KANO on two toxicity-related benchmarks: BBBP (blood-brain barrier permeability) and SIDER (drug-\ninduced liver injury). Fig. 4a displays quantitative statistics of disagreement cases between MotiL and KANO on the\ncomplete datasets. On the BBBP dataset, the two models produce conflicting predictions on 57 out of 203 test\nsamples (28.1%). Among these, MotiL correctly predicts 40 instances (70.2%), while KANO is correct in only 17\n(29.8%). On the SIDER dataset, the models disagree on 63 of 295 samples (21.4%), with MotiL accurately predicting\n43 samples (68.3%) and KANO only 20 (31.7%). Fig. 4b and Supplementary Figs. 10&11 present the results of the\nactivity cliff experiment, designed to evaluate model robustness on challenging molecular pairs with high structural\nsimilarity but divergent labels. We identified activity cliff samples from BBBP and SIDER by selecting molecular\npairs with a Tanimoto similarity above 0.85 and differing binary labels. Among the disagreement cases on these\nactivity cliffs, MotiL achieves higher accuracy than KANO: on BBBP, MotiL correctly predicts 10 out of 15 samples\n(66.7%) compared to KANO’s 5 (33.3%); on SIDER, MotiL is correct on 11 of 19 samples (57.9%), while KANO\npredicts 8 correctly (42.1%). The results in Fig. 4a and Fig. 4b demonstrate that MotiL outperforms KANO in\nhandling ambiguous or challenging samples. Fig. 4c and Supplementary Fig. 9a illustrate molecular structures from\nselected cases in the BBBP dataset where MotiL outperforms KANO. Streptomycin, a large and highly polar\naminoglycoside antibiotic, has poor BBB permeability due to its molecular weight (581 Da) and multiple hydrophilic\ngroups. While KANO misclassifies it as permeable, MotiL correctly predicts its non-permeability. Mexazolam, a\nbenzodiazepine derivative, exhibits high lipophilicity and low molecular weight, making it BBB-permeable. MotiL\ncorrectly identifies this molecule's property, while KANO fails. Fig. 4d and Supplementary Fig. 9b show\nrepresentative cases from the SIDER dataset. Erlotinib, a tyrosine kinase inhibitor, contains aromatic rings and is\nknown to form reactive metabolites, contributing to hepatotoxicity. MotiL successfully predicts this risk.\nVincristine, with its complex alkaloid structure and nitrogenous rings, has documented hepatotoxic potential due to metabolism-\ninduced stress. Again, MotiL correctly classifies this molecule, while KANO does not. These qualitative results\ncomplement our statistical findings and provide insight into MotiL’s decision-making process, especially in\nchemically intricate or borderline cases.”\na Prediction statistics on the full datasets b Prediction statistics on activity cliff samples\nc Cases in BBBP (blood-brain barrier permeability) d Cases in SIDER (hepatotoxicity)\nStreptomycin Mexazolam Erlotinib Vincristine\nKANO: 1 ✗ KANO: 0 ✗ KANO: 0 ✗ KANO: 0 ✗\nMotiL: 0 ✓ MotiL: 1 ✓ MotiL: 1 ✓ MotiL: 1 ✓\nFigure 7: Quantitative and qualitative analysis of prediction differences between KANO [12] and MotiL on BBBP\nand SIDER. a, Statistics on disagreements between the two models on the full datasets. b, Statistics on\ndisagreements between the two models on activity cliff samples. c, Prediction cases for blood-brain barrier\npermeability on BBBP. d, Prediction cases for hepatotoxicity on SIDER.\nComment 20: The authors should specify a license for the source code.\nReply:\nWe thank the reviewer for the suggestion regarding licensing. In response, we have specified the license for our\nsource code. The MotiL codebase is now publicly available at https://github.com/Young0222/MotiL, and is released\nunder the MIT License. This permissive open-source license allows users to freely reuse, modify, and distribute the\ncode with proper attribution. We have included the license information clearly in both the LICENSE file and the\nREADME file of the repository. This change ensures transparency, compliance with open-science practices, and\nease of use for the research community. As shown in Figure 8, the repository explicitly declares the license\n(highlighted by red ellipses) on GitHub.\nFigure 8: MIT License specified in the MotiL code repository.\nComment 21: Supplementary L68: I think some evidence (or elaboration) needs to be provided for the claim that\n\"the negative effects of molecular graph augmentation have hindered the advancement of MCL\".\nReply:\nWe thank the reviewer for the thoughtful suggestion to substantiate our claim regarding the negative effects of graph\naugmentation in Molecular Contrastive Learning (MCL). In the revised version, we have provided empirical and\ntheoretical evidence from recent literature to support this point.\nSpecifically, several studies have pointed out that handcrafted or random augmentations, such as node masking,\nedge deletion, and subgraph sampling, can significantly distort graph topology and task-relevant information,\nparticularly in molecular domains where subtle structural details are critical. In detail, the work in [1] clearly states:\n“Handcrafted graph augmented strategies may destroy the graph structure resulting in failing to maintain the task-\nrelated information intact… Removing important edges can severely damage graph topology… resulting in poor\ngraph embedding quality”. The work in [2] also emphasizes: “The choice of data augmentation for a given task relies\nheavily on trial and error… Hyperparameters of data augmentations exponentially expand the search space…which\nis certainly time-consuming and computationally resource-intensive”. The work in [3] notes that: “In molecular\ngraphs, dropping a carbon atom from the phenyl ring of aspirin breaks the aromatic system and results in an alkene\nchain.” Collectively, these studies highlight that inappropriate augmentations not only fail to provide consistent\nbenefits but may even degrade performance, particularly in chemically structured data where topology and function\nare tightly coupled.\nWe have revised the supplementary sentence to include supporting evidence and have directly cited the\naforementioned references to substantiate our claim. The revised text in Supplementary Section A.2 “Graph\nrepresentation learning on small molecules” is as follows:\n“The negative effects of molecular graph augmentation have hindered the advancement of MCL. The\nlimitations of molecular graph augmentations, such as their tendency to disrupt semantic consistency, have been\nshown to hinder the progress of MCL, motivating the development of augmentation-free approaches [19, 30, 31].\nFor example, Zhao et al. [18] introduce an augmentation-free MCL using a graph transformer, but they overlook\nthe model’s robustness to the noise perturbations in the molecular graph, which also limits the model’s effectiveness.”\n[1] Li Y, Zhang H, Yuan Y. Edge Contrastive Learning: An Augmentation-Free Graph Contrastive Learning\nModel[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2025, 39(17): 18575-18583.\n[2] Li H, Cao J, Zhu J, et al. Augmentation-free graph contrastive learning of invariant-discriminative\nrepresentations[J]. IEEE Transactions on Neural Networks and Learning Systems, 2023.\n[3] Lee N, Lee J, Park C. Augmentation-free self-supervised learning on graphs[C]//Proceedings of the AAAI\nconference on artificial intelligence. 2022, 36(7): 7372-7380.\nComment 22: \"L120. To the best of our knowledge, we are the first to propose the concept of molecular motif learning.\". Could the authors\nplease elaborate on this and clarify what exactly is being claimed? I could find a few works that purport to address this to\nsome degree.\nReply:\nThank you for this important question. We appreciate the reviewer’s careful reading and effort to connect our work\nwith the related literature. Indeed, molecular motifs have been used in several prior work, but their role in these\nstudies is fundamentally different from the motif representation learning proposed in our work. Below, we\nsummarize five representative motif-related studies and highlight the key distinctions.\n1. Summary of prior work:\n(1) MGSSL [1]: Leverages chemical fragmentation (BRICS + rules) to predefine a motif vocabulary and\nformulates a motif generation task. GNNs are trained to autoregressively predict the motif tree structure\nof a molecule. Motifs are treated as targets for generative prediction, not as semantic representation\ncarriers.\n(2) DGPM [2]: Proposes a dual-level graph pretraining framework where motifs are automatically\ndiscovered via edge pooling, and used as intermediate substructures for aligning subgraph kernels.\nMotifs serve as auxiliary self-supervised targets to enhance graph-level learning.\n(3) MOTOR [3]: Focuses on drug-drug interaction prediction, using BRICS-fragmented motifs as\npredefined substructures. Motifs are utilized to model intra- and inter-drug subgraph interactions,\nemphasizing the relational context rather than learning motif semantics.\n(4) HM-GNN [4]: Constructs a heterogeneous graph of motifs and molecules, using motifs as nodes to\nenable message passing between molecules sharing similar substructures. However, the model learns\nmolecular representations on this augmented graph, not explicit motif embeddings.\n(5) MICRO-Graph [5]: Employs motif-like subgraph clustering to guide subgraph sampling for contrastive\nlearning. Motifs here serve as prototypes for generating informative subgraphs, facilitating better graph-\nlevel contrastive pretraining. Motifs are tools for data augmentation, not learned representation units.\n2. Key distinction (motif-based learning vs. motif representation learning): From the above, we observe that\nprior work fall under what we would describe as motif-based learning or learning with the help of motifs:\n(1) Motifs are used as tools to assist molecular learning, e.g., for subgraph sampling, pretraining tasks,\nmessage passing, or generative prediction.\n(2) These approaches do not explicitly learn transferable motif representations that can generalize across\ndifferent molecules.\nIn contrast, our work is the first to explicitly propose molecular motif representation learning, where:\n(1) Molecular motifs are the representation units to be learned, not merely tools or targets;\n(2) We optimize for motif-level semantic consistency and alignment across molecules, enabling motifs to\nserve as transferable, interpretable, and task-agnostic representations;\n(3) Our framework is designed to work across micro- and macro-molecules, generalizing beyond small\nmolecule property prediction.\nTo better reflect this contribution, we have revised the last paragraph of Section “Introduction” in the\nmain text as follows:\n“To the best of our knowledge, we are the first to systematically formulate molecular motif representation\nlearning as a core pretraining objective, rather than using motifs merely as auxiliary tools for molecular\nlearning. Moreover, we successfully applied MotiL to both micro- and macromolecular property prediction,\ndemonstrating its versatility across molecules of different scales.”\nWe have also added further discussion to Supplementary Section A.2 “Graph representation learning on\nsmall molecules” to compare these distinctions explicitly:\n“Existing works have explored various ways to incorporate molecular motifs into representation learning.\nSpecifically, MGSSL [28] uses BRICS-derived motifs as generation targets in an autoregressive framework,\nwhile DGPM [32] discovers motifs via edge pooling to align subgraph kernels in a pretraining task. MOTOR\n[33] employs motifs as relational descriptors for drug-drug interaction prediction, and HM-GNN [34]\nintegrates motifs as heterogeneous graph nodes to facilitate message passing between molecules. MICRO-\nGraph [35], on the other hand, clusters motif-like subgraphs to guide contrastive learning. While these\nmethods demonstrate the utility of motifs for tasks like generation, pretraining, or interaction modeling, they\nshare a key limitation: motifs serve as fixed tools or intermediate constructs rather than explicitly learned,\ntransferable representations. In contrast, MotiL repositions motifs as the central representation units,\noptimizing them for semantic consistency and alignment across diverse molecular contexts. Unlike prior work\nthat use motifs to assist molecule-level learning, MotiL directly learns generalizable, interpretable motif\nembeddings that can transfer across tasks and scales, ranging from small molecules to macromolecules.”\n[1] Zhang Z, Liu Q, Wang H, et al. Motif-based graph self-supervised learning for molecular property\nprediction[J]. Advances in Neural Information Processing Systems, 2021, 34: 15870-15882.\n[2] Yan P, Song K, Jiang Z, et al. Empowering dual-level graph self-supervised pretraining with motif\ndiscovery[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2024, 38(8): 9223-9231.\n[3] Zhang R, Wang X, Liu G, et al. Motif-Oriented Representation Learning with Topology Refinement for\nDrug-Drug Interaction Prediction[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2025, 39(1):\n1102-1110.\n[4] Yu Z, Gao H. Molecular representation learning via heterogeneous motif graph neural\nnetworks[C]//International Conference on Machine Learning. PMLR, 2022: 25581-25594.\n[5] Zhang S, Hu Z, Subramonian A, et al. Motif-driven contrastive learning of graph representations[J]. IEEE\nTransactions on Knowledge and Data Engineering, 2024, 36(8): 4063-4075.\nResponse to Reviewer #2:\nComment 1: It would be helpful to clarify why the authors chose MoleculeNet.\nReply:\nWe thank the reviewer for raising this valuable point regarding our dataset selection. We chose to evaluate our model\non the MoleculeNet benchmark suite for several important and well-justified reasons:\n1. Widespread adoption and fair comparison with strong baselines: MoleculeNet is widely recognized as a\nstandard benchmark for molecular property prediction, and has been extensively adopted in the field. Many\nrecent state-of-the-art methods, including MolCLR [1], GROVER [2], KANO [3], and MGSSL [4], have\nreported results on MoleculeNet tasks using the same data splits and evaluation protocols. We follow this\nwidely accepted setup to ensure that our results are directly comparable to prior work. This consistency enables\nus to isolate the effects of our architectural and pretraining innovations, especially our motif-level contrastive\ndesign, without introducing confounding factors from dataset differences.\n2. Rich diversity of tasks and molecular properties: MoleculeNet covers a wide variety of molecular property\nprediction tasks that test different aspects of model capability, including: (1) Physicochemical properties, such\nas solubility (ESOL) and hydration free energy (FreeSolv), (2) Biological activity, such as BACE (enzyme\ninhibition) and HIV (replication inhibition), (3) ADMET-related properties, including blood–brain barrier\npermeability (BBBP), drug side effects (SIDER), and toxicity. This breadth allows us to evaluate whether our\nproposed MotiL framework generalizes effectively across different molecular domains and task types (both\nclassification and regression). It also helps assess whether motif-level representation learning can discover\nmeaningful patterns shared across these diverse endpoints.\n3. Standardized data splits and evaluation protocols: MoleculeNet provides predefined data splits (e.g.,\nscaffold split, which is particularly challenging as it forces the model to generalize to novel chemical scaffolds)\nand standard metrics (e.g., RMSE, ROC-AUC), facilitating transparent, fair, and reproducible evaluation. This\nstandardization ensures that performance comparisons are not influenced by arbitrary splitting strategies or\ninconsistent evaluation criteria.\n4. Relevance to pretraining-based methods and contrastive learning: Our work builds upon the line of\nresearch in self-supervised pretraining and contrastive learning for molecular graphs. Most relevant prior work\nin this area, including MolCLR [1], GROVER [2], and KANO [3], adopt MoleculeNet as their primary\nevaluation benchmark. Using the same setup not only strengthens the methodological continuity but also\nensures that any performance gains of MotiL can be credibly attributed to our design choices (e.g., motif-level\nalignment and diffusion priming), rather than dataset biases or incompatible experimental setups.\nTo further clarify our rationale and ensure transparency for all readers, we have added a detailed explanation\nof our dataset selection in Supplementary Section B (Overview of experimental datasets) as follows:\n“We selected the MoleculeNet benchmark suite for several key reasons, which we summarize here for\ncompleteness. First, MoleculeNet has become a widely recognized standard for evaluating molecular property\nprediction models, particularly in the context of graph neural networks and self-supervised learning. Many state-of-\nthe-art methods, including MolCLR [22], GROVER [65], KANO [23], and MGSSL [28], have been extensively\nevaluated on these datasets using consistent splits and protocols, establishing a clear basis for comparative studies.\nSecond, the diversity of molecular tasks in MoleculeNet enables a comprehensive assessment of model generalization\nacross different property types, including physicochemical, biological, and pharmacokinetic domains. This diversity\naligns with our goal of validating whether motif-level contrastive learning can uncover transferable substructures\nrelevant to distinct molecular functions. Third, MoleculeNet provides standardized scaffold splits and evaluation\nmetrics, facilitating fair comparisons and reproducible experiments. The scaffold split, in particular, is well-suited\nfor testing the ability of models to generalize beyond known chemical scaffolds, which is an essential consideration\nfor real-world molecular discovery tasks. Finally, as our work builds on the self-supervised pretraining paradigm,\nit is critical to benchmark against datasets and splits adopted in prior contrastive learning frameworks. It ensures\nthat observed improvements can be attributed to our architectural contributions (such as motif-level alignment and\ndiffusion-based pretraining) rather than differences in dataset selection or experimental design.”\n[1] Wang, Y., Wang, J., Cao, Z. & Farimani, A. B. Molecular contrastive learning of representations via graph neural\nnetworks. Nature Machine Intelligence 4, 279–287 (2022).\n[2] Rong, Y. et al. Self-supervised graph transformer on large-scale molecular data. Advances in Neural Information\nProcessing Systems 33, 12559–12571 (2020).\n[3] Fang, Y. et al. Knowledge graph-enhanced molecular contrastive learning with functional prompt. Nature\nMachine Intelligence 5, 542–553 (2023).\n[4] Zhang, Z. et al. Motif-based graph self-supervised learning for molecular property prediction. Advances in Neural\nInformation Processing Systems 34, 15870–15882 (2021).\nComment 2: It was unclear from the main text that the metrics obtained for other methods in Tables 1 and 2 were\nnot calculated by the authors but rather sourced from previous studies ([12], [27], [24]). Since the metrics for other\nmodels were taken from past articles, how can the authors guarantee that the comparison is fair and not influenced\nby differences in training and evaluation methods? Is the cross-validation method identical? Additionally, this\napproach limits further comparison. Also, it might be harder to answer why RMSE was used to evaluate physical\nchemistry properties and MAE for quantum properties and why more metrics for regression and classification\nweren’t explored?\nReply:\nWe thank the reviewer for pointing out the need for clearer disclosure regarding the source of baseline results in\nTables 1 and 2, and for raising valid concerns about the fairness and consistency of comparisons as well as the choice\nof evaluation metrics.\n1. On the source and fairness of baseline results: We clarify that all baseline results presented in Tables 1 and\n2 are directly sourced from existing literature, as follows:\n(1) For small-molecule datasets (Table 1), all baseline results, including those for MolCLR, MGSSL,\nGROVER, and KANO, are taken from the KANO paper [1], which uses a standardized and rigorous\nevaluation protocol. We have removed the previously included method (denoted as “Zhao et al.” in the\noriginal manuscript) [2], as its evaluation setup differs significantly from other baselines: the method\nused linear probing, which introduces supervised information via a frozen encoder and a separate linear\nclassifier. Moreover, the code was not publicly available, making consistent reproduction and evaluation\ndifficult. We acknowledge this inconsistency and have excluded the method from the revised manuscript\nto ensure fair comparison.\n(2) The remaining baselines all follow the same evaluation setup defined in the KANO paper [1]: “three\nindependent runs using three random-seeded scaffold splits for all datasets, except QM9, with a\ntraining/validation/testing ratio of 8:1:1.” Our own experiments were conducted following this exact\nsetup for all MoleculeNet datasets, thereby ensuring comparability across methods.\n(3) For macromolecule datasets (Table 2), we also ensured identical evaluation conditions for all methods.\nSpecifically, for GO-BP, GO-MF, GO-CC, and EC prediction tasks, we used the same 10-fold cross-\nvalidation protocol and dataset splits defined in the CDConv paper [3]). The split ratios are:\na) GO-BP, GO-MF, GO-CC: 29,898 / 3,322 / 3,415 (train / val / test)\nb) EC: 538 / 1,729 / 1,919 (train / val / test)\nWe have updated the manuscript to make this provenance and consistency clearer. The updated sentence\nin the caption of Table 1 now reads:\n“All baseline results are directly taken from Ref. [12], which follows the same evaluation protocol as\nthis study: three random-seeded scaffold splits for small molecules”.\nThe updated sentence in the caption of Table 2 now reads:\n“All baseline results in this table are directly reported from Ref. [24], which follows the same evaluation\nprotocol as this study: 10-fold cross-validation splits for proteins.”\n2. On the choice of evaluation metrics: We appreciate the reviewer’s inquiry about the rationale for selecting\ndifferent metrics (RMSE vs. MAE) across datasets. This choice aligns with standard practices adopted in\nthe respective prior work that introduced the datasets:\n(1) For physical chemistry datasets (ESOL, FreeSolv, and Lipophilicity), we use RMSE (abbreviation of\nRoot Mean Square Error) following MoleculeNet and prior work such as MolCLR, MGSSL, and\nGROVER.\n(2) For quantum datasets (QM7, QM8, and QM9), we use MAE (abbreviation of Mean Absolute Error),\nagain following the convention from MoleculeNet and its successors, due to its interpretability and\nwidespread use in quantum property prediction.\n3. On the other evaluation method: We agree that more evaluation metrics could enrich the analysis. However,\nto ensure fair and reproducible comparison with existing benchmarks, we followed the most commonly\nreported metrics in the literature. In addition, we have further improved visualization-based evaluations for\nboth small molecules and macromolecules, which provide intuitive insights into the learned representations\nbeyond scalar metrics. Specifically, we chose t-SNE for small molecules and UMAP for proteins based on the\ndata characteristics and visualization needs. For small molecules, t-SNE works well to show whether\ncompounds with similar scaffolds (e.g., naphthalene, barbituric acid) cluster together, as it highlights local\nstructure. Small molecules are simpler and lower-dimensional, making t-SNE a good fit for revealing scaffold-\nlevel patterns. For proteins, which have more complex and high-dimensional representations, UMAP better\npreserves both local and global relationships. It helped reveal clearer functional groupings, such as GO terms\nand EC numbers, which are often hierarchical and multi-label. For example, in the UMAP visualization for\nproteins, the representations obtained by MotiL for ATP binding proteins are discriminative and the prediction\nresults are relatively accurate.\nNo pretraining MCL with augmentation MCL without augmentation MotiL\na b id:1 id:6 id:7 id:9 id:10id:11\nid: 6\n1WLE-A\n3LSS-A\nFigure 1: Visualization of representations using t-SNE for small molecules (upper panel) and UMAP for proteins\nc d\n(bottom panel).\n4AU8-A\n(id: 3)\nCommon GO terms:\n4. Manuscript revisions: To address these concerns apnrodte iinm kpinraosev aec tcivliatyr , i…ty , we have made the following changes\n2BPM-A\nin the revised manuscrip(itd:: 3)\n(1) Clarified in Table 1 caption that all baseline results are taken from the KANO paper [1] using the same\n1WLE-A\nexperimental pro(tiod:c 6o)l; Removed the Zhao et al. baseline [2] due to differences in evaluation\nCommon GO terms:\nmethodology and lack of reproducibility. The updtaRNteAd b inTdainbgl, e… 1 and its caption are shown in Table 1.\n3LSS-A\n(2) Clarified in Table (2id :c 6a)ption that all baseline results are taken from the CDConv paper [3] using the same\nexperimental protocol. The updated Table 2 and its caption are shown in Table 2.\n[1] Fang, Y. et al. Knowledge graph-enhanced molecular contrastive learning with functional prompt. Nature\nMachine Intelligence 5, 542–553 (2023).\n[2] Zhao, H., Yang, X., Wei, K., Deng, C. & Tao, D. Unsupervised graph transformer with augmentation-free\ncontrastive learning. IEEE Transactions on Knowledge and Data Engineering (2024).\n[3] Fan, H., Wang, Z., Yang, Y. & Kankanhalli, M. S. Continuous-discrete convolution for geometry-sequence\nmodeling in proteins. The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali,\nRwanda, May 1-5, 2023 (2023).\nPBBB\nECAB\nLOSE\n8MQ\nTable 1: Performance comparison on small molecules. We use the metric of ROC-AUC (%) for classification tasks\nand RMSE or MAE for regression tasks. A lower value of RMSE or MAE indicates better performance. The best\nresult for each dataset is highlighted in bold, and the runner-up result is italicized. All baseline results are directly\ntaken from Ref. [12], which follows the same evaluation protocol as this study: three random-seeded scaffold splits\nfor small molecules.\nTable 2: Performance comparison on protein macromolecules. All baseline results in this table are directly reported\nfrom Ref. [24], which follows the same evaluation protocol as this study: 10-fold cross-validation splits for\nproteins. The pretraining phase of MotiL does not incorporate extra protein data, which is consistent with baselines\nsuch as GearNet.\nComment 3: In the SI, it is mentioned that the standard deviations (std) were calculated from 3 independent runs.\nCould the authors clarify what this means? Given the std values, is it possible to perform statistical analyses?\nSince the paper claims that MotiL surpasses other models, is this improvement statistically significant?\nReply:\nWe thank the reviewer for this important and insightful comment. We address each point below and have revised the\nmanuscript accordingly for clarity and completeness.\n1. Clarification on “3 independent runs”: When we report mean and standard deviation (std) values for\nMotiL’s performance across different datasets and tasks, these were obtained by:\n(1) Training the MotiL model three times independently, using the same dataset split and model\nconfiguration.\n(2) Each run was initialized with a different random seed to account for variability from random weight\ninitialization, data loading, and optimizer behavior.\n(3) The mean and standard deviation were then computed over these three runs.\nWe have clarified this procedure in the revised Supplementary Section B (Overview of experimental\ndatasets):\n“For the above benchmark experiments, we trained MotiL using three independent runs for each dataset,\nwith different random seeds for initialization. The same dataset splits and model configurations were used\nacross runs to ensure comparability. We report the average performance and the unbiased standard deviation\nacross the three runs to account for variability arising from stochastic optimization and data sampling.”\n2. On statistical significance and interpretability of standard deviations: On one hand, we observed that the\nstandard deviations across runs are consistently low (typically less than 1.5% in absolute terms), suggesting\nthat the performance of MotiL is stable and reproducible under different random initializations. On the other\nhand, to further address this concern and validate the reliability of our reported improvements, we have\nconducted additional statistical significance tests. Specifically, we performed paired two-tailed t-tests\ncomparing MotiL with the strongest baseline KANO across all 14 MoleculeNet benchmark datasets. The\nresulting p-values are summarized in Table 3 and Table 4 (also included in Supplementary Table 3 and Table\n4).\nThese p-values demonstrate that the improvements of MotiL over KANO are statistically significant (p\n< 0.05) on all tested datasets, confirming that the performance gains are consistent and unlikely to be due to\nrandom variation. The corresponding statistical analysis has been included in the revised manuscript under\nSupplementary Section D (Complete performance comparison on small molecules), as presented below:\n“To assess whether the observed performance gains of MotiL over the strongest baseline (KANO) are\nstatistically significant, we performed paired two-tailed t-tests across all MoleculeNet benchmark datasets\nusing the results from three independent runs. The corresponding p-values are summarized in Table 3 and\nTable 4. All values are below the 0.05 threshold, confirming that MotiL’s improvements are statistically\nsignificant and robust across different datasets.”\nTable 3: ROC-AUC (%) performance comparison on small molecules. The testing results are presented as the\nmean and standard deviation, calculated from three independent runs. The best result for each dataset is\nhighlighted in bold, and the runner-up result is italicized. All baseline results are directly taken from Ref. [23],\nwhich follows the same evaluation protocol as this study: three random-seeded scaffold splits for small molecules.\nTable 4: Performance comparison on small molecules. We use the metric of RMSE or MAE where a lower value\nindicates better performance. The testing results are presented as the mean and standard deviation, calculated from\nthree independent runs. The best result for each dataset is highlighted in bold, and the runner-up result is italicized.\nAll baseline results are directly taken from Ref. [23], which follows the same evaluation protocol as this study:\nthree random-seeded scaffold splits for small molecules.\nComment 4: The study in SI section F, which evaluates the different components of MotiL, is interesting and\nimportant. I agree that the results suggest that the full MotiL architecture performs better, especially in regression\ntasks. It would be beneficial, if possible, to add a sentence to the main text summarizing the key conclusions from\nthese ablation studies.\nReply:\nWe thank the reviewer for highlighting the value of the ablation study presented in Supplementary Section F. We\nagree that a concise summary of its findings in the main text would improve the clarity and completeness of the\nmanuscript. In response to this suggestion, we have added the following sentence to the main paper to summarize\nthe key conclusions:\n“Our ablation study (Supplementary Fig. 2) shows that each component of MotiL (DiffMoM, bi-scaled training,\nand network pruning) contributes to performance gains, with DiffMoM playing the most critical role in enhancing\nrobustness and generalization across diverse molecular tasks.”\nThis addition succinctly reflects the main insights derived from the ablation results, particularly the pivotal\nrole of DiffMoM in improving performance, especially in regression tasks such as QM7, QM8, and QM9. We\nappreciate the reviewer’s thoughtful recommendation, which has helped us improve the presentation of our findings.\nComment 5: I find Figure 2a very insightful, as it clearly shows that MotiL can distinguish different molecular\nscaffolds. However, I suggest using more contrastive colors, particularly in the BBBP plots, where some shades (e.g.,\npinks) are too similar. Additionally, what was the rationale behind choosing the specific molecules, scaffolds, and\ndatabase for Figure 2a?\nReply:\nWe sincerely thank the reviewer for appreciating the value of Figure 2a and for the constructive suggestions\nregarding its color scheme and design rationale.\n1. Color enhancement: We agree that some of the original color shades in the BBBP plot, particularly the pink\ncolor, were visually similar and may have caused confusion. In the revised version of the manuscript, we have\nupdated Figure 2a by adopting a set of more contrastive and distinguishable colors across different scaffolds,\nwhich improves the interpretability. The color palette was carefully selected using colorblind-friendly principles\nto ensure clarity for a broad range of viewers. The updated figure has been incorporated into the revised\nmanuscript as Fig. 2a and is also presented as Figure 2 below for reference.\n2. Selection rationale: We selected the specific molecules, scaffolds, and datasets in this figure to highlight the\nclustering capability of MotiL and to ensure representative diversity across both chemical and task domains.\n(1) Dataset selection: We included four benchmark datasets-ESOL, QM8, BBBP, and BACE-each\nrepresenting a distinct task type (e.g., regression vs. classification). It provides a comprehensive view\nof how well MotiL generalizes in learning representations across various molecular properties.\n(2) Scaffold selection: For each dataset, we selected molecules that belong to chemically distinct scaffolds.\nIt was determined using the Bemis-Murcko scaffold definition to ensure that the scaffolds differ in core\nring systems or frameworks. Our goal was to visualize whether MotiL can cluster these distinct\nscaffolds cleanly in the embedding space.\n(3) Molecule sampling: Within each scaffold group, we randomly sampled a small number of molecules\nthat span different activity or property values (e.g., log solubility in ESOL or BBB permeability in\nBBBP). It helps evaluate whether structurally related molecules are represented in a locally coherent\nway.\nWe have added a short explanatory note in the main text (Section “MotiL learns micromolecular representations\nconsistent with molecular structures”) to clarify the rationale behind the molecule and scaffold selection process.\nThe corresponding revision in the manuscript reads:\n“For example, we select four chemically diverse scaffolds from ESOL where the color palette has been\noptimized to enhance visual contrast and interpretability. The selected scaffolds, such as Naphthalene, are frequent\nand structurally distinct within ESOL, making them suitable for evaluating the model’s ability to differentiate\nsolubility-relevant chemotypes.”\nNo pretraining MCL with augmentation MCL without augmentation MotiL\nFigure 2: T-SNE visualization of the molecular representations learned by MotiL. Different colors represent\ndifferent molecular scaffolds, and a lower DB index reflects better separation and clearer distinction between the\nclusters.\nPBBB\nECAB\nLOSE\n8MQ\nComment 6: In Figure 2b, and especially in Figure 3 in the SI, some dashed lines are difficult to see.\nReply:\nWe thank the reviewer for pointing out the visibility issue regarding the dashed lines in Figure 2b and Figure 3 of\nthe Supplementary Information. To address this concern and enhance clarity, we have updated both figures by\nreplacing the original dashed lines with darker and thicker styles. These visual enhancements improve the contrast\nand ensure that key structural relationships and augmentation paths are clearly distinguishable to readers. The revised\nfigures are now included in the updated manuscript (Figure 2b) and Supplementary Information (Figure 3),\nrespectively. The revised figures are presented below as Figure 3 and Figure 4.\nSimilarity:\n0.947\nSimilarity: 0.951\nCID: 6196\nSimilarity: 0.747\nFigure 3: Comparison of positions in 2D space for structurally similar and dissimilar molecular pairs on BBBP.\naugment\nSimilarity:\n0.947\nSimilarity: 0.951\nCID: 6196\nSimilarity: 0.974\nSimilarity: 0.747\nFigure 4: Visualization of micromolecular representations learned by MotiL via t-SNE on multiple datasets.\nComment 7: In not sure if I fully understand the data behind Figure 3a. UMAP and DBSCAN were used to project\nthe representations learned by MotiL for ATP-binding proteins. Why was this specific class chosen? Also, does the\nshape in the figure indicate whether a prediction is correct in identifying an ATP-binding protein?\nReply:\nWe thank the reviewer for the thoughtful questions. We selected ATP-binding proteins for this analysis because they\nrepresent a biologically important and well-characterized functional class. ATP-binding proteins play critical roles\nin numerous cellular processes, including energy transfer, signal transduction, and molecular transport, and they are\ncommonly studied in drug discovery as therapeutic targets. Moreover, the functional class of ATP-binding proteins\nhas sufficient diversity and sample size in our dataset, making it suitable for evaluating whether the learned\nrepresentations capture meaningful biological distinctions.\nIn Figure 3a, we apply UMAP to visualize the protein representations produced by MotiL’s encoder\n(specifically from the last graph convolution layer), and use DBSCAN to reveal natural clusters in the embedding\naugment\nSubgraph removal Subgraph modification Subgraph addition\naugment\nCID: 6028\nSimilarity:\n0.234\nSimilarity: 0.990\nSimilarity: 0.901\nBBBP Tox21\nSimilarity:\n0.947\nSimilarity: 0.204\nCID:7585\nSimilarity: 0.969\nSimilarity: 0.986\nToxCast SIDER\nClinTox BACE\nMUV HIV\naugment\nSimilarity:\n0.905\nSimilarity: 0.790\nCID: 32797\nSimilarity: 0.956\nSimilarity: 0.935\naugment\naugment CID: 72199430\nCID: 28145931\nSimilarity: Similarity: Similarity:\n0.947 0.337 0.965\nSimilarity: 0.831\nSimilarity: 0.924\nSimilarity: 0.905 Similarity: 0.931\naugment\nSimilarity: 0.755\nCID: 2999678\nSimilarity: 0.983\nSimilarity: 0.917\naugment\naugment CID: 433668\nSimilarity: 0.413\nSimilarity: 0.925\nSimilarity: 0.950\nspace. These clusters reflect how MotiL organizes proteins with similar functional signatures. We chose Cluster 6 as\nan illustrative example where MotiL groups together ATP-binding proteins such as 1WLE-A and 3LSS-A.\nRegarding the visual encoding, we confirm that the shape of each marker in the zoomed-in panel indicates\nwhether the ATP-binding function of that protein was predicted correctly by MotiL: circular markers denote\ncorrect predictions, while triangular markers denote incorrect predictions. As highlighted in the example, all proteins\nin Cluster 6 are predicted correctly, suggesting that MotiL’s embedding space supports functionally consistent\nclustering.\nWe have now clarified the visual encoding more explicitly: in the updated figure (as shown in Figure 5 below),\ncircular markers represent correct predictions, while triangular markers indicate incorrect ones. This clearer\nannotation ensures that readers can easily distinguish prediction correctness based on marker shape.\nFalse prediction True prediction id: 6\n1WLE-A\n3LSS-A\nFigure 5: UMAP visualization of the representations for ATP binding proteins (Cluster ID -1 represents noise\npoints as defined by DBSCAN, i.e., points not assigned to any cluster).\nComment 8: Figure 3c could be better organized. I prefer the layout used in Figure 5 SI, and I think more GO terms\nshould be written to highlight similarities. For example, in ID 3, the common GO terms, protein kinase activity,\nKinase activity, and Protein tyrosine kinase activity, could be included.\nReply:\nWe sincerely thank the reviewer for their insightful suggestion regarding the layout and annotation of Figure 3c,\nand we fully agree that enhancing the clarity and biological interpretability of this figure would improve the presentation\nof our results.\nTo address the reviewer’s concerns, we have made the following revisions:\n1. Improved figure layout: We have redesigned the layout of Figure 3c following the structure used in Figure 5\nof the Supplementary Information, as suggested. Instead of displaying only a few protein pairs, the updated\nversion (shown in Figure 6 below) presents representative protein structures from the same DBSCAN-identified\ncluster (id: 3). This revision facilitates a more intuitive and visually systematic comparison of the structural\nsimilarities among proteins within the same cluster, helping to better showcase the ability of MotiL to group\nproteins with related 3D folds.\n2. Expanded GO term annotation for functional similarity: We now include more comprehensive sets of\nshared Gene Ontology (GO) terms in each cluster to better reflect the biological consistency of the grouped\nproteins. As you suggest, we annotate the following common GO terms: “protein kinase activity”, “kinase\nactivity”, and “protein tyrosine kinase activity” for Cluster ID 3 (e.g., 4AU8-A, 1G3N-A, 2BPM-A, 5XVU-A,\nand 3VN9-A). These GO terms are extracted based on shared functional annotations from UniProt and are\nselected to emphasize high-confidence, biologically relevant commonalities. Specifically, protein kinase\nactivity (GO:0004672) refers to the enzymatic function of transferring a phosphate group, typically from ATP,\nto a protein substrate. This post-translational modification is crucial for regulating protein function, activity,\nlocalization, and interactions in a wide range of cellular processes. Kinase activity (GO:0016301) is a broader\nterm encompassing all enzymes that catalyze the phosphorylation of substrates, not limited to proteins. It\nincludes protein kinases, lipid kinases, carbohydrate kinases, and others involved in signal transduction and\nmetabolic regulation. Protein tyrosine kinase activity (GO:0004713) is a subtype of protein kinase activity that\nspecifically phosphorylates tyrosine residues on proteins. These kinases are key regulators of signaling\npathways that control cell growth, differentiation, immune response, and oncogenic transformation.\n3. Clarified caption and in-text description: To guide interpretation, we revised the caption of Figure 3c in the\nmain text as follows: “c, 3D structure comparison of proteins in cluster ID 3. Proteins within the same cluster\nshow similar 3D conformations and share consistent GO annotations such as protein kinase activity.”\nFurthermore, in the main text, we have revised the description of Figure 3c in the section “MotiL learns\nmacromolecular representations consistent with protein structures and chemical properties” to better reflect the\nimproved layout and the inclusion of representative GO terms that highlight functional similarity among clustered\nproteins:\n“We then focus on specific cases, examining the three-dimensional (3D) structures of proteins (Fig. 3c,\nSupplementary Fig. 5). Fig. 3c illustrates that within Cluster 3, proteins 4AU8-A, 1G3N-A, 2BPM-A, 5XVU-A,\nand 3VN9-A display significant structural similarity to each other. Additionally, these proteins in the same cluster are\nlinked by shared GO terms, such as protein kinase activity.”\n4AU8-A 1G3N-A 2BPM-A\n5XVU-A 3VN9-A\nCommon GO terms for cluster ID 3 include: protein kinase\nactivity, kinase activity, and protein tyrosine kinase activity.\nFigure 6: 3D structure comparison of proteins in cluster ID 3.\na Prediction statistics on the full datasets b Prediction statistics on activity cliff samples\nc d\nCases in BBBP (blood-brain barrier permeability) Cases in SIDER (hepatotoxicity)\nStreptomycin Mexazolam Erlotinib Vincristine\nKANO: 1 ✗ KANO: 0 ✗ KANO: 0 ✗ KANO: 0 ✗\nMotiL: 0 ✓ MotiL: 1 ✓ MotiL: 1 ✓ MotiL: 1 ✓\nFigure 7: Quantitative and qualitative analysis of prediction differences between KANO and MotiL on BBBP\nand SIDER. a, Statistics on disagreements between the two models on the full datasets. b, Statistics on\ndisagreements between the two models on activity cliff samples. c, Prediction cases for blood-brain barrier\npermeability on BBBP. d, Prediction cases for hepatotoxicity on SIDER.\nComment 9: The authors also make a more in-depth comparison between the 2nd best algorithm (KANO) and\nMotiL, but I struggle to understand the significance of this section. In fact, improvements between KANO and MotiL\nappear to be marginal. While I acknowledge that MotiL correctly predicts BBB permeability and toxicity for key\nmolecules while KANO does not, I find this section unnecessarily long and less relevant. I believe a more challenging\ncomparison could be beneficial. For example, evaluating performance on activity cliffs.\nReply:\nWe sincerely thank the reviewer for this insightful and constructive comment. We fully agree that evaluating on more\nchallenging cases, such as activity cliffs, can better highlight the differences between models and is more meaningful\nthan simply reporting average scores. In this revision, we have made the following comprehensive updates to address\nthis concern:\n1. On the significance of the case-level comparison with KANO: While the average metric improvements\nbetween MotiL and KANO may seem modest (e.g., ROC-AUC: +0.9% on BBBP, +0.4% on SIDER), we argue\nthat such summary statistics can obscure critical differences in model behavior, especially on hard-to-predict or\nchemically ambiguous samples.\nTo better highlight these differences, we focused on the subset of disagreement cases, i.e., samples where\nthe two models (KANO vs. MotiL) give different predictions. As shown in Figure 7a, these disagreement sets\naccount for 28.1% of the BBBP dataset and 21.4% of the SIDER dataset. Within these subsets:\n(1) On BBBP, MotiL is correct in 40 out of 57 disagreement cases (70.2%), while KANO is correct in only\n17 (29.8%).\n(2) On SIDER, MotiL is correct in 43 out of 63 disagreement cases (68.3%), compared to 20 (31.7%) for\nKANO.\nThe above results demonstrate that MotiL tends to make the right decision when the models disagree,\nimplying stronger reliability in high-uncertainty or decision-boundary situations. Also, we provide examples in\nFigure 7c–d, where MotiL correctly predicts toxicity or permeability for drugs like Streptomycin, Mexazolam,\nErlotinib, and Vincristine, which KANO misclassifies.\n2. Activity cliff evaluation – experimental setup and findings: We fully agree with the reviewer that evaluating\non activity cliffs provides a more rigorous and chemically meaningful comparison. Following your suggestion,\nwe designed a specific activity cliff evaluation protocol as follows:\n(1) From the BBBP and SIDER datasets, we identified molecular pairs with high Tanimoto similarity (>0.85)\nbut significantly different labels.\n(2) We selected individual molecules involved in such pairs to construct the activity cliff test sets.\n(3) For each dataset, we analyzed prediction disagreements between KANO and MotiL on these challenging\nsamples.\nAs shown in Figure 7b, on BBBP, among 15 disagreement cases, MotiL correctly predicted 10 (66.7%),\nwhile KANO only predicted 5 (33.3%); on SIDER, among 19 disagreement cases, MotiL predicted 11 correctly\n(57.9%), compared to 8 (42.1%) for KANO.\nMoreover, Figures 8 and 9 provide concrete activity cliff examples, including: pairs with Tanimoto\nsimilarity of 0.85–1.0, with minor differences in functional groups; cases where KANO fails to distinguish\nstructurally similar molecules with different activities, while MotiL succeeds in capturing subtle motif-level\ndistinctions in most cases.\nThese results suggest that the motif-level representation learning and diffusion-based denoising\nmechanism in MotiL better capture fine-grained chemical variations, enabling the model to distinguish subtle\nfunctional group differences that often drive activity cliffs. It highlights MotiL’s stronger inductive bias toward\nchemically meaningful substructures.\n3. Clarifying the novelty of MotiL beyond KANO: While KANO performs well and incorporates chemically\nmotivated augmentations, MotiL offers a conceptually distinct and more expressive framework:\n(1) Motif-level contrastive alignment: MotiL explicitly identifies and aligns representations of functional\ngroups across molecules, but KANO does not support it.\n(2) Bi-scaled training: MotiL simultaneously aligns representations at both the graph and motif level,\ncapturing both global molecular semantics and local substructure invariants.\n(3) Diffusion-based pretraining (DiffMoM): The integration of a denoising diffusion model improves\nrobustness to structural perturbations, a key weakness in most augmentation-based contrastive models\nincluding KANO.\nThese novel contributions allow MotiL to better generalize in noisy, or structurally ambiguous regimes,\nincluding activity cliffs.\n4. Revision summary in the manuscript:\n(1) We revised Figure 4 in the main text to summarize prediction disagreements on the full datasets (BBBP\nand SIDER). We have added the following text to the Section “MotiL corrects the erroneous prediction\nof the strongest baseline” to introduce the activity cliff experiment and its corresponding results:\n“Fig. 4b, Supplementary Fig. 10, and Supplementary Fig. 11 present the results of the activity cliff\nexperiment, designed to evaluate model robustness on challenging molecular pairs with high structural\nsimilarity but divergent labels. We identified activity cliff samples from BBBP and SIDER by selecting\nmolecular pairs with a Tanimoto similarity above 0.85 and differing binary labels. Among the\ndisagreement cases on these activity cliffs, MotiL achieves higher accuracy than KANO: on BBBP, MotiL\ncorrectly predicts 10 out of 15 samples (66.7%) compared to KANO’s 5 (33.3%); on SIDER, MotiL is\ncorrect on 11 of 19 samples (57.9%), while KANO predicts 8 correctly (42.1%). The results in Fig. 4a\nand Fig. 4b demonstrate that MotiL outperforms KANO in handling ambiguous or challenging samples.”\n(2) The specific setting and case study in the activity cliff experiment have been added to Supplementary\nSection L (Activity cliff evaluation):\nSection L.1 Experimental setup: “To rigorously evaluate the robustness of MotiL against small\nstructural perturbations that cause large changes in molecular properties, we conducted an activity cliff\n(AC) experiment on the BBBP and SIDER datasets.\nThe identification of activity cliff samples follows a widely accepted protocol in cheminformatics:\nl Data source: We use the test set molecules from the BBBP and SIDER benchmark datasets.\nl Similarity filtering: We compute pairwise Tanimoto similarity scores based on RDKit Morgan\nfingerprints. Only molecular pairs with a similarity score greater than 0.85 are retained,\nindicating high structural similarity.\nl Label divergence: Among these similar pairs, we select those where the two molecules have\ndifferent binary labels. These cases constitute activity cliff pairs, where slight structural\ndifferences lead to opposite bioactivities.\nl Sample selection: From each AC pair, we extract individual molecules and evaluate whether\nthe model correctly predicts their true label. We then compare MotiL and KANO on the subset\nof disagreement cases within this AC group.\nThis setting is intentionally challenging, as it tests the model's ability to distinguish subtle\nfunctional variations that significantly impact the biological activity.”\nSection L.2 Experimental results and case study: “As shown in Fig. 4b (main paper), the activity\ncliff experiment reveals that MotiL demonstrates stronger predictive robustness than KANO:\nOn the BBBP dataset, among 15 AC disagreement samples, MotiL predicts 10 correctly\nl\n(66.7%), while KANO predicts 5 (33.3%).\nOn the SIDER dataset, MotiL correctly predicts 11 out of 19 (57.9%), whereas KANO\nl\npredicts 8 (42.1%).\nThese results indicate that MotiL is more effective at resolving difficult cases involving high\nstructural similarity but diverging functional outcomes, which are common in medicinal chemistry and\ntoxicity analysis.\nTo better illustrate this advantage, we present selected case studies in Supplementary Fig. 10\n(BBBP) and Fig. 11 (SIDER), where each figure shows: 1. the SMILES strings of two molecules from the\nAC pair; 2. their binary labels (0 or 1); 3. their Tanimoto similarity score; 4. prediction outcomes from\nboth MotiL and KANO.\nCases in BBBP (Fig. 10): In the first three pairs, MotiL correctly predicts both molecules in each\npair, whereas KANO misclassifies one molecule per pair. These examples illustrate MotiL’s stronger\nsensitivity to local functional group variations, such as substitutions between carboxyl and ester groups,\nor small changes in stereochemistry on steroid-like scaffolds (e.g., pair with similarity 0.851). In the last\ntwo pairs, both models show imperfect performance. Each model misclassifies one molecule per pair,\nindicating that certain structural subtleties remain challenging even for MotiL, despite its overall\nsuperior performance on activity cliffs.\nCases in SIDER (Fig. 11): In the SIDER dataset, multiple AC pairs are shown with high similarity\n(e.g., 0.956 and 1.0) and opposite toxicity labels. In the first pair (Tanimoto similarity = 0.956), KANO\ngives identical predictions, while MotiL distinguishes the two correctly. It demonstrates that when the\nstructural similarity is high but the relevant substructural cues are distinguishable, MotiL can succeed.\nIn the remaining three pairs, the two models show mixed behavior: For each pair, one molecule is\nmisclassified by KANO and the other is misclassified by MotiL. It reflects that both models struggle to\nsome extent in resolving the subtle chemical cues that define the activity cliff boundary in these samples.\nThese results further support the finding that MotiL demonstrates better robustness to activity cliffs\noverall, while also highlighting specific cases where further improvement in motif-level discrimination\nis still needed.”\nTanimoto\nsmilarity\nLabel: 1 Label: 0\n1.0 KANO: ✓ KANO: ✗\nMotiL: ✓ MotiL: ✓\nC1=CC=CC(=C1C(O)=O)OC(C)=O CC(=O)Oc1ccccc1C(O)=O\nLabel: 1 Label: 0\n0.851 KANO: ✓ KANO: ✗\nMotiL: ✓ MotiL: ✓\nCCC(=O)C1(O)CCC2C3CCC4=CC(=O)CCC4(C)C3C(O)CC21C C[C@]12CCC(=O)C=C1CC[C@H]3[C@@H]4CC[C@]\n(O)(C(=O)CO)[C@@]4(C)C[C@H](O)[C@H]23\nLabel: 1 Label: 0\n1.0 KANO: ✓ KANO: ✗\nMotiL: ✓ MotiL: ✓\n[C@H](CC1=CC(=C(C=C1)O)O)(C(O)=O)N N[C@@H](Cc1ccc(O)c(O)c1)C(O)=O\nLabel: 1 Label: 0\n1.0 KANO: ✓ KANO: ✗\nMotiL: ✓ MotiL: ✗\nCN1C2CCC1CC(C2)OC(=O)[C@H](CO)c3ccccc3 CN1C2CCC1CC(C2)OC(=O)C(CO)c3ccccc3\nLabel: 1 Label: 0\n0.972 KANO: ✓ KANO: ✗\nMotiL: ✗ MotiL: ✓\n[Cl].CC(C)NCC(O)COc1cccc2ccccc12 CC(C)NCC(O)COc1cccc2ccccc12\nFigure 8: Prediction results for activity cliff samples on BBBP.\nTanimoto\nsmilarity\nLabel: 1 Label: 0\n0.956 KANO: ✓ KANO: ✗\nMotiL: ✓ MotiL: ✓\nCC(C)C[C@@H](C(=O)N[C@@H](CCCN=C(N)N)C(= CC(C)CC(C(=O)NC(CCCN=C(N)N)C(=O)N1CCCC1C(=\nO)N1CCC[C@H]1C(=O)NCC(=O)N)NC(=O)[C@@H]( O)NCC(=O)N)NC(=O)CNC(=O)C(CC2=CC=C(C=C2)O)\nCC2=CNC3=CC=CC=C32)NC(=O)[C@H](CC4=CC=C(C NC(=O)C(CO)NC(=O)C(CC3=CNC4=CC=CC=C43)NC(=\n=C4)O)NC(=O)[C@H](CO)NC(=O)[C@H](CC5=CNC6 O)C(CC5=CN=CN5)NC(=O)C6CCC(=O)N6\n=CC=CC=C65)NC(=O)[C@H](CC7=CN=CN7)NC(=O)[\nC@@H]8CCC(=O)N8\nLabel: 1 Label: 0\n1.0 KANO: ✓ KANO: ✗\nC(CNCCNCCNCCN)N MotiL: ✗ C(CNCCNCCN)N MotiL: ✓\nLabel: 1 Label: 0\n0.860 KANO: ✓ KANO: ✗\nMotiL: ✗ MotiL: ✓\nCCCN1CCCCC1C(=O)NC2=C(C=CC=C2C)C CCCCN1CCCCC1C(=O)NC2=C(C=CC=C2C)C\nLabel: 1 Label: 0\n0.883 KANO: ✓ KANO: ✗\nMotiL: ✗ MotiL: ✓\nCC(=O)N(CC(CO)O)C1=C(C(=C(C(=C1I)C(=O)NCC(CO CC(=O)N(CC(CO)O)C1=C(C(=C(C(=C1I)C(=O)NCC(CO\n)O)I)C(=O)NCCO)I )O)I)C(=O)NCC(CO)O)I\nFigure 9: Prediction results for activity cliff samples on SIDER.\nComment 10: I appreciate the author’s commitment to improving the model in future work, with already some\ndefined future steps that could further accelerate drug discovery.\nReply:\nWe sincerely thank the reviewer for their supportive and encouraging feedback. We are pleased that our discussion\nof future work was found meaningful and aligned with the overarching goal of accelerating drug discovery through\nmachine learning.\nAs outlined in the manuscript, we see multiple promising avenues for extending our work with MotiL, many\nof which are aimed at addressing critical challenges in molecular modeling and real-world drug development\npipelines. Below, we elaborate on these directions with more technical depth:\n1. Advancing the diffusion priming phase with more expressive generative models: In the current version of\nMotiL, the diffusion priming phase utilizes a graph-based generative denoising model (DiffMoM) to improve\nthe encoder’s robustness and capture chemically consistent structures. However, more expressive generative\nmodels, such as those incorporating equivariant neural networks or continuous-time score-based generative\nmodels, may significantly enhance the structural fidelity and diversity of the diffusion process. This\nimprovement could lead to: (1) finer control over local motif-level perturbations, (2) stronger structural priors\nembedded in representations, and 3) better generalization to novel chemical scaffolds.\n2. Hybridizing representation learning and molecule generation: We are particularly excited about bridging\nthe gap between representation learning and molecular generation. Given that MotiL captures semantically\nmeaningful motif-level representations, one natural next step is to employ these embeddings as building blocks\nin a motif-conditioned generative framework. Such a hybrid approach could: (1) facilitate the design of novel\nmolecules with desired pharmacological properties, (2) enable property-guided optimization (e.g., minimizing\ntoxicity while preserving activity), and (3) support de novo scaffold generation with biologically relevant\nsubstructures. This direction aligns with our vision of using representation learning not only for predictive\nmodeling, but also for proactive design and exploration in chemical space.\n3. Accelerating drug discovery and personalized medicine: By extending MotiL’s capabilities into the molecule\ndesign space, we foresee broader applications in: (1) targeted therapy development (e.g., designing ligands for\nspecific protein targets), (2) personalized medicine (e.g., generating patient-specific drug variants based on\nmolecular fingerprints or genomic data), and (3) synthetic biology and diagnostic tool design (e.g., engineering\nmolecules with specific reactivity or detectability). The ability to learn from known functional motifs and\ngeneralize to unseen yet plausible bioactive compounds may uncover treatments for unmet clinical needs and\ndrive more efficient lead identification, optimization, and validation.\nWe thank the reviewer once again for acknowledging the importance of these directions. We are committed to\nadvancing MotiL in a way that not only improves learning algorithms but also contributes to translational impact\nin biomedical applications.\nComment 11: Finally, I recommend revising the references. For example, reference [1], Diaz, N. et al. Discovery\nof potent small molecule inhibitors of lipoprotein (a) formation. Nature 1–6 (2024), it’s written on the journal’s\nwebsite as Diaz, N. et al. Discovery of potent small-molecule inhibitors of lipoprotein(a) formation. Nature 629,\n945–950 (2024). Reference [7] Wan, Z. et al. Unconventional superconductivity in chiral molecule-tas2 hybrid\nsuperlattices. Nature 1–6 (2024) it’s written on the journal’s website as Wan, Z, et al. Unconventional\nsuperconductivity in chiral molecule–TaS2 hybrid superlattices. Nature 632, 69–74 (2024).\nGiven the increasing importance of ML in advancing drug discovery and the innovative methodology presented in\nthis work, I recommend this article be accepted for revision.\nReply:\nWe thank the reviewer for carefully identifying inaccuracies in the formatting of our references, and we fully agree\nthat accurate and properly formatted citations are essential for maintaining scientific rigor and clarity. In response,\nwe have conducted a comprehensive revision of all references in our manuscript to ensure correctness, consistency,\nand compliance with official journal formats.\n1. Specific corrections made: We have carefully updated the two references mentioned by the reviewer based on\ntheir official records from the Nature website: Reference [1] was previously cited as: “Diaz, N. et al. Discovery\nof potent small molecule inhibitors of lipoprotein (a) formation. Nature 1–6 (2024).” It has been revised to:\n“Diaz, N. et al. Discovery of potent small-molecule inhibitors of lipoprotein(a) formation. Nature 629, 945–\n950 (2024).” Key corrections include: (1) hyphenation of “small-molecule” per journal usage; (2) parenthetical\nformatting of “lipoprotein(a)” with no added space; (3) insertion of correct volume number (629) and page\nrange (945–950) as published. Reference [7] was previously cited as: “Wan, Z. et al. Unconventional\nsuperconductivity in chiral molecule-tas2 hybrid superlattices. Nature 1–6 (2024).” It has been revised to:\n“Wan, Z. et al. Unconventional superconductivity in chiral molecule–TaS₂ hybrid superlattices. Nature 632,\n69–74 (2024).” Key corrections include: (1) correct casing and formatting of the compound “TaS₂”, with a\nsubscript “2”; (2) insertion of official volume number (632) and page numbers (69–74).\n2. Global reference audit: In addition to the two references explicitly pointed out, we took this opportunity to\nsystematically review all references (total of 21 references in the main text and 23 references in the appendix).\nWe made the following corrections where necessary:\n(1) Ensured that volume numbers and page ranges were accurately included for all journal articles. For\nexample:\na) “Du, Y. et al. Machine learning-aided generative molecular design. Nature Machine Intelligence\n1–16 (2024).” -> “Du, Y. et al. Machine learning-aided generative molecular design. Nature\nMachine Intelligence 6, 589–604 (2024).”\nb) “Liu, S., Demirel, M. F. & Liang, Y. N-gram graph: Simple unsupervised representation for graphs,\nwith applications to molecules. Advances in neural information processing systems 32 (2019).” ->\n“Liu, S., Demirel, M. F. & Liang, Y. N-gram graph: Simple unsupervised representation for graphs,\nwith applications to molecules. Advances in neural information processing systems 32, 8464–8476\n(2019).”\n(2) Corrected any title formatting inconsistencies. For example:\na) “Hermosilla, P. & Ropinski, T. Contrastive representation learning for 3d protein structures. arXiv\npreprint arXiv:2205.15675 (2022).” -> “Hermosilla, P. & Ropinski, T. Contrastive representation\nlearning for 3D protein structures. arXiv preprint arXiv:2205.15675 (2022).”\nb) “Qiang, B. et al. Coarse-to-fine: a hierarchical diffusion model for molecule generation in 3d.\nInternational Conference on Machine Learning 28277–28299 (2023).” -> “Qiang, B. et al.\nCoarse-to-fine: a hierarchical diffusion model for molecule generation in 3D. Proceedings of the\n40th International Conference on Machine Learning 28277–28299 (2023).”\n(3) Standardized journal or conference titles to match official publication style. For example:\na) “Nat. Mach. Intell.” -> “Nature Machine Intelligence”\nb) “Advances in Neural Information Processing Systems 34: Annual Conference on Neural\nInformation Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual” ->\n“Advances in Neural Information Processing Systems 34”\nc) “Bioinform.” -> “Bioinformatics”\nFinally, we sincerely thank the reviewer for the positive assessment of our work and for recognizing both the\nrelevance of machine learning to modern drug discovery and the novelty of our proposed methodology. We greatly\nappreciate your recommendation for revision and have carefully addressed all comments and suggestions to further\nimprove the clarity, rigor, and completeness of the manuscript. Your feedback has been instrumental in helping us\nrefine our contributions and better position our work for impact in this rapidly evolving field.\nComment 12 (Remarks on code availability):\n-The library versions used are indicated\n- Databases used for small molecules are available on git-hub. However, for the MUV, QM7, and ToxCast, the\n#molecules written on the article don't appear to match the CSV file lengths.\n- I couldn't access the databases for macromolecules, I was unable to access the Baidu Cloud.\n- The pretrained models for small molecule predictions are available on github providing a readme file with a clear\ndescription of MotiL, requirements.txt, and instructions on how to run MotiL\n- I couldn't access the pretrained models for macromolecule predictions, I was unable to access the Baidu Cloud\n- I used codeocean website to run the available code and the example of the bbbp database led to the same output\nfile (Results: 0.97404 +/- 0.00289). It was reproducible, but was it supposed to match Table 1?\nReply:\nWe sincerely thank the reviewer for their careful inspection of our codebase, data resources, and reproducibility\npipeline, as well as for the positive feedback on several key aspects. Below, we address each point in detail:\n1. Typographical errors in molecule counts for MUV, QM7, and ToxCast: We appreciate the reviewer’s\nattention to the molecule counts for the small molecule datasets MUV, QM7, and ToxCast. After thorough\nverification, we confirm that the CSV files hosted on GitHub are correct. The molecule numbers reported in the\nmanuscript contained typographical errors introduced during manuscript preparation. Specifically:\n(1) MUV: The actual number of molecules in the dataset is 93,087.\n(2) QM7: The correct number is 6,830.\n(3) ToxCast: The correct number is 8,597.\nWe have corrected these numbers in both the main paper (Table 1) and in Supplementary Table 1 to match\nthe true values in the released data files.\n2. Macromolecule dataset and pretrained model checkpoints inaccessible due to Baidu Cloud restrictions:\nWe acknowledge the reviewer's difficulty in accessing the macromolecular datasets and pretrained models\npreviously hosted on Baidu Cloud, which may be inaccessible in certain regions. To address this and ensure\nglobal accessibility without registration or regional limitations, we have made the following improvements:\n(1) Re-hosted all macromolecule benchmark datasets on Hugging Face Hub, which is widely accessible\ninternationally and does not require regional authentication. The datasets are available at:\nhttps://huggingface.co/datasets/Young0222/MotiL/tree/main.\n(2) Uploaded the pretrained model checkpoints for macromolecule prediction tasks to the same Hugging\nFace repository.\n(3) Updated the relevant links in our GitHub repository (e.g., the README file as shown in Figure 10) to\nreflect this new hosting location.\nFigure 10: User instructions for macromolecule tasks provided in the README file on GitHub.\nThe Hugging Face repository now contains all the necessary files to reproduce the experiments in the\nmacromolecule benchmarks, and the pretrained models can be freely accessed worldwide without geographical or\ntechnical restrictions.\n3. BBBP result reproduced via Code Ocean: We have publicly provided the full training logs to ensure\nreproducibility. They are available at: https://github.com/Young0222/MotiL. The BBBP result you obtained on\nCode Ocean (0.97404 ± 0.00289) is indeed slightly higher than the result reported in our paper (0.969 ± 0.003,\nSupplementary Table 3). This variation is primarily due to differences in hardware and software\nenvironments between Code Ocean and our local training setup. Specifically, as shown in Table 5:\nTable 5: Environment setup comparison between Code Ocean and our server.\nIn this table, for example:\n(1) GPU hardware architecture differences: The Tesla T4 (Turing architecture) and RTX 3090 (Ampere\narchitecture) differ significantly in their numerical precision handling, Tensor Core capabilities, and\nsupport for mixed-precision computation (float16 vs. float32). The T4 is optimized for efficiency and\nlow-power inference tasks, with a stronger emphasis on float16 computation. These architectural\ndifferences can introduce greater numerical drift during training, potentially leading to different\nconvergence paths even under identical seed settings.\n(2) PyTorch and cuDNN version differences: PyTorch 1.12.1 and 1.13.0 introduce subtle behavioral\ndifferences, particularly in components such as LayerNorm, Dropout, and optimizer initialization (e.g.,\nAdamW). Additionally, cuDNN version 8.3 vs. 8.5 may alter internal precision vs. speed trade-offs for\nconvolution and normalization layers. Some operators in newer versions may favor more aggressive\nkernel implementations, which can affect numerical stability, especially in smaller datasets or edge-case\nlearning scenarios.\nThese differences, particularly in GPU architecture and software versions, can introduce slight variations in\ntraining dynamics, which lead to minor differences in final results. Code Ocean recommends using their default\nenvironment configuration, and we adhered to that requirement during submission. To further ensure that our\nreported results can be exactly reproduced, we have uploaded the trained models to Code Ocean. You can run the\nprediction step directly with these models and obtain the same results as reported in our paper (see Figure 11).\nThe Code Ocean link is: https://codeocean.com/capsule/9672051/tree. We appreciate your attention to\nreproducibility and hope this clarification resolves your concerns.\nFigure 11: Output results from MotiL's prediction for BBBP on Code Ocean.\n4. Code and models for small molecules – accessibility and clarity: We are pleased that the reviewer found the\nresources for small molecule tasks, such as the GitHub code, pretrained models, requirements.txt, and\nREADME instructions, to be clear, functional, and easy to follow. We put considerable effort into ensuring our\nrepository is well-documented and reproducible, and we will continue to maintain and improve the codebase\nfor community use.\nWe have correspondingly revised the Sections of Data availability and Code availability in the main text:\n“The pretraining data and molecular property prediction benchmarks used in this work are available on the\nHugging Face Hub at https://huggingface.co/datasets/Young0222/MotiL/tree/main and the GitHub repository at\nhttps://github.com/Young0222/MotiL.”\n“The source code of this work is freely available in the Code Ocean capsule\nhttps://codeocean.com/capsule/9672051/tree and the GitHub repository at https://github.com/Young0222/MotiL.”"}, {"round": 2, "reviewer_comments": {"reviewer_1": "I appreciate the authors diligence in addressing the great majority of my concerns. Overall, I am satisfied with the the\ntechnical quality of the work. My concerns with the impact & relevance to the readership of the journal remain though I, of\ncourse, defer to the judgment of the editors.", "reviewer_2": "The authors have addressed my suggestions.\nParticularly:\n- They clarified their commitment to a fair comparison between methods. It is now clear how the experimental setup enables\na fair model comparison.\n- I appreciate how they made the colors in Fig. 2 more accessible. However, I unsure whether it needs to be mentioned\nin the main text (\"where the color palette has been optimized to enhance visual contrast and interpretability\", lines 330–332).\nAfter this change, I would also suggest harmonizing the colors used throughout the manuscript.\n- I believe the authors have more clearly highlighted the strengths of MotiL over KANO.\n- I appreciate the authors' commitment to continuing to work on the algorithms in future steps, particularly in molecule\ngeneration.\nAs before, I recommend this article for publication.", "reviewer_3": "Dear Reviewers,\nWe sincerely thank the reviewers for the thorough evaluation of our manuscript and for recognizing the significance\nof our work. We are also grateful for the reviewers’ constructive comments and insightful suggestions, which have\nhelped us identify areas for improvement and significantly enhanced the quality of our manuscript. In the revised\nmanuscript, we have carefully addressed all comments and highlighted the corresponding changes in blue font for\nclarity. In this response letter, we provide a point-by-point reply to each of the comments from Reviewer #1 (pages\n1 to 24) and Reviewer #2 (pages 25 to 45), respectively.\nResponse to Reviewer #1:\nComment 1: For the MoleculeNet-based benchmark datasets, do the authors perform any further processing or\nstandardization? At this point, it is well known that these datasets contain erroneous labelling and duplicate examples\nas well as non-standardized structures [1]. I think benchmarking on these types of datasets is fine (if not super\nmeaningful) for a ML conference-type publication but hinders the case for publication in a journal with a broader\naudience. Furthermore, these benchmarks are in general quite uninformative for real-world applications. Have the\nauthors considered benchmarking on more application-focused datasets such as [2]? Overall, I believe the impact is\nlimited. While the proposed method does consistently improve on previous methods, there is only a marginal\nimprovement over KANO in the small molecule tasks. And the benefit on the macromolecular tasks is less clear\nwhen compared to pre-trained structural encoders. Taken together with the weakly motivated benchmarks I think\nthis incremental performance improvement does not warrant publication in a journal with such broader readership.\nAs it stands, I think there is little for non-ML researchers to take from this paper and so targeting a venue with amore\nML-focused readership would be more suitable for this particular work.\n[1] https://practicalcheminformatics.blogspot.com/2023/08/we-need-better-benchmarks-for-machine.html\n[2] https://doi.org/10.1021/acs.jcim.3c00160\nReply:\nWe sincerely thank the reviewer for this constructive and detailed feedback. We have carefully addressed each aspect\nof the comment and have made significant clarifications and additions in the revised manuscript, summarized below:\n1. On the use of MoleculeNet-based benchmark datasets: We acknowledge the reviewer’s concern regarding\nthe known limitations of MoleculeNet datasets, such as erroneous labels, duplicated entries, and lack of\nmolecular standardization [1]. As with many previous studies (e.g., GROVER, MolCLR, KANO), our intention\nin using MoleculeNet is to ensure fair and directly comparable benchmarking under widely recognized settings.\nThese benchmarks, while imperfect, continue to serve as a de facto standard in the field of molecular machine\nlearning, offering controlled comparisons across methods. We emphasize that we did not apply any additional\nprocessing or standardization to these datasets beyond the standard splits provided, in order to strictly follow\nthe established evaluation protocols used by prior work. This design choice facilitates meaningful comparisons\nwith strong baselines and highlights the incremental benefits offered by our method within this common\nframework.\n2. On evaluating with more application-oriented datasets: We appreciate the reviewer’s excellent suggestion\nto evaluate MotiL on more realistic, application-focused datasets. To this end, we have conducted new\nexperiments on a benchmark introduced by [2], which focuses on absorption, distribution, metabolism, and\nexcretion (ADME) properties that are central to real-world drug development and pharmacokinetics. There are\nsix ADME-related prediction tasks in this dataset: HLM clearance, human plasma protein binding (hPPB),\nMDR1-MDCK efflux ratio, rat liver microsome (RLM) stability, rat plasma protein binding (rPPB), and\naqueous solubility. These properties are clinically relevant and span diverse pharmacological mechanisms. We\ncompare our method against strong baselines (CMPNN, MGSSL, MolCLR, KANO), and the RMSE results\nunder three independent runs using three random-seeded are summarized in Table 1 (lower RMSE value\nindicates better performance):\nTable 1: Prediction result comparison of different methods on ADME datasets.\nThe results demonstrate that MotiL achieves consistently stable and competitive performance across all\nsix ADME-related prediction tasks. On most tasks, MotiL not only achieves strong predictive performance but\nalso maintains low standard deviations (e.g., ±0.0054 for HLM and ±0.0047 for Solubility), indicating its\nrobustness and reproducibility. In Table 1, we use bold font to denote the best performance on each task and\nitalic font to indicate the second-best. Statistical analysis further confirms that MotiL significantly outperforms\nthe second-best method (e.g., KANO on the task of hPPB) on each task (p < 0.05), demonstrating its clear\nsuperiority. Overall, MotiL consistently ranks among the best-performing methods with minimal fluctuations\nacross diverse pharmacokinetic properties, highlighting its strong generalization ability. Moreover, MotiL\nhandles these tasks within a unified framework without the need for task-specific tuning, demonstrating its\npractical utility for real-world drug discovery applications.\n3. On the perceived marginal improvement and general impact:\n(1) Statistically significant improvements over KANO: While the numerical differences between MotiL and\nKANO on small-molecule tasks may seem modest in some cases, we emphasize that these improvements are\nstatistically significant and consistently reproducible. To rigorously verify it, we conducted paired two-tailed t-\ntests across all 14 MoleculeNet benchmark datasets comparing MotiL with the strongest baseline, KANO. The\nresults, presented in Table 2 and Table 3 (also included as Supplementary Tables 3 and 4), show that all p-\nvalues are below 0.05, confirming that the improvements are statistically significant and unlikely to result\nfrom random fluctuations. Moreover, the standard deviations across multiple runs are consistently low\n(typically <1.5%), indicating that the performance is stable across different random seeds and initializations.\nTherefore, the observed gains, though sometimes small in absolute value, are consistent, statistically valid, and\nreproducible, reflecting the robustness of our approach.\n(2) Addressing both micro- and macromolecular property prediction: Beyond numerical gains, the key\ncontribution of MotiL is its unified framework that supports both small-molecule and macromolecule\nproperty prediction tasks, which most prior methods cannot do. For example, models such as KANO and\nMolCLR are designed specifically for small-molecule graphs and do not generalize to macromolecular graphs\n(e.g., peptides, nucleotides). Some macromolecular methods rely on pre-trained protein language models or\n3D structural encoders. However, these methods require expensive and curated protein structures; cannot\nhandle small-molecule graphs effectively; lack a unified representation space for both molecular types. In\ncontrast, MotiL jointly pre-trains on small-molecule and macromolecular graphs using both 1D sequences and\n3D geometric information, enabling it to generalize across diverse chemical spaces without task-specific tuning.\nThis versatility meets real-world drug discovery needs, supporting both small-molecule ADME property\nprediction and macromolecular interaction modeling, even when structural data are incomplete.\nTable 2: ROC-AUC (%) performance comparison on small molecules. The testing results are presented as the\nmean and standard deviation, calculated from three independent runs. The best result for each dataset is\nhighlighted in bold, and the runner-up result is italicized. All baseline results are directly taken from Ref. [23],\nwhich follows the same evaluation protocol as this study: three random-seeded scaffold splits for small molecules.\nTable 3: Performance comparison on small molecules. We use the metric of RMSE or MAE where a lower value\nindicates better performance. The testing results are presented as the mean and standard deviation, calculated from\nthree independent runs. The best result for each dataset is highlighted in bold, and the runner-up result is italicized.\nAll baseline results are directly taken from Ref. [23], which follows the same evaluation protocol as this study:\nthree random-seeded scaffold splits for small molecules.\n[1] https://practicalcheminformatics.blogspot.com/2023/08/we-need-better-benchmarks-for-machine.html\n[2] https://doi.org/10.1021/acs.jcim.3c00160\nTo better reflect these distinctions, we have revised the manuscript (Supplementary Section G) to include:\n(1) a more explicit discussion of the limitations of MoleculeNet and motivation for ADME evaluation; (2) an\nexpanded section comparing real-world ADME performance; (3) additional clarity in the scope, positioning,\nand impact of our work across application contexts.\nWe have also incorporated the above content into Supplementary Section I (Expanded discussion on\nbenchmark limitations and real-world ADME evaluation) as follows:\nSection I.1 (Limitations of MoleculeNet benchmarks): “While MoleculeNet remains the most widely\nadopted benchmark suite for evaluating molecular property prediction models, its use is not without challenges.\nPrevious studies and domain experts have highlighted issues such as label noise, duplicate molecules, and\nstructural inconsistencies (e.g., protonation states, tautomers, stereochemistry) within its datasets [72]. These\nlimitations can affect the reliability of comparative evaluation, especially for tasks requiring high-fidelity\nchemical understanding. Nevertheless, due to its established baselines and broad community adoption,\nMoleculeNet provides a useful foundation for measuring progress in molecular machine learning under\nstandardized settings. To maintain comparability with prior work such as GROVER, MolCLR, and KANO, we\nused the unmodified MoleculeNet datasets, adhering to the same data splits and evaluation protocols. This\nenables fair head-to-head comparisons while acknowledging the benchmark’s inherent constraints.”\nSection I.2 (Motivation for evaluating on ADME datasets): “To move beyond synthetic benchmarks and\ntoward more application-driven evaluation, we conducted additional experiments on six ADME-related\ndatasets [73]. These datasets reflect core pharmacokinetic properties relevant to drug development, including\nmetabolism (HLM, RLM), plasma protein binding (hPPB, rPPB), membrane permeability (MDR1-MDCK ER),\nand aqueous solubility. Unlike MoleculeNet tasks, these ADME endpoints are measured in realistic laboratory\nor clinical settings, often featuring complex structure-activity relationships and significant data heterogeneity.\nThey serve as practical testbeds for evaluating a model's ability to generalize to pharmacologically meaningful\nand commercially relevant tasks.”\nSection I.3 (Comparative analysis and discussion): “We report the performance of MotiL and four baseline\nmethods (CMPNN, MGSSL, MolCLR, and KANO) on the six ADME prediction tasks, with results summarized\nin Table 5. All metrics are averaged over three independent runs, and standard deviations are reported to assess\nstability. All baseline models are implemented using publicly available code with their default hyperparameter\nconfigurations. The results demonstrate that MotiL achieves highly stable and competitive performance. While\nKANO achieves relatively good results on certain tasks, its results often come with larger variances (e.g.,\n±0.0225 for HLM and ±0.0588 for Solubility), suggesting potential inconsistency. In contrast, MotiL exhibits\nlow standard deviations across most tasks (e.g., ±0.0054 for HLM and ±0.0047 for Solubility), indicating\nsuperior robustness. Although the absolute performance gap between MotiL and top-scoring models may\nappear marginal on some tasks, MotiL consistently ranks among the best-performing methods with minimal\nfluctuation, highlighting its strong generalization ability. Statistical analysis further confirms that MotiL\nsignificantly outperforms the second-best method (e.g., KANO on the task of hPPB) on each task (p < 0.05),\ndemonstrating its clear superiority. Moreover, MotiL’s ability to handle diverse pharmacokinetic properties\nwithin a unified framework, without the need for task-specific tuning, emphasizes its practical utility for real-\nworld applications in drug discovery.”\nSection I.4 (Broader impact and positioning): “By including the evaluation on real-world datasets and\nexplicitly acknowledging the known limitations of earlier benchmarks, we aim to position MotiL as a bridge\nbetween foundational research in molecular representation learning and its applications in drug discovery\npipelines. Our framework supports both small molecule and macromolecular tasks within a unified architecture\nand demonstrates improvements in consistency, interpretability, and practical relevance. We hope these\nadditional experiments and clarifications provide greater context for our design choices and reinforce the\nbroader significance of our contributions.”\nWe have also added the following experimental conclusion into the Section of “MotiL boosts the\nperformance of molecular property prediction” in the main text:\n“Our experiments on absorption, distribution, metabolism, and excretion (ADME) prediction tasks\n(Supplementary Table 5) show that MotiL consistently delivers robust and competitive performance, achieving\nstrong predictive accuracy and lower variance compared to baseline models. These results highlight the\npotential of MotiL for pharmacologically relevant applications.”\nThe corresponding statistical analysis on the MoleculeNet benchmark dataset has been included in the\nrevised manuscript under Supplementary Section D (Complete performance comparison on small molecules),\nas presented below:\n“To assess whether the observed performance gains of MotiL over the strongest baseline (KANO) are\nstatistically significant, we performed paired two-tailed t-tests across all MoleculeNet benchmark datasets\nusing the results from three independent runs. The corresponding p-values are summarized in Table 3 and Table\n4. All values are below the 0.05 threshold, confirming that MotiL’s improvements are statistically significant\nand robust across different datasets.”\nComment 2: L45 (+L89) I'd rephrase the use of the word \"generate\" here to avoid conflation with generative\nmodelling.\nReply:\nWe thank the reviewer for highlighting the potential ambiguity caused by our use of the word “generate” in the\ncontext of molecular and protein representation learning (Lines 45 and 89). We agree that the term may inadvertently\nsuggest a connection to generative modeling, which is not the intended meaning in our work.\nTo avoid this conflation and to align with standard terminology in the field, we have revised the manuscript to\nreplace “generate” with “produce” where appropriate. Specifically, we made the following changes:\n1. Line 45: “Contrastive learning methods [11, 12] typically produce two augmented versions of a molecular\ngraph through techniques like subgraph removal or perturbation, and then use them to form positive and\nnegative sample pairs.”\n2. Line 89: “At the graph level, MotiL produces similar graph representations for small molecules sharing the\nsame scaffold (e.g., naphthalene and barbituric acid) or for proteins exhibiting similar three-dimensional\nstructures and overlapping chemical functions (e.g., protein kinase activity and tRNA binding).”\nIn addition, we proactively revised several other instances of the term “generate” to “produce” to ensure\nconsistent and unambiguous language throughout the manuscript. Some examples are as follows:\n1. Line 49: “It is crucial for molecular representation learning to produce representations that accurately reflect\nthe underlying chemical structures and functions.”\n2. Caption of Figure 1: “The pretrained GNN encoders produce representations for unseen molecules, which are\nthen fed into a multilayer perceptron classifier to learn the relationship between these graph representations\nand the true molecular property labels.”\nComment 3: The authors state current methods struggle to effectively capture representations of local motifs. Is\nthere evidence to support this claim?\nReply:\nWe thank the reviewer for raising this important question regarding our claim that current methods struggle to\neffectively capture representations of local motifs. Our motivation for this claim is based on both observations from\nprior studies and experimental evidence in our work:\n1. Prior work acknowledges limitations in capturing local motifs: Several recent studies have pointed out that\nmany GNN-based molecular representation models, particularly those relying on global pooling or contrastive\nstrategies over entire graphs, tend to underemphasize fine-grained local substructures. For instance:\n(1) GROVER [1] primarily uses contrastive objectives over whole-molecule views, without explicit\nmodeling of localized functional groups or motifs.\n(2) MolCLR [2] applies contrastive learning on augmented molecular views but does not explicitly model or\npreserve chemically meaningful local motifs.\n(3) KANO [3], while improving representation through domain knowledge, still does not include an explicit\nmechanism for local motif identification or encoding.\nThese approaches often rely on the model to implicitly learn local patterns, which may be insufficient for\ntasks where specific substructures (e.g., pharmacophores, binding pockets) are strongly tied to functional\noutcomes. To address the reviewer’s concern, we have revised the manuscript to more carefully justify this\nstatement (the current methods struggle to effectively capture representations of local motifs) and added\nnecessary citations:\n“However, existing methods struggle to effectively capture the representations of motifs - such as\nfunctional groups in small molecules or amino acid residues in proteins - due to their limited ability to model\nlocalized chemical semantics or hierarchical substructures. For instance, prior contrastive learning methods\n[11, 12, 27] often rely on random subgraph sampling or global-level supervision, which may overlook\npharmacologically or structurally meaningful motifs.”\n2. Evidence from our experiments: To provide empirical evidence supporting our claim that current methods\nstruggle to effectively capture representations of local motifs, we conducted activity cliff experiments as part\nof our evaluation. Activity cliffs refer to pairs of molecules that are highly similar in overall structure (global\nscaffold), yet exhibit large differences in biological activity due to small variations in local motifs. This\nexperimental setup provides a direct way to test whether a model can capture the impact of subtle local changes\non molecular properties. We designed a dedicated activity cliff evaluation protocol as follows:\n(1) We first identified molecular pairs from the BBBP and SIDER datasets that exhibit high structural\nsimilarity (Tanimoto similarity > 0.85) but substantial differences in biological activity labels.\n(2) We then extracted the individual molecules from these pairs to construct the activity cliff test sets,\nfocusing on cases where subtle local differences drive large activity changes.\n(3) Finally, for each dataset, we analyzed the prediction disagreements between KANO and MotiL on these\nchallenging examples to assess their ability to capture local motif variations.\nAs shown in Figure 1, we found that KANO, the best-performing baseline in our main results, exhibited\nsubstantial performance degradation on activity cliff molecule pairs. It indicates that KANO fails to capture\nthe subtle yet critical differences arising from local motifs. These experimental results provide concrete\nevidence that current methods, exemplified by KANO, have limitations in learning local motif representations.\nOn the BBBP dataset, among 15 activity cliff disagreement cases, MotiL correctly predicted 10 pairs (66.7%),\nwhile KANO only predicted 5 correctly (33.3%). On the SIDER dataset, among 19 disagreement cases, MotiL\ncorrectly predicted 11 pairs (57.9%), compared to 8 correct predictions (42.1%) by KANO.\nMoreover, Figures 2 and 3 provide concrete examples of activity cliff pairs. These examples include pairs\nwith Tanimoto similarity between 0.85 and 1.0, where only minor differences in functional groups exist. In\nthese examples, KANO fails to distinguish the structurally similar molecules with distinct activities,\nwhereas MotiL successfully captures subtle motif-level differences in most cases. In the BBBP dataset (Figure\n2), we observe that in the first three molecular pairs, MotiL successfully predicts the correct labels for both\nmolecules in each pair, while KANO misclassifies one molecule per pair. These cases highlight MotiL’s\nstronger sensitivity to local functional group variations, such as substitutions between carboxyl and ester groups,\nor subtle stereochemical differences on steroid-like scaffolds (e.g., a pair with a Tanimoto similarity of 0.851).\nIn contrast, in the last two pairs, both models incorrectly classify one molecule per pair, indicating that certain\nfine-grained structural changes still present significant challenges, even for MotiL, despite its overall superior\nperformance on activity cliff prediction. In the SIDER dataset (Figure 3), several activity cliff pairs with high\nstructural similarity (e.g., 0.956 and 1.0) but opposite toxicity labels are presented. In the first pair (similarity\n= 0.956), MotiL correctly distinguishes between the two molecules, while KANO produces identical predictions,\nfailing to capture the subtle discriminative features. However, in the remaining three pairs, both models exhibit\nmixed outcomes: for each pair, one molecule is misclassified by KANO and the other by MotiL. These\nexamples suggest that both models still face difficulties in fully resolving the subtle chemical cues that\ndefine the activity cliff boundary in complex toxicity scenarios. These results highlight that MotiL is more\nrobust to subtle substructural changes than KANO, which are often the root cause of activity cliffs. We\nattribute this improvement to MotiL’s motif-level representation learning framework and the diffusion-based\ndenoising mechanism, which enhance the model’s ability to focus on local structural variations that drive\nbiological activity.\nPrediction statistics on activity cliff samples\nFigure 1: Statistics on disagreements between KANO and MotiL on activity cliff samples.\nTanimoto\nsmilarity\nLabel: 1 Label: 0\n1.0 KANO: ✓ KANO: ✗\nMotiL: ✓ MotiL: ✓\nC1=CC=CC(=C1C(O)=O)OC(C)=O CC(=O)Oc1ccccc1C(O)=O\nLabel: 1 Label: 0\n0.851 KANO: ✓ KANO: ✗\nMotiL: ✓ MotiL: ✓\nCCC(=O)C1(O)CCC2C3CCC4=CC(=O)CCC4(C)C3C(O)CC21C C[C@]12CCC(=O)C=C1CC[C@H]3[C@@H]4CC[C@]\n(O)(C(=O)CO)[C@@]4(C)C[C@H](O)[C@H]23\nLabel: 1 Label: 0\n1.0 KANO: ✓ KANO: ✗\nMotiL: ✓ MotiL: ✓\n[C@H](CC1=CC(=C(C=C1)O)O)(C(O)=O)N N[C@@H](Cc1ccc(O)c(O)c1)C(O)=O\nLabel: 1 Label: 0\n1.0 KANO: ✓ KANO: ✗\nMotiL: ✓ MotiL: ✗\nCN1C2CCC1CC(C2)OC(=O)[C@H](CO)c3ccccc3 CN1C2CCC1CC(C2)OC(=O)C(CO)c3ccccc3\nLabel: 1 Label: 0\n0.972 KANO: ✓ KANO: ✗\nMotiL: ✗ MotiL: ✓\n[Cl].CC(C)NCC(O)COc1cccc2ccccc12 CC(C)NCC(O)COc1cccc2ccccc12\nFigure 2: Prediction results for activity cliff samples on BBBP.\nTanimoto\nsmilarity\nLabel: 1 Label: 0\n0.956 KANO: ✓ KANO: ✗\nMotiL: ✓ MotiL: ✓\nCC(C)C[C@@H](C(=O)N[C@@H](CCCN=C(N)N)C(= CC(C)CC(C(=O)NC(CCCN=C(N)N)C(=O)N1CCCC1C(=\nO)N1CCC[C@H]1C(=O)NCC(=O)N)NC(=O)[C@@H]( O)NCC(=O)N)NC(=O)CNC(=O)C(CC2=CC=C(C=C2)O)\nCC2=CNC3=CC=CC=C32)NC(=O)[C@H](CC4=CC=C(C NC(=O)C(CO)NC(=O)C(CC3=CNC4=CC=CC=C43)NC(=\n=C4)O)NC(=O)[C@H](CO)NC(=O)[C@H](CC5=CNC6 O)C(CC5=CN=CN5)NC(=O)C6CCC(=O)N6\n=CC=CC=C65)NC(=O)[C@H](CC7=CN=CN7)NC(=O)[\nC@@H]8CCC(=O)N8\nLabel: 1 Label: 0\n1.0 KANO: ✓ KANO: ✗\nC(CNCCNCCNCCN)N MotiL: ✗ C(CNCCNCCN)N MotiL: ✓\nLabel: 1 Label: 0\n0.860 KANO: ✓ KANO: ✗\nMotiL: ✗ MotiL: ✓\nCCCN1CCCCC1C(=O)NC2=C(C=CC=C2C)C CCCCN1CCCCC1C(=O)NC2=C(C=CC=C2C)C\nLabel: 1 Label: 0\n0.883 KANO: ✓ KANO: ✗\nMotiL: ✗ MotiL: ✓\nCC(=O)N(CC(CO)O)C1=C(C(=C(C(=C1I)C(=O)NCC(CO CC(=O)N(CC(CO)O)C1=C(C(=C(C(=C1I)C(=O)NCC(CO\n)O)I)C(=O)NCCO)I )O)I)C(=O)NCC(CO)O)I\nFigure 3: Prediction results for activity cliff samples on SIDER.\n[1] Rong, Y. et al. Self-supervised graph transformer on large-scale molecular data. Advances in Neural Information\nProcessing Systems 33, 12559–12571 (2020).\n[2] Wang, Y., Wang, J., Cao, Z. & Farimani, A. B. Molecular contrastive learning of representations via graph neural\nnetworks. Nature Machine Intelligence 4, 279–287 (2022).\n[3] Fang, Y. et al. Knowledge graph-enhanced molecular contrastive learning with functional prompt. Nature\nMachine Intelligence 5, 542–553 (2023).\nComment 4: Contrastive learning is described twice in the introduction. I would recommend tightening the prose\nhere a little.\nReply:\nWe thank the reviewer for pointing out the redundancy in our introduction regarding the description of contrastive\nlearning. We agree that the prose can be made more concise and have accordingly revised the relevant section to\nimprove clarity and reduce repetition.\nOriginally, contrastive learning was described both when introducing the main categories of molecular\nrepresentation learning methods and again when discussing its impact on motif representation. To improve clarity\nand reduce redundancy, we have refined the exposition. The third paragraph in the Section of Introduction now\nconcisely introduces the contrastive learning mechanism, while the subsequent discussion focuses exclusively on its\nlimitations, specifically its challenges in capturing the representations of chemically meaningful motifs, without\nreiterating the methodology. This restructuring helps streamline the introduction and emphasizes the motivation\nbehind our proposed approach.\nThe revised portion in the Section of Introduction of the main text now reads:\n“Existing deep learning-based molecular representation learning methods can be divided into two categories:\ncontrastive learning and predictive learning. Contrastive learning methods [11, 12] typically produce two\naugmented versions of a molecular graph through techniques like subgraph removal or perturbation, and then use\nthem to form positive and negative sample pairs. While these methods can distinguish chemically different molecules,\nthey may inadvertently disrupt chemically meaningful substructures (e.g., functional groups or residues),\nthereby weakening the preservation of critical motif information. Predictive learning methods [27–30], in contrast,\naim to learn molecular representations by predicting masked subgraphs [27, 28], contexts [29], or attributes [30].\nBeyond these graph-based methods, recent methods based on SMILES or sequences have also shown promise\nby leveraging larger unlabeled corpora (e.g., ChemBERTa [31], SMILES-BERT [32], ESM [33]). While these\nmethods benefit from scalability, they may sacrifice finer chemical granularity, such as stereochemistry or bonding\ntopology, which graph-based methods aim to preserve.\nIt is crucial for molecular representation learning to produce representations that accurately reflect the\nunderlying chemical structures and functions. However, existing methods struggle to effectively capture the\nrepresentations of motifs, such as functional groups in small molecules or amino acid residues in proteins, due to\ntheir limited ability to model localized chemical semantics or hierarchical substructures. For instance, prior\ncontrastive learning methods [11, 12, 27] often rely on random subgraph sampling or global-level supervision,\nwhich may overlook pharmacologically or structurally meaningful motifs. Similarly, predictive molecular\nrepresentation learning, which involves masking subgraphs, contexts, or attributes, also compromises the integrity\nof motif information in the original molecule, further hindering the preservation of critical chemical properties.”\nComment 5: The introduction implicitly assumes structure-based GNN encoders are the only line of research here\nby not considering alternate representation such as sequence or SMILES that do not require GNNs. These methods\nalso often enable access to much larger quantities of data for training and pre-training which is a trade-off that should\nbe mentioned.\nReply:\nWe thank the reviewer for this insightful comment regarding alternative molecular representations such as SMILES\nstrings and protein sequences. We agree that these representations constitute an important line of research and offer\npractical advantages, especially in terms of data availability and scalability. In response, we would like to make the\nfollowing clarifications:\n1. On the choice of molecular graphs vs. sequences or SMILES: We acknowledge that SMILES- and sequence-\nbased approaches (e.g., ChemBERTa [1], SMILES-BERT [2], ESM [3]) enable access to massive corpora of\nmolecular or protein data and have shown promising performance in various domains. However, our focus on\ngraph-based representations is motivated by their closer alignment with the underlying physical and chemical\nstructure of molecules. Molecular graphs preserve explicit atom- and bond-level information, spatial topology,\nand connectivity constraints, which are properties often distorted or omitted in linear representations. In this\nsense, graph representations can be viewed as a more complete and chemically faithful form of sequence or\nSMILES data, which makes them particularly suitable for tasks that depend on local substructure integrity, such\nas motif learning or binding affinity prediction.\n2. Trade-offs between data accessibility and structural fidelity: We agree with the reviewer that there exists a\nfundamental trade-off between data scale and structural richness. SMILES and sequence formats are often\neasier to collect in large quantities from public databases, and thus allow for training models at scale. Graph-\nbased models, while more expressive in encoding chemistry, often suffer from more limited data resources due\nto the need for preprocessed structural annotations. We have now explicitly acknowledged this trade-off in the\nrevised introduction to present a more balanced view of current representation learning paradigms. While\nsequence- and SMILES-based models benefit from greater data availability and computational efficiency,\ngraph-based approaches retain richer chemical semantics by preserving the topological and bonding structures\nof molecules. In this work, we focus on graph representations as they offer a more direct and faithful encoding\nof molecular motifs, which are often lost in linearized formats.\n3. Addition of discussion in both the main text and supplementary material: To better address the reviewer’s\npoint and improve the accessibility of our paper for broader audiences, we have made two specific additions:\nIn the section of Introduction, we now briefly mention representative works based on SMILES or sequences\n(e.g., ChemBERTa [1], SMILES-BERT [2], ESM [3]), highlighting their scalability and complementary role\nin molecular representation learning:\n“Beyond these graph-based methods, recent methods based on SMILES or sequences have also shown\npromise by leveraging larger unlabeled corpora (e.g., ChemBERTa [31], SMILES-BERT [32], ESM [33]).\nWhile these methods benefit from scalability, they may sacrifice finer chemical granularity, such as\nstereochemistry or bonding topology, which graph-based methods aim to preserve.”\nFurthermore, we have added a dedicated subsection in the appendix (i.e., Supplementary Section A.4,\nSequence-based and SMILES-based molecular representation learning) that summarizes prominent SMILES-\nand sequence-based approaches. This section discusses their relative advantages, limitations, and how they\ncompare to graph-based models in terms of expressiveness, data availability, and applicability to motif-level\nlearning:\n“In addition to graph-based representations, molecular property prediction has also benefited from linear\nrepresentations such as SMILES strings and amino acid sequences. These formats treat molecules and proteins\nas sequences, enabling the application of natural language processing (NLP) techniques to molecular modeling.\nFor instance, models like ChemBERTa [46] and SMILES-BERT [47] leverage transformer-based architectures\nto learn contextual embeddings from large-scale unlabeled SMILES corpora. Similarly, the models such as\nESM [48] utilize masked language modeling on millions of amino acid sequences to capture biological\nsemantics. A key advantage of sequence-based or SMILES -based approaches is their scalability: the simplicity\nof SMILES and FASTA formats allows access to massive datasets and efficient pretraining pipelines. However,\nthis scalability comes with trade-offs. Unlike molecular graphs, which naturally encode topological and\nstereochemical information, SMILES strings or amino acid sequences often linearize or obscure such structural\nnuances. Therefore, while sequence models offer practical benefits in terms of data volume and pretraining\nefficiency, they may struggle to preserve fine-grained spatial or functional substructures critical to downstream\nmolecular tasks. Our work focuses on graph-based models as they offer a more faithful structural\nrepresentation, particularly advantageous for capturing and aligning motifs in molecular graphs.”\nTogether, these additions clarify the landscape of molecular representation learning and provide context\nfor our graph-based choice, while acknowledging the strengths of alternative input modalities.\n[1] Chithrananda, S., Grand, G. & Ramsundar, B. ChemBERTa: Large-Scale Self-Supervised Pretraining for\nMolecular Property Prediction. NeurIPS Workshop on Machine Learning for Molecules (2020).\n[2] Wang, S., Guo, Y., Wang, Y., Sun, H. & Huang, J. SMILES-BERT: Large Scale Unsupervised Pre-Training for\nMolecular Property Prediction. Proceedings of the 10th ACM International Conference on Bioinformatics,\nComputational Biology and Health Informatics 429–436 (2019).\n[3] Rives, A. et al. Biological structure and function emerge from scaling unsupervised learning to 250 million\nprotein sequences. Proceedings of the National Academy of Sciences 118, e2016239118 (2021).\nComment 6: L136 \"Gaussian noise into 𝒢 in a controlled manner.\" This is a shallow description. The authors\nshould describe that the adjacency matrix is noised.\nReply:\nWe thank the reviewer for this valuable suggestion. Based on your suggestions, we have revised the text in the\nSection of Methods to explicitly state that the noise is added to the adjacency matrix of the molecular graph. The\nrevised sentence now reads:\n“In the noise addition process, Gaussian noise is introduced directly into the adjacency matrix of the molecular\ngraph 𝒢, producing a perturbed adjacency matrix 𝑨 at each time step t.”\n!\nMeanwhile, we have revised the content in the Section of Methods as follows:\n“The generative diffusion model DiffMoM progressively perturbs the adjacency matrix 𝑨 of the molecular\n!\ngraph 𝒢 by adding Gaussian noise, where the level of noise is controlled through a predefined diffusion schedule.”\nThis clarification more accurately reflects the operation performed in the diffusion process and improves the\ntechnical precision of our description.\nComment 7: L148. Grammar. \"the local feature\" -> the local features.\nReply:\nWe thank the reviewer for their attentive reading and constructive feedback regarding the grammatical issue in Line\n148. We have revised the phrase “we integrate the local feature in a molecule...” to “we integrate the local features\nin a molecule...” to correct the singular/plural mismatch. In addition, we carefully checked the manuscript for other\ninaccurate expressions and made further refinements to improve clarity. For example, since Figure 2 presents\nvisualization results of multiple methods, not only MotiL, we have revised the caption of Figure 2a from “T-SNE\nvisualization of the molecular representations learned by MotiL.” to “T-SNE visualization of the molecular\nrepresentations learned by MotiL and other methods.”.\nComment 8: L149. Typo \"encodes\" -> encoders\nReply:\nWe appreciate the reviewer’s careful attention to detail. We have corrected “two GNN encodes” to “two GNN\nencoders” in the revised manuscript. In addition, we have thoroughly reviewed the entire manuscript to identify and\ncorrect any typographical errors, ensuring consistency and clarity throughout the paper. In addition, we carefully re-\nexamined the main text and the supplementary material to identify and correct other typographical and formatting\nissues, ensuring consistency and clarity throughout the paper. For example, in the supplementary material (Line 119)\nof the originally submitted manuscript, we corrected a missing space in the sentence: “Lipophilicity[59] is extracted”\n-> “Lipophilicity [59] is extracted.”\nComment 9: L245-250 this is quite repetitive of what has been previously stated.\nReply:\nWe thank the reviewer for pointing out the redundancy in Lines 245–250. Upon review, we agree that this portion\nof the manuscript reiterates points already discussed in the Introduction and Methodology sections, particularly the\nlimitations of augmentation-based contrastive learning and the motivation behind our motif-preserving strategy.\nTo address this concern, we have revised the paragraph to reduce repetition while preserving its function as a\ntransition to the performance evaluation. The updated version now concisely summarizes the contrast between\nbaseline approaches and our method, emphasizing key insights without duplicating previous content. The revised\ntext reads:\n“The augmentation-based methods like MolCLR, CMPNN, and KANO outperform supervised and predictive\nself-supervised baselines, but they often overlook the negative impact of graph augmentation on motif integrity. In\ncontrast, MotiL avoids aggressive augmentation, integrates a generative diffusion model to enhance robustness, and\nachieves consistently superior performance across benchmarks. These results highlight the effectiveness of motif-\npreserving representation learning.”\nComment 10: In Table 2 the authors are comparing to the variants of GearNet without pre-training. This seems like\nan unfair comparison. Additionally, the authors should denote the source of the results for the baseline methods if\nthey did not run them themselves.\nReply:\nWe thank the reviewer for raising this important concern regarding the fairness and clarity of the comparisons\npresented in Table 2.\n1. On comparing MotiL with non-pretrained GearNet variants: We agree with the reviewer that differences\nin pretraining can influence the validity of performance comparisons. In our case, although MotiL employs a\npretraining phase, it is important to clarify that no external protein data were introduced during this phase. As\ndescribed in the Section of Datasets in the main text, “We use 1D sequential orders and 3D geometric\ncoordinates of amino acids from the protein datasets for MotiL pretraining, and incorporate the label\ninformation of GO terms or EC numbers during the task-specific fine-tuning phase.” That is, all pretraining for\nMotiL was performed strictly using the proteins in the benchmark datasets themselves, without incorporating\nadditional protein sequences or structures from external sources. In contrast, GearNet’s pretraining variant\nexplicitly incorporates a large external protein structure corpus for pretraining. As stated in Section 5.1 of the\nGearNet paper [1]: “We use the AlphaFold protein structure database (CC-BY 4.0 License) (Varadi et al., 2021)\nfor pretraining.” This use of an extensive external protein structure database distinguishes GearNet’s\npretraining setup from ours in both scope and data resources. In Table 2 of our manuscript, we chose to report\nthe non-pretrained versions of GearNet and its variants (e.g., GearNet-Edge, GearNet-IEConv, GearNet-Edge-\nIEConv), following their original evaluation protocols without pretraining. These results were drawn directly\nfrom Ref. [24], and as the original paper does not report performance for the same variants with pretraining, we\ndid not include pretrained counterparts to avoid inconsistencies. We now explicitly clarify this setting in the\nrevised manuscript and have added a note to the Table 2 caption indicating this difference: “The pretraining\nphase of MotiL does not incorporate extra protein data, which is consistent with baselines such as GearNet.”\n2. On denoting the source of baseline results: We also appreciate the reviewer’s suggestion to make the\nprovenance of baseline results more transparent. As indicated in the Section of Methods and now explicitly\nclarified in the caption of Table 2, the results for all baseline models (including GearNet and CDConv) are\ndirectly reported from Ref. [24]. The updated caption now reads: “All baseline results in this table are directly\nreported from Ref. [24], which follows the same evaluation protocol as this study: 10-fold cross-validation\nsplits for proteins.”\nTherefore, the updated caption for Table 2 is as follows:\n“Performance comparison on protein macromolecules. All baseline results in this table are directly reported\nfrom [24], which follows the same evaluation protocol as this study: 10-fold cross-validation splits for proteins. The\npretraining phase of MotiL does not incorporate extra protein data, which is consistent with baselines such as\nGearNet.”\n[1] Zhang, Z. et al. Protein representation learning by geometric structure pretraining. The 11th International\nConference on Learning Representations, ICLR 2023 (2023).\nComment 11: L282 EC prediction is a classification task.\nReply:\nWe sincerely thank the reviewer for pointing out this important clarification regarding the nature of the EC number\nprediction task in Line 282. You are correct that EC number prediction is a multi-label classification task, not a\nregression task. Each protein can be associated with multiple enzyme commission (EC) numbers, corresponding to\nhierarchical functional annotations based on the type of chemical reactions catalyzed. Thus, the task is best modeled\nas a classification problem, where the goal is to predict one or more correct EC labels for a given protein.\nIn our original manuscript, the sentence in Line 282 was imprecise and may have inadvertently implied that\nEC prediction was a regression task. We have corrected the sentence in the revised manuscript to more accurately\nreflect the task type. The updated version now reads:\n“It suggests that MotiL has a strong advantage in classification tasks like EC number prediction.”\nWe have also reviewed the surrounding text in Section 4.4 to ensure that other task types (e.g., ATP binding\nclassification and GO prediction) are correctly described and consistently referred to using appropriate terminology\n(i.e., classification vs. regression).\nComment 12: L309. Rephrase messy clustering.\nReply:\nWe sincerely thank the reviewer for pointing out the use of the informal phrase “messy clustering” in Line 309. To\nimprove clarity and rigor, we have revised the sentence to use more appropriate technical terminology that better\nreflects the clustering behavior observed in the representation space.\nSpecifically, we have modified the original sentence from: “No pretraining yields the worst results, with messy\nclustering on QM8 (DB index: 4.93), highlighting the necessity of pretraining representation learning for effective\nmolecular representation.” to the following revised version: “No pretraining yields the worst results, with\ndisorganized and poorly separated clusters on QM8 (DB index: 4.93), highlighting the necessity of pretraining\nrepresentation learning for effective molecular representation.”\nThis updated wording provides a clearer and more objective description of the low-quality clustering results\nobserved when pretraining is absent. We also ensure that the interpretation remains directly supported by the\nquantitative DB index value reported.\nComment 13: The authors make frequent to reference to their proposed methods robustness to noise but do not\nperform any experiments to demonstrate this. Nor do they describe the source of the noise they are referring to.\nSurely, given that SE(3) invariance is built into the model, the effect of structural noise is likely to be minimal? As\nfar as I can tell from the description, coordinates are used in to model.\nReply:\nWe sincerely thank the reviewer for this insightful and constructive comment. We understand the importance of\nproviding empirical evidence to support our claim of noise robustness and clarifying what we mean by \"noise\" in\nour study.\n1. Clarification of noise definition: In the context of our work, the “noise” refers specifically to structural\nperturbations in molecular graphs, particularly edge-level noise such as missing chemical bonds, which may\narise due to experimental uncertainty in graph construction or preprocessing artifacts [1-4]. Although our model\nleverages 3D geometric features and is designed to be SE(3)-invariant, graph-based representations (especially\nthe topological structure of the molecular graph) remain central to the encoder. Perturbations to this topology\ncan still significantly impact learned representations in most models.\n2. New experiments to quantify noise robustness: To address the reviewer’s concern, we have added a new\ncontrolled experiment. In this experiment, we simulate graph-level noise by randomly deleting 10%, 20%, and\n30% of edges from each molecular graph in the BBBP (from MoleculeNet) and Solubility (from ADME)\ndatasets. We evaluate the ROC-AUC or RMSE performance of three competitive baselines (MolCLR, MGSSL,\nKANO), our model (MotiL) and an ablation version of MotiL without diffusion priming under each noise level.\nAs shown in Figure 4, MotiL exhibits significantly smaller decreases in ROC-AUC or smaller increases in\nRMSE as noise levels rise, demonstrating its superior robustness to graph structural perturbations. This\nrobustness is attributed to the diffusion priming phase, which learns denoising-aware representations by\nmodeling and reversing a noise corruption process during pretraining.\n(a) BBBP (b) Solubility\nFigure 4: Performance of different methods under increasing graph structural noise.\nTo reflect this improvement, we have revised the manuscript by:\n(1) Clarifying our definition of noise as structural perturbations in molecular graphs.\n(2) Adding the new experiment and detailed analysis in Supplementary Section G (Robustness evaluation to\ngraph structural noise);\nThe revised sentence in the main text now reads:\n“We propose diffusion priming as the first phase of MotiL to perceive the global structure of a molecule\nand thus enhance the robustness of GNNs to structural noise perturbations in the molecular graph 𝒢 (Fig.\n1a).”\n“On the other hand, MCL without augmentation removes the reliance on molecule graph augmentation,\nbut meanwhile ignores the GNN encoder's robustness to structural noise perturbations in molecule graphs\nand thus does not yield better results than MCL with augmentation.”\n“Also, Supplementary Fig. 8 indicates that the diffusion priming phase used in MotiL effectively mitigates\nthe sensitivity of our method to structural noise perturbations.”\nThe additional content in Supplementary Section G (Robustness evaluation to graph structural noise) is\npresented as follows:\n“To evaluate the robustness of molecular representation models under structural noise, we introduce\nvarying levels of edge deletion to the molecular graphs in the BBBP and Solubility datasets. Fig. 8 shows the\nROC-AUC or RMSE results for MotiL, MotiL w/o diffusion priming, and three representative baselines\n(MolCLR, MGSSL, and KANO). MotiL exhibits significantly better resistance to noise perturbations, with a\nflatter error curve and superior RMSE across all noise levels. It demonstrates the effectiveness of the diffusion\npriming phase in enhancing the stability of learned representations under noisy graph conditions.”\n[1] Heid E, McGill C J, Vermeire F H, et al. Characterizing uncertainty in machine learning for chemistry[J]. Journal\nof Chemical Information and Modeling, 2023, 63(13): 4012-4029.\n[2] Ryu S, Kwon Y, Kim W Y. A Bayesian graph convolutional network for reliable prediction of molecular\nproperties with uncertainty quantification[J]. Chemical science, 2019, 10(36): 8438-8446.\n[3] Crusius D, Cipcigan F, Biggin P C. Are we fitting data or noise? analysing the predictive power of commonly\nused datasets in drug-, materials-, and molecular-discovery[J]. Faraday Discussions, 2025"}, "author_response": ""}]

Decision letter