Multi-to-uni modal knowledge transfer pre-training for molecular representation learning

Domain: neuroscience
Published: 2026-02-14
Editorial outcome: major_revision

Abstract
The pre-training molecular representation learning (MRL) has shown considerable potential in computer-aided drug discovery. Recently, many multimodal pre-training MRL methods have been presented, incorporating multimodal molecular data for pre-training and achieving high-accuracy predictions in downstream tasks. However, most current methods require completeness of modality for molecular data in the pre-training phase and often overlook their adaptation to real-world scenarios where, for example, molecular modalities except 2D topological graphs (2D modality) are often unavailable. In this study, we propose a multimodal pre-training MRL framework called M2UMol, which separately matches 2D modality to multiple modalities and undergoes pre-training jointly with a modality classifier. In this way, M2UMol elegantly transfers multimodal knowledge into the 2D modal encoder and allows for inputting incomplete modalities in the pre-training stage. Moreover, in downstream tasks with only the 2D modality given, M2UMol enables the precise simulation of molecular multimodal information based on the pre-trained 2D modal encoder. Comprehensive experimental results show the superior performance of M2UMol in a wide range of molecular tasks with higher efficiency in pre-training than pioneer models and demonstrate the validity of the multimodal knowledge transfer. Furthermore, we developed a user-friendly package based on M2UMol, integrating molecular representation learning, key functional group analysis, molecular multimodal retrieval, etc. It may be conveniently used in diverse fields related to drug discovery and promises to facilitate the process of developing drugs. Our code, pre-trained weights of M2UMol, and the package are available at https://github.com/Zhankun-Xiong/M2UMol.

Introduction
In the last decades, artificial intelligence (AI) technologies, particularly machine learning, have got a wider range of applications in drug discovery, such as molecular property prediction [1, 2], drug-drug interaction (DDI) prediction [3, 4, 5] and drug-target interaction (DTI) prediction [6, 7], etc. Among these applications, a fundamental challenge is the featurization of molecules into numerical vectors, i.e., molecular representations. Traditional molecular representations, such as descriptor-based features [8], are handcrafted features heavily limited to human-summarized physicochemical domain knowledge. By contrast, Molecular Representation Learning (MRL) has been viewed as a promising approach, which leverages deep learning to automatically mine latent knowledge from molecules and produce superior molecular representations that significantly outperform traditional handcrafted features in important tasks of drug discovery [9, 10, 11].

The MRL methods typically consider molecular input data in various formats (modalities), including one-dimensional (1D) molecular sequences [12, 13], often represented by the Simplified Molecular-Input Line-Entry System (SMILES) [14]; two-dimensional (2D) topological graphs [15, 16] with atoms as vertices and covalent bonds as edges; and three-dimensional (3D) conformer graphs [17, 18] derived from molecule conformations, which can be encoded into molecular representations by various encoders. Commonly, many MRL methods train the encoders on specific-task labeled molecules in a supervised manner. Nevertheless, the scarcity of labeled molecular data significantly impacts the quality of molecular representations and especially limits their out-of-distribution (OOD) generalization. In recent years, there has been a surge in the development of pre-training models within the field of MRL. These MRL methods begin their training process on unlabeled molecular data through self-supervised pretext tasks. Subsequently, they undergo fine-tuning using a small set of labeled molecular data tailored to specific downstream tasks, resulting in notably enhanced performance.

The self-supervised pre-training MRL methods are broadly classified into unimodal-based methods and multimodal-based methods. Unimodal pre-training MRL methods focus on a single molecular modality, typically SMILES or graphs, and learn effective molecular representations by a range of self-supervised tasks, such as predicting specific parts of molecules [19, 20, 21, 22, 23] and contrasting molecules with their augmented counterparts [24, 25, 26]. In contrast, multimodal pre-training MRL methods enhance the quality of molecular representations by capturing complementary information from different molecular modalities. These methods can be divided into two categories. The first category can be summarized as a "one-one" paradigm, which aims to model the relationship between two modalities in pre-training to grasp multimodal information. Typical pre-training tasks in this category include aligning the representations of two molecular modalities using contrastive learning [27, 28, 29, 30] or generating a molecular modality from another [31, 32, 30]. In such frameworks, downstream fine-tuning usually relies on a single pre-trained encoder, typically a 2D molecular encoder. As a result, the single pre-trained encoder cannot fully exploit the multimodal knowledge learned in pre-training. Moreover, this category is inherently limited to two molecular modalities and provides an insufficient understanding of molecules due to the lack of knowledge from more modalities. It is challenging to design pre-training strategies involving three or more molecular modalities. The second category includes recent efforts [33, 34] which attempt to address these challenges through a "one-others" paradigm. These methods select one modality as the anchor and integrate representations of all other modalities through concatenation or summation. They then adopt contrastive learning to align the representations of the anchor modality with the integrated multimodal representations. In their fine-tuning phase, all the encoders for distinct modalities serve as pre-trained modules, and their learned representations are integrated for downstream tasks. However, these methods often necessitate that the molecules have complete modalities during both the pre-training and fine-tuning phases, resulting in two flaws. On the one hand, they fail to exploit larger-scale molecules with missing modalities for more sufficient pre-training. On the other hand, they are not competent in practical downstream applications, where all modalities are often unavailable except molecular 2D topological graphs (usually converted from SMILES). For more clearly distinguishing the existing multimodal pre-training MRL methods, we summarize their characteristics in Supplementary Table S2, including some details such as modality usage and pre-training strategy.

In this study, we propose a multi-to-uni modal knowledge transfer pre-training MRL method, termed M2UMol. M2UMol first learns 2D molecular representations and multimodal representations through corresponding encoders, and the modal-specific adapters are designed to generate pseudo multimodal representations based on 2D molecular representations. Then M2UMol undergoes pre-training using the two pretext tasks: generated-actual multimodal contrastive learning and modality classification. The former aims at alignment of the generated pseudo multimodal representations with the corresponding actual multimodal representations in three independent vector spaces, guiding the transfer of multimodal knowledge to the 2D modal encoder and enabling the modal-specific adapters to generate more reliable multimodal representations; and the latter attempts to classify the generated representations into three types of actual modalities, guiding the adapters capturing modal-specific knowledge and generating higher-distinguishability representations. In the subsequent fine-tuning phase, the pre-trained 2D encoder and modal-specific adapters are utilized to learn 2D representations and generate 3D/Text/Bio representations, respectively, and these representations are adaptively integrated by a multi-head attention mechanism to obtain the final molecular representations for various downstream tasks.

Comprehensive experimental results validate the favorable performances of M2UMol on both molecular property prediction tasks and molecular interaction prediction tasks, especially under a harsh evaluation scenario of the scaffold split. In addition, compared to models pre-trained on millions of molecules, M2UMol is only pre-trained on 11k molecules with significantly less computational cost and shorter pre-training time (See Supplementary Table S3). It demonstrates the high scalability and efficiency of our designed pre-training strategies. Moreover, we conducted extensive experiments to evaluate the quality of generated multimodal representations and the benefits of multimodal pre-training, and verified that M2UMol can learn discriminative and uniform molecular representations for downstream tasks and provide accurate and interpretable predictions. We also developed a user-friendly package based on M2UMol, which integrates multiple functions including molecular representation learning, multimodal data retrieval, and key structure identification. The application of the M2UMol package can be found in Section 4 of the Supplementary Information.

The contributions of this paper include the following:
• We propose M2UMol, a multi-to-uni modal knowledge transfer pre-training MRL method, which is tailored to two practical application challenges: incomplete modalities in pre-training and unimodal downstream tasks.
• The modal-specific adapters are designed to receive the multimodal knowledge in the pre-training stage and generate pseudo multimodal representations solely from 2D molecular representations for supplementing multimodal information.
• Through the two well-designed multimodal self-supervised tasks, we achieve the guidance of multi-to-uni modal knowledge transfer and empower the generated multimodal representations to be more reliable and keep modal-specific information.
• Comprehensive experimental results show the superior performances of M2UMol in a wide range of molecular tasks with high efficiency in pre-training, and verify that M2UMol can learn high-quality molecular representations and provide interpretable predictions. Our codes, pre-trained weights, and package are open-source and can be conveniently used in diverse related fields in AI-assisted drug discovery.

Results
Overview of M2UMol
The proposed M2UMol is shown in Figure 1. In this paper, we constructed a multimodal molecular dataset, which includes 11,571 drug-like molecules with four modalities: 2D topological graphs (2D), 3D conformer graphs (3D), textual descriptions (Text), and biochemical features (Bio) (Figure 1a), among which biochemical features of molecules that are believed to contain rich domain knowledge, can be considered as an important modality of data, but usually ignored in previous works. M2UMol encompasses four modal encoders and the modal-specific adapters and undergoes pre-training on the constructed dataset using two innovative self-supervised pretext tasks: generated-actual multimodal contrastive learning and modality classification (Figure 1b). Then, the pre-trained 2D encoder and the modal-specific adapters, as well as a learnable multi-head attention module, compose our plug-in fine-tuning architecture, which outputs molecular representations for various downstream tasks with only 2D modality (Figure 1c).

It is worth noting that the generated-actual multimodal contrastive learning contains three contrastive learning objectives that aim to separately align the pseudo representations for each modality from {3D, Text and Bio} with the corresponding actual representations. Hence, the molecules with both 2D graph structures and at least one of the three modalities can be used for our pre-training, without requiring complete modalities for molecules. Taking our pre-training dataset as an example, this capability enables M2UMol to scale pre-training from 4,325 molecules (the black-bordered area in the Venn diagram of Figure 1a) with complete modalities to 11,571 molecules (the area bordered by the red line), by incorporating molecules with incomplete modalities. This leads to approximately a threefold increase in pre-training data, which significantly enhances the performance of M2UMol (see Supplementary Figure S3).

M2UMol outperforms the baseline methods on molecular property prediction and molecular interaction prediction tasks
In this section, we assess the effectiveness of our proposed method M2UMol by comparing it with baseline methods across two types of molecular downstream tasks, including molecular property prediction (MPP) and molecular interaction prediction (MIP). The details about fine-tuning on these downstream tasks can be found in Section 2.3 of the Supplementary Information.

Molecular property prediction (MPP). Molecular property prediction is a crucial task in drug discovery, beneficial to scientists for screening out molecules with desired properties. This field has been rapidly growing in AI for Science, with a significant number of methods developed, including pre-training MRL methods. We follow most existing methods to evaluate our method M2UMol on scaffold-based splits. A scaffold split is a challenging and realistic evaluation setting, commonly used to assess the OOD generalization capabilities of MRL methods. Here, we compare M2UMol with recent strong pre-training MRL methods for property prediction. The results are shown in Table 1.

A general observation from Table 1 is that multimodal pre-training MRL methods tend to produce better performance than unimodal methods. The performance enhancement is particularly noteworthy when more modalities are incorporated in the pre-training phase, as exemplified by our method M2UMol. Such a trend suggests that the use of more modalities for pre-training may have more potential benefits for downstream molecular property prediction tasks. Among the baselines, M2UMol produces the best performances on 5 out of 8 datasets with average improvements of 5.21%, 10.00%, 10.69%, 5.57%, and 8.79% on AUC, respectively. We attribute the favorable results of M2UMol on unimodal molecular property prediction tasks to our design of transferring multimodal knowledge into 2D molecular representation learning in the pre-training phase and simulating the multimodal information fusion solely based on the 2D modality in the fine-tuning phase. Additionally, as summarized in Supplementary Table S3, M2UMol is pre-trained on only 11k molecules using a single GPU (RTX 3090) within 11 hours, yet still achieves superior predictive performance. This demonstrates the high scalability and efficiency of our model. More results about performance comparison can be found in Section 3.1 of the Supplementary Information.

To sum up, our proposed method, M2UMol, shows promise as an effective tool for predicting molecular properties for those molecules with only 2D modal data available.

Molecular interaction prediction (MIP). Molecular interactions can provide significant information for sophisticated biological system analysis, which is a powerful resource for drug discovery. Herein, we choose two representative molecular interaction prediction tasks: the DDI prediction task and the DTI prediction task, to further evaluate the performance of M2UMol.

DDIs can arise when multiple drugs are taken simultaneously, leading to unexpected adverse effects, known as DDI events (DDIEs). Predicting DDIEs, typically considered as a multi-classification task within DDI prediction, is crucial for public health security and medicine safety surveillance, garnering much attention in the deep learning and bioinformatics communities. For this task, we focus predominantly on quite challenging scenarios: the cold start split and the scaffold split. For the former scenario, the drugs used for testing are not included in the DDI data on which our method is fine-tuned, while for the latter scenario, not only the testing drugs but also their scaffolds are unseen in the fine-tuning phase. Note that the second scenario is more intractable than the first one, but has been little concerned in previous works. We choose recent advanced DDIE prediction methods, which can be applied to both scenarios, as baselines. The results are shown in Figure 2a-b. Overall, M2UMol significantly outperforms all baselines under both scenarios. Specifically, compared with the baselines, M2UMol achieves average performance gains of 9.66% and 6.84% in ACC, 11.92% and 7.74% in Macro-F1, 12.03% and 7.47% in Macro-R, as well as 12.27% and 10.43% in Macro-P, respectively, in the cold start split and scaffold split scenarios.

DTI prediction is an integral step in drug discovery, which helps to narrow down the search scope of compound candidates. We replace the drug representation learning module in DrugBAN [57], a milestone method for predicting DTIs, with our fine-tuning architecture to adapt M2UMol to DTI prediction. Then, we follow two evaluation scenarios in [57], i.e., the random split and the scaffold split, to exhibit the abilities of M2UMol in improving DTI prediction. Figure 2c-d shows the results of different methods in DTI prediction. We observe that compared with DrugBAN, the result of M2UMol is not significantly improved, while the improvement of M2UMol_unpre is extra pronounced. To be more specific, M2UMol performs better than DrugBAN with average improvements of 2.60% in AUC and 2.48% in AUPR on two datasets across two evaluation scenarios.

In summary, our proposed method, M2UMol, realizes superior performance on both molecular interaction prediction tasks, showcasing its reliability, flexibility, and robustness, which are attributed to its ability to offer high-quality representations for drug molecules. Its superiority in predicting molecular interactions underscores its extraordinary potential to promote AI-aided drug discovery.

M2UMol acquires diverse knowledge from multimodal data in pre-training
This section shows how M2UMol learns knowledge from unlabeled multimodal data during pre-training. We evaluate the quality of multimodal representations generated by M2UMol and discuss the capability of M2UMol to identify the key structures of molecules without labeled molecular data. In Section 3.2 of the Supplementary Information, we further discuss the importance of the three modalities used, explore multimodal synergy in facilitating cross-modal understanding, and analyze the impact of the five designed components (modality-specific adapters, modality classifier, and other components) on the performance of M2UMol. The results shown in Supplementary Figure S3 demonstrate that M2UMol can effectively learn multimodal knowledge, addressing the practical application challenges of incomplete modalities in pre-training and unimodal downstream tasks, and thus achieving superior performance.

M2UMol can generate reliable multimodal representations from 2D molecule graphs. For the actual representation of each molecule output from the encoder for each modality (3D/Text/Bio), the modal-specific adapters in M2UMol are to generate the matched pseudo representation solely from the 2D representation of the corresponding molecule. We visualized the actual and generated multimodal representations by t-Distributed Stochastic Neighbor Embedding (t-SNE) [58]. As shown in Figure 3a, we observe that both actual modality representations and generated modality representations are distributed separately into three clusters according to the modality types. This observation demonstrates that M2UMol can capture differentiated knowledge of different modalities, which is transferred to the adapters, such that it has the ability to generate distinguishable representations only based on 2D representations. Further, we conducted a multimodal retrieval task [59] to assess the quality of the generated multimodal representations. Recall that all actual and generated representations have identical dimensions in our setting, so we consider all the matched generated-actual representation pairs in a common Euclidean space, and we check whether the actual representation in each pair is within the top-k closest representations to the matched generated representation in the common space. We utilized the widely used metric Recall@K [59, 60, 29, 33] to evaluate the quality of generated multimodal representations in the multimodal retrieval task. We adopted Recall@K with optional k in {1, 3, 5, 10}. As shown in the bar chart in Figure 3a, we can observe that: (1) the generated 3D representations are higher-quality than the generated representations of other modalities, likely due to the narrow modality gap between 3D and 2D which both depict molecular structural information; (2) despite the wide modality gap between Text and 2D as well as smaller-scale data of Text than both 3D and Bio, the generated Text representations still approximate to their matched actual representations well (Recall@1=0.72, Recall@10=0.98), indicating our models’ ability to bridge modality gaps effectively; (3) the quality of the generated Bio representations are relatively poor while acceptable (Recall@1=0.35, Recall@10=0.76), potentially due to the sparsity of the binary biochemical feature vectors. Overall, the generated and actual representations are matched well in the pre-training phase, which demonstrates the robust capability of M2UMol in generating reliable multimodal representations only using 2D molecule graphs.

In addition, inspired by previous studies [61, 62, 24], we conducted case studies to show that M2UMol effectively captures semantic information, resulting in similar generated representations for molecules sharing similar characteristics. We selected Diflorasone (ID: DB00223) as the query molecule, and retrieved its top-4 closest molecules in the 2D representation space or in the generated 3D/Text/Bio representation space. The modality data of the retrieved molecules are visualized in Figure 3b. The top-4 closest molecules in each modality representation space exhibit characteristics akin to Diflorasone. Specifically, in the 2D modality, Diflorasone and the nearest Flumethasone are a pair of conformational isomers, which differ from the other three molecules only in a few groups, such as the substitution of F with CL and CH3 with OH. In 3D modality, Diflorasone and its neighboring molecules have highly similar atomic compositions and structures. In the Text modality, all the listed molecules are corticosteroids with anti-inflammatory effects. In Bio modality, the illustrated molecules target the Glucocorticoid Receptor and can also be metabolized by the enzymes in the CYP3 family, meanwhile sharing similar drug categories. This illustrates M2UMol’s ability to capture the intrinsic relationships between molecules. Moreover, the closest molecules vary across different modalities, which highlights M2UMol’s capability to grasp modality-specific information and generate distinct representations for each modality. This capability ensures the provision of complementary and comprehensive information for each molecule. More results about the multimodal retrieval case studies can be found in Section 3.3 of the Supplementary Information.

M2UMol enables precise focus on the key structures of molecules without labeled data. Since M2UMol uses more modal molecular data than most existing pre-trained MRL models that often include two modalities, a problem naturally arises whether using more modalities in our multi-to-uni modal knowledge transfer pre-training has more benefits for M2UMol to understand key molecular knowledge from a more comprehensive perspective. To answer this problem, we choose several molecules from the pre-training dataset and visualize the attention weights on the atoms and bonds of the molecules learned by our pre-trained M2UMol (2D+3D+Text+Bio) and its three variants pre-trained with partial modalities (2D+3D, 2D+Text, or 2D+Bio) in Figure 3c. In addition, we exhibited the interaction potential maps computationally generated from Molecular Operating Environment (MOE) (http://www.chemcomp.com/), which provide graphical representations of the regions where chemical probes (Na+, N1+, and O in this case) have favorable interactions with molecular surfaces. Based on the interaction potential maps, we can also pinpoint key active groups of the exemplified molecules serving as the reference for comparative analysis with the attention visualization.

As shown in Figure 3c, with the progressive inclusion of modalities during the pre-training phase (left to right), M2UMol gradually allocates its attention to specific molecular fragments that are highly consistent with the key active groups identified from the interaction potential maps. We here take Oxymetazoline, an adrenergic agonist, as an example: M2UMol pays much attention to the hydroxy and the imidazoline ring, which are involved in hydrogen bonding interactions and ionic interactions during the binding of Oxymetazoline to adrenergic receptors [63], and the other variants disperse their attention. The results demonstrate that M2UMol has the potential to understand molecular structure-activity relationships and identify the key structure of the molecule only based on the multimodal molecular data itself, and the more modalities it incorporates, the more precise the understanding becomes. More details about molecular key structures identification can be found in Section 3.4 of the Supplementary Information.

M2UMol improves performances on downstream tasks with high-quality representations.
In this section, we discuss how molecular representations delivered by M2UMol benefit the various downstream tasks. Specifically, we consider the testing samples (i.e., molecules for MPP task; drug-drug or -target pairs for MIP tasks) in the scaffold split scenario, and visualize their representations from M2UMol and M2UMol_unpre via t-SNE [58]. Notice that herein, ’M2UMol’ denotes the pre-trained M2UMol with fine-tuning, and ’M2UMol_unpre’ represents our fine-tuning architecture directly trained on downstream tasks with random initialization. As shown in Figure 4a, we can see that the representations learned by M2UMol for positive and negative samples in two MPP datasets, BBBP and BACE, are more clearly separated into two parts than those from M2UMol_unpre. For two DTI datasets, BindingDB and BioSNAP, M2UMol achieves a lower Davies–Bouldin (DB) index (7.1770 and 3.8673) than M2UMol_unpre. For the DDI dataset, we choose the drug-drug pairs of the sparsest 10 DDI events within the testing set for the t-SNE visualization, where it is observed that the representations learned by M2UMol are more tightly clustered compared to M2UMol_unpre. In summary, the superior performance of our M2UMol on various downstream tasks can be owed to its success in enhancing the representations of molecules with only unimodal inputs in downstream tasks by learning transferable knowledge from pre-training on multimodal molecule data.

Inspired by previous studies [64, 65], high-quality representations of molecules should be roughly uniformly distributed on the unit hypersphere, preserving as much data information as possible. Herein, we projected the representations of molecules in testing sets from the above-mentioned M2UMol and M2UMol_unpre onto a unit circle by undergoing dimensionality reduction through t-SNE, followed by l2 normalization [58]. Then, we visualize the density distributions of the representations by using non-parametric Gaussian kernel density estimation (KDE) [66] on the circle. Besides, we calculate the average pairwise Gaussian potential [64] scores, abbreviated as APGP scores, for quantifying the uniformity of representations. Additionally, we show the density estimations of angles for each point to present the distribution more clearly. As shown in Figure 4b, for M2UMol_unpre, the distributions of the representations consistently exhibit a relatively high degree of clustering, and the angle density estimation curves are sharp with distinct peaks. This may be due to the fact that without knowledge from pre-training, the representations learned by M2UMol_unpre only contain limited unimodal information, which may make it difficult to learn unique characteristics when the molecules are similar in structure. As for our pre-trained M2UMol, the distributions of molecular representations become more uniform with lower APGP scores, and the angle density estimation curves are also markedly smoother. This is because after undergoing our designed multi-to-uni knowledge transfer pre-training, the model is able to generate representations of different modalities based solely on the 2D topological graphs of the molecules, which provides more diverse information that enhances learning the unique characteristics of molecules and makes the representations more evenly distributed in the space.

M2UMol uncovers critical structures of molecules related to downstream tasks
In this section, we further conduct the interpretability analysis to investigate M2UMol’s capacity to capture task-induced key structures or functional groups after fine-tuning it on the downstream tasks. To this end, we visualize the attention weights of molecules learned by M2UMol on MPP and DDI prediction tasks.

For MMP, we select testing molecules in the scaffold split scenario from different datasets for visualization analysis, and the results are shown in Figure 5a. For example, Fluorouracil (PubChem ID: 3385) is a molecule designed for treating tumors, but without the approval of the FDA, and it is labeled as toxic in the Clintox dataset. M2UMol notices the halogen group and two amide groups of Fluorouracil. The former is the key group that causes toxicity of Fluorouracil, and for the latter, a study [67] has shown that using Fluorouracil may cause hyperammonemia, which may be related to the two amide groups. Phenylhydrazine (PubChem ID: 7516) is a molecule labeled as toxic in the Toxcast dataset. We can see that the hydrazino is assigned high attention weights by M2UMol, which is proven as the key functional group that can be oxidized by oxyhemoglobin, leading to hemolysis [68]. The Aromatic heterocycles is labeled as the inhibitors of human b-secretase 1 in BACE datasets, and M2UMol focuses on the imidazole group, which belongs to the aromatic heterocyclic family that shows an inhibitory effect on human b-secretase 1 in a study [69]. Aspirin (PubChem ID: 2244), a molecule from the BBBP datasets, has the ability to cross the blood-brain barrier. M2UMol focuses on the carboxyl and ester groups, which are acidic and lipophilic functional groups, respectively, and has been proven to help molecules cross the blood-brain barrier [70].

For DDI prediction, we choose a commonly used drug for Analgesic and anti-inflammatory in the clinic, namely Loxoprofen, a nonsteroidal anti-inflammatory drug. As shown in Figure 5b, M2UMol focuses precisely on the carboxyl in the propionic group of Loxoprofen, which is the key active group for the high effectiveness [71] and helps inhibit the synthesis of prostaglandins involved in promoting the diuretic effect and antihypertensive effect. Therefore, taking Loxoprofen together with diuretics or antihypertensive medications may potentially decrease their activity [72]. Here, we randomly choose two diuretic drugs (Furosemide and Etacrynic acid) and two hypotensive drugs (Hydralazine and Practolol) that are predicted by M2UMol to have "decrease the diuretic activities" and "decrease the antihypertensive activities" DDI events with Loxoprofen, respectively, for visualization. As shown in Figure 5b, for Furosemide, M2UMol focuses on the sulfonyl and amino, which are the key groups resulting in potent diuretic activity [73]. Besides, M2UMol also focuses on the carboxyl group of Furosemide, which has been proved to be an important active group that has considerable potential for intermolecular interaction [74, 75]. For Etacrynic acid, M2UMol focuses on the halogen group and carboxyl group. The former can interact with the targets about diuretic activity [76], and the latter is important to the pH suitability and solubility improvement [77], which can affect the absorption and effect of drugs. For Hydralazine, the focused hydrazine group is the most reactive portion of the molecule and the part most likely to spontaneously bind to proteins [78]. For Practolol, acylamino is focused, which was shown to control both the degree of agonism and the cardioselectivity [79] which is directly related to the antihypertensive effect, and the focused hydroxyl group can affect the solubility and absorption [80], which can affect the effectiveness of the drug. More results about the interpretability analysis can be found in Section 3.5 of the Supplementary Information.

Overall, M2UMol can understand molecular structures from multimodal perspectives, and thus can comprehensively capture key structures or functional groups of molecules, and the visualized results are consistent with human understanding of the structural properties of drug molecules. It confirms that M2UMol has good interpretability and can provide diversified and abundant molecular-related knowledge to assist the drug discovery process.

Methods
Multimodal molecule data construction
For the convenience of the follow-up description and discussion about our pre-training model, we here symbolize the construction of our multimodal molecular data.

2D topological graph (2D) Let M denote a set of molecules, each of which can be represented as a 2D topological graph with its atoms as nodes and bonds as edges. For a molecule m ∈ M, its corresponding 2D topological graph is denoted as G_i.

3D conformer graph (3D) Formally, the 3D conformer graph for a molecule contains its original 2D topological graph as well as Cartesian coordinates for all nodes. Let M_c denote a set of molecules with their 3D conformers available. For a molecule m ∈ M_c, its corresponding 3D conformer graph is denoted as C_i.

Textual description (Text) For the textual description modality, we extract two categories of drug information from DrugBank: (1) concise summaries containing key properties, mechanisms of action, and usage guidelines; and (2) comprehensive background information covering historical development, discovery processes, and clinical trial outcomes. These texts are concatenated to create a unified textual description T_i for each molecule m ∈ M_t, where M_t is the set of molecules with textual descriptions.

Biochemical feature (Bio) For the biochemical feature modality, we integrate three types of biochemical features from biological functions for each molecule, i.e., molecules’ drug categories as well as associations with targets and enzymes from DrugBank. Specifically, given a molecule m ∈ M_o, its biochemical features can be represented as a set of binary vectors O_i = {o_{i1}, o_{i2}, o_{i3}}, where M_o is the set of molecules with biochemical features and each element value 0 or 1 in o_{i1} ∈ {0,1}^{n1}, o_{i2} ∈ {0,1}^{n2} or o_{i3} ∈ {0,1}^{n3} indicates the presence or absence of the drug category/target/enzyme of the molecule. There are 3607 types of drug categories, 4463 types of targets, and 419 types of enzymes, so n1=3607, n2=4463, n3=419.

Note that for each molecule in M_a, its other modalities (3D, Text, and Bio) are not all available (incomplete modalities). Hence, we have M_c, M_t, M_o ⊂ M_a.

Multi-to-uni modal knowledge transfer pre-training
In this section, we introduce our multi-to-uni modal knowledge transfer pre-training framework.

Modal encoders
M2UMol employs four independent encoders to process inputs from different modalities. For the 2D encoder f_g, GraphGPS [81], a transformer-based graph neural network, is utilized to learn molecular 2D embeddings from 2D topological graphs of molecules in M_a. For the 3D encoder f_c, we employ ComENet [82] for learning molecular 3D embeddings from 3D conformer graphs of molecules in M_c. For the Text encoder f_t, we first tokenized the textual descriptions of molecules in M_t using the pre-trained tokenizer of PubMedBERT [83], a large language model pre-trained from scratch using abstracts from PubMed and full-text articles from PubMedCentral. Then the tokens are fed into PubMedBERT, resulting in 768-dimensional vectors from the pooler layer as the Text representations. For the Bio encoder f_o, the three types of biochemical features of each molecule in M_o are first projected into the d-dimensional vectors, and then we take the mean of the vectors as the Bio embedding. Formally, for a molecule m ∈ M_a, we obtain its 2D embedding by h_i^g = f_g(G_i). If the molecule m also belongs to M_c, M_t or M_o, we can obtain its 3D, Text or Bio embedding h_i^c = f_c(C_i), h_i^t = f_t(T_i), or h_i^o = f_o(O_i). All these embeddings are d-dimensional vectors. More details about the selection of the encoders and the encoders’ architecture can be found in Section 2.1 of the Supplementary Information.

Modal-specific adapters
The modal-specific adapters are designed to serve as generators capable of generating pseudo representations of other modalities from 2D modality.

Specifically for the molecule m_i and its 2D representations h_i^g, modal-specific adapters projects h_i^g into 3D, Text and Bio modality representation spaces respectively, with projection heads f_{2D→3D}(·), f_{2D→Text}(·) and f_{2D→Bio}(·):
h̃_i^c = f_{2D→3D}(h_i^g) = LayerNorm(W_1 h_i^g + b_1)
h̃_i^t = f_{2D→Text}(h_i^g) = LayerNorm(W_2 h_i^g + b_2) (1)
h̃_i^o = f_{2D→Bio}(h_i^g) = LayerNorm(W_3 h_i^g + b_3)
where W_1, W_2, W_3 ∈ R^{d×d} are trainable parameter matrices and b_1, b_2, b_3 ∈ R^d denotes trainable bias. h̃_i^c ∈ R^d, h̃_i^t ∈ R^d, h̃_i^o ∈ R^d denote generated representation of 3D, Text and Bio modalities. More details about the rationale behind the design of the modal-specific adapter can be found in Section 2.2 of the Supplementary Information.

Self-supervised learning tasks
In M2UMol, we designed two self-supervised learning tasks, including the generated-actual multimodal contrastive learning and modality classification.

Generated-actual multimodal contrastive learning. In order to empower the model to generate representations of 3D, Text and Bio modalities which are as similar as possible to actual representations, we align the generated pseudo multimodal representations h̃_i^c, h̃_i^t, h̃_i^o with actual multimodal representations h_i^c, h_i^t, h_i^o by contrastive learning (CL).

Let Δ = {c, t, o} denote a set of indices on behalf of the three modalities {3D, Text, Bio}. For every δ ∈ Δ and each molecule m_i^δ ∈ M_δ, we consider its generated/actual representation as the anchor sample, and its actual/generated representation as the positive sample. The generated and actual representations of other molecules in M_δ are considered as the negative samples. Then we maximize the mutual information of positive pairs (each positive pair contains an anchor sample and a corresponding positive sample) while minimizing the mutual information of negative pairs (each negative pair contains an anchor sample and a negative sample). We adopt independent CL objectives for each modality, and the total loss is formulated by [84]:
L_cont = - ∑_{δ∈Δ} ∑_{m_i^δ ∈ M_δ} (log (E(h̃_i^δ, h_i^δ) / (∑_{m_j^δ ∈ M_δ, j≠i} E(h̃_i^δ, h̃_j^δ) + E(h̃_i^δ, h_j^δ))) + log (E(h_i^δ, h̃_i^δ) / (∑_{m_j^δ ∈ M_δ, j≠i} E(h_i^δ, h_j^δ) + E(h_i^δ, h̃_j^δ)))) (2)
where h̃_i^δ and h_i^δ denote the generated representation and actual representation of the molecule m_i^δ in M_δ, respectively; E(u, v) = exp(sim(u, v)/τ), and here sim(u, v) = u^⊺v/∥u∥∥v∥ denote the cosine similarity; and τ is the temperature parameter. Note that we contrast the generated and actual representations of each modality independently. Hence, we need each molecule to have at least one modality (3D/Text/Bio) besides 2D, but without the requirement for the presence of all modalities, endowing the model with the ability to train on incomplete-modal molecular data.

Modality classification Inspired by [85, 86], we designed a modality classifier f_{mcls}(·) to further enhance the modality specificity and distinguishability contained in the generated multimodal representations.

The modality classifier f_{mcls}(·) takes the generated representations h̃_i^c, h̃_i^t and h̃_i^o as inputs, and identifies which modality the generated representation belongs to. To optimize the modality classifier f_{mcls}(·), we minimize the cross-entropy (CE) loss that is formulated as:
L_mcls = - ∑_{m_i ∈ M_a} ∑_{δ∈Δ} y_{i,δ}^⊤ log ŷ_{i,δ}, (3)
where y_{i,δ} ∈ {0, 1}^3 is the one-hot modality label with 1 at the index of modality δ ∈ Δ and 0 elsewhere, and ŷ_{i,δ} ∈ R^3 is the predictive values of the modality classifier f_{mcls}(·) for h̃_i^δ, i.e., ŷ_{i,δ} = f_{mcls}(h̃_i^δ). In the concrete implementation, f_{mcls}(·) is a one-layer linear projection followed by the softmax function.

pre-training objective We optimize the total pre-training loss L_pre-train that combines Eq.(2) and Eq.(3) for pre-training our model M2UMol:
L_pre-train = L_cont + αL_mcls, (4)
where α is a hyperparameter that balances the contribution of the modality classifier.

Fine-tuning with attention-based knowledge fusion
After pre-training, the 2D Modal encoder f_g as well as the modal-specific adapter including projection heads f_{2D→3D}, f_{2D→Text} and f_{2D→Bio} in M2UMol are further fine-tuned with the attention-based knowledge fusion strategy on downstream tasks, simulating multimodal fusion with only molecular 2D topological graph inputs.

Attention-based knowledge fusion For a molecule m_i, we first obtain the topological graph and utilize the pre-trained 2D graph encoder f_g to learn the 2D representations h_i^g. Then the modal-specific projection heads f_{2D→3D}, f_{2D→Text} and f_{2D→Bio} respectively generate 3D, Text and Bio representations ĥ_i^c, ĥ_i^t and ĥ_i^o, which are subsequently fused into the final representations h_i by a multi-head attention mechanism:
Q = (h_i^g)^⊤ W_Q, K = P W_K, V = P W_V
h_i = Concat(head_1, ..., head_h) W_O
head_j = Softmax(Q_j W_Q^j (K_j W_K^j)^T / √d_k) V_j W_V^j, (5)
where h_i^g ∈ R^d and P = {h̃_i^c, h̃_i^t, h̃_i^o} ∈ R^{3×d}; W_Q, W_K, W_V ∈ R^{d×d} are trainable parameters and d is the hidden dimension; the projections are parameter matrices W_Q^j ∈ R^{d×d_k}, W_K^j ∈ R^{d×d_k}, W_V^j ∈ R^{d×d_v} and W_O ∈ R^{d×d}, h denotes the number of attention heads and d_k = d_v = d/h. Finally, h_i denotes the final representations of molecular m_i, which can be fed into different predictors for various downstream tasks. More details for the fine-tuning on downstream tasks can be found in Section 2.3 of the Supplementary Information.

Experimental setting
The dataset for pre-training. DrugBank [87] is a widely utilized and web-enabled database that contains comprehensive molecular information about drugs, providing detailed drug data, including drug targets and drug action information. This database is particularly suitable for constructing multimodal pre-training data. Initially, we downloaded the complete DrugBank database in XML format and extracted 12,227 small molecules with their SMILES, including approved drugs, experimental drugs, withdrawn drugs, and others. Then we obtain their corresponding 2D topological graphs, 3D conformers, textual descriptions, the associations with targets and enzymes, and the drug categories they belong to from the DrugBank database as 2D/3D/Text/Bio modal data. Finally, our dataset contains 11,571 molecules. Out of these molecules, all molecules have their 2D modal data, 9,468 molecules have 3D modal data, 5,899 molecules have Text modal data, and 10,581 molecules have Bio modal data. More details about pre-training dataset construction can be found in Section 1.1 of the Supplementary Information.

The datasets for downstream tasks. For molecular property prediction, we adopted 8 types of benchmark datasets from MoleculeNet [88] for molecular properties prediction, comprising 678 binary classification tasks. The datasets cover molecular data from a wide range of domains, such as drugs, biology, physics, and chemistry. For DDI prediction, we utilized the DDI data from [49], which consists of 191,570 DDIs between 1,700 drugs with 86 types of DDI events, and every DDI is associated with a DDI event. For drug-target interaction prediction, we utilize the BindingDB [55] and BioSNAP [56] dataset, following [57]. The BindingDB dataset consists of 49,199 DTIs between 14,643 drugs and 2,623 target proteins, and the BioSNAP dataset consists of 27,464 DTIs between 4,510 drugs and 2,181 target proteins. To comprehensively evaluate the performance of M2UMol, we used random split, cold split and scaffold split settings. More details about the downstream datasets and split settings can be found in Section 1.2 of the Supplementary Information.

Baselines. For molecular property prediction, as recent advances in this area are predominantly driven by the pre-trained MRL methods, we utilized two types of pre-training MRL methods as baselines: the first is the multimodal pre-training MRL methods that utilize multimodal data for pre-training (3D-Infomax [27], GraphMVP [30], MOLEBLEND[41], MoleculeSDE[32], KV-PLM [31], MoleculeSTM [28], MoleculeSTM_1, MoMu[29], MolFM[33], and MEMO[34]); the second is the unimodal pre-training MRL methods that utilize one modality for pre-training (ChemBERTa [22], MegaMolBART, Molformer-XL[23], EdgePred [37], AttrMask [20], GPT-GNN [19], InfoGraph [38], ContextPred [20], GraphLoG [39], G-Contextal [40], G-Motif [40], GraphCL [25], JOAO[26], MolCLR[24]). For DDI prediction, we mainly choose representative and strong models that are widely adopted in the field as baselines. We summarized four different types of DDI prediction methods which can be applied to cold start and scaffold split settings: 1D fingerprint-based method (STNN-DDI[42]), 2D topological graph-based methods (SSI-DDI[43], GMPNN[44], SA-DDI[45], MSAN [46], DSN-DDI [47]), and 3D conformer graph-based methods (3DGT-DDI [48]). For DTI prediction, we compared the performance of M2UMol with five models (SVM[50], RF[51], DeepConv-DTI [52], GraphDTA [53], MolTrans [54]) on DTI prediction following [57]. More details about baselines can be found in Section 1.3 of the Supplementary Information.

Evaluation metrics. For the molecular property prediction task, we utilize AUROC as the evaluation metric. For the drug-drug interaction prediction task, we use mean accuracy (ACC), macro precision (Macro-P), macro recall (Macro-R), and macro F1 (Macro-F1) as the evaluation metrics. For the drug-target interaction prediction task, we utilize AUROC and AUPRC as the evaluation metrics. For the scaffold split setting, we report the mean and the standard deviation of the metrics of three independent runs with different random seeds. For random and cold start split scenarios, we report the mean and the standard deviation of the metrics of three-fold cross-validation. For the multimodal retrieval task, we use Recall@K as the evaluation metric. More details about the evaluation metrics and implementation can be found in Sections 1.4 and 1.5 of the Supplementary Information.

Peer review rounds
[{"round": 1, "reviewer_comments": {"reviewer_1": "This paper introduces M2UMol, a novel method for pre-training molecular representations by transferring knowledge from multiple modalities to a 2D molecular encoder. The authors also present a new multimodal dataset of drug-like molecules with four modalities: 2D topological graphs, 3D conformer graphs, textual descriptions, and biochemical features. They demonstrate the effectiveness of M2UMol on various molecular tasks, including property prediction and interaction prediction.\nStrength:\n1. M2UMol outperforms baseline methods on most molecular property prediction tasks and molecular interaction prediction tasks, particularly under challenging evaluation scenarios like scaffold split.\n2. Addressing the challenge of incomplete modalities in pre-training and unimodal downstream tasks, which is often encountered in real-world scenarios.\n3. M2UMol distinguishes itself from previous multimodal pre-training methods by explicitly addressing the issue of incomplete modalities and focusing on knowledge transfer to a 2D encoder.\nWeaknesses:\n1. The paper lacks a detailed analysis of M2UMol's performance variation across different datasets, particularly the reasons for suboptimal results on certain tasks.\n2. How did they handle molecules with missing modalities? Were these molecules discarded, or were imputation techniques used?\n3. How were the textual descriptions and biochemical features preprocessed and encoded for model input?\n4. The paper employs different encoders for each modality (GraphGPS for 2D, ComENet for 3D, PubMedBERT for Text). The authors should justify their selection of these specific encoders and discuss potential alternatives. They should also provide more details on the architecture and hyperparameters used for each encoder.\n5. The modal-specific adapter is a crucial component of M2UMol, but its architecture and training process could be described in more detail. What is the rationale behind the design choices for the adapter, the adapter is trained to generate pseudo representations for different modalities, Potential limitations of the adapter and its impact on the overall performance of M2UMol?\n6. While the paper claims that biochemical features are often ignored as a modality in previous works, this statement needs further support. Several studies have explored incorporating rich domain knowledge, including biochemical features, as a modality in molecular representation learning. It's debatable whether treating biochemical features as a separate modality is essential, as they could potentially be integrated with other feature sets. To justify their choice, the authors should include an ablation study demonstrating the specific contribution of biochemical features as an independent modality.\n7. he paper's claim that M2UMol can perform predictions using only 2D modal data, despite being pre-trained on 4 modalities (2D, 3D, Text, Bio), needs clearer explanation. Presumably, the pre-training phase on the 4D data enables the model to extract and encode comprehensive molecular information, allowing it to generalize well even when only 2D data is available during prediction. The authors should explicitly clarify this point.\n8. The paper mentions projecting 2D information using corresponding encoders to obtain 3D, Text, and Bio embeddings for comparison with actual data embeddings. This process, termed \"modal-specific adapter,\" generates pseudo representations for the other modalities based on the 2D molecular representation. The adapter is central to M2UMol's ability to bridge information across different modalities, even when some are missing.\n9. Table 1 shows that M2UMol achieves superior performance on 6 out of 8 datasets. The authors should address the performance discrepancy on the remaining 2 datasets. Analyzing these cases could reveal limitations of the approach or highlight specific molecular properties where the multi-to-uni modal knowledge transfer is less effective. Providing such insights would strengthen the evaluation and contribute to a more nuanced understanding of M2UMol's capabilities.\n10. While the paper uses standard evaluation metrics for molecular tasks (AUROC, ACC, Macro-F1, etc.), it could benefit from including additional metrics that capture different aspects of model performance. For example, metrics for evaluating the quality of generated multimodal representations could provide further insights into the model's capabilities.\n11. The paper briefly mentions some limitations but could expand this discussion to include a more in-depth analysis of potential weaknesses and areas for future improvement.\n(Remarks on code availability)\nI can find the README file. However, I did not have time to review the code.", "reviewer_2": "In this paper, a new pre-training molecular representation learning framework called M2UMol is presented. M2UMol generates pseudo multimodal representations solely from 2D molecular representations thanks to a multi-modal pre-training phase where contrastive learning is leveraged. The paper is well structured, presents the idea clearly and the included results and studies indicate that the model can generate high-quality molecular representations and that is highly competitive in the selected fine-tuning tasks, outperforming the baselines in most of cases.\nI have two comments: (i) Some parts of this work appear similar to another study accepted at a different venue earlier this year (https://openreview.net/forum?id=oM7Jbxdk6Z). A comparison of Table 1 from both papers reveals that the performance of the two models is quite similar across most of the tasks presented. Given this, it would be helpful to explain the differences between the two works in the paper and include this model as an additional baseline. Furthermore, please clarify how the baseline scores were obtained, as there are discrepancies in the reported scores for the same baselines between the two papers. (ii) Secondly, as it concerns the modality classification component. Interestingly, its inclusion enhances the model's performance as it can be seen from the main results and the provided supplementary material. It would be useful to report the selected value of the hyperparameter α that balances the contribution of the modality classifier as this is a key parameter of the model. Additional experiments clarifying the choice of α would be beneficial.\n(Remarks on code availability)\nThe repository contains all the needed components to reproduce the presented results. Also, the authors provide a README with instructions on how to install the package, download the dataset and perform all the needed operations described in the paper.", "reviewer_3": "The manuscript introduces M2UMol, a multi-to-uni modal knowledge transfer framework for molecular representation learning (MRL). This approach is novel because it leverages multimodal data (2D, 3D, text, biochemical features) during pre- training but requires only 2D graphs at inference, addressing a critical real-world challenge where multimodal data is often incomplete. M2UMol distills its final representations into a single 2D-based model suitable for downstream tasks where only 2D data is available at inference. This approach is both timely and practical, given that multimodal data can be incomplete in real-world drug discovery yet can still provide valuable signals at training time for downstream tasks. Although the idea is practical and well-motivated, the current evaluation omits key benchmarks and state-of-the-art (SOTA) baselines, limiting the ability to validate its claimed advantages fully.\nMajor Concerns:\n1. While the paper compares to some strong methods (like GraphMVP, 3D-Infomax, etc.), it omits or only briefly mentions recent leading methods, such as UniMol, ChemRL-GEM, MOLFORMER-XL, and molCLR, all of which have reported excellent results on similar tasks.\n2. Critical drug discovery benchmarks (e.g., ESOL, FreeSolv, QM9 for solubility/energy prediction) are not evaluated, leaving gaps in assessing M2UMol's versatility. Similarly, no direct experiments confirm the pseudo-3D or pseudo-text embeddings can handle purely 3D-driven or text-driven tasks at near-SOTA levels—though the paper suggests strong retrieval alignment, that alone might not prove domain-specific prowess.\nMinor Concerns:\n1. Ensure consistent usage of method names and dataset references - e.g., \"DrugBank\" vs. \"Drugbank\" (Inconsistency);\n\"multimodal\" vs. \"multimodal.\"\n2. Relying on a single generated 3D conformer can be problematic, especially for large or flexible molecules with multiple biologically relevant conformations.\nOverall, I feel the method would be much stronger with evaluation/experiments done with regression-based tasks as well to demonstrate its clear superiority.\nSide Note:\n1. UniMol, ChemRL-GEM, MOLFORMER-XL, and molCLR perform better than M2UMol on the same tasks (binary classification BBBP, BACE, ClinTox, etc.).\n2. Should zero-shot SMILES-to-text molecule captioning on ChEBI-20 also be included to evaluate the model's efficacy (Similar to MolFM)?\n(Remarks on code availability)", "reviewer_4": "I co-reviewed this manuscript with one of the reviewers who provided the listed reports. This is part of the Nature Communications initiative to facilitate training in peer review and to provide appropriate recognition for Early Career Researchers who co-review manuscripts.\n(Remarks on code availability)\nI was able to easily install the environment and successfully use the pretrained model for downstream tasks as described in the README file, including Drug-Drug Interaction (DDI) and Molecular Property Prediction. Additionally, the code was straightforward to apply to custom datasets. The README provides clear instructions for installation and usage, making it a usable resource for the community. The implementation is well-structured, and I encountered no major issues while running the code."}, "author_response": "Response Summary\nWe sincerely appreciate reviewers’ time and effort in handling our manuscript, as well as the positive feedback from the reviewersonourmethod. Theconstructivecommentsarevaluableforimprovingourwork. Inresponse,wehavecarefully\nrevisedthemaintextandtheSupplementaryInformationaccordingtothereviewers’suggestions. Wealsoreorganizedthe\nSupplementaryInformationtoenhanceitsclarityandreadability. Thedetailedrevisionsareoutlinedbelow:\n• Inthe\"Introduction\"section. (1)TomaketheinnovationsandcontributionsofM2UMolmoreprominent,wehave\nrevisedthethirdparagraphofthe\"Introduction\"sectioninthemanuscript,clarifyingthechallengefacedbyexisting\nmethodsmoreclearly. Andwehavesummarizedvariousexistingmultimodalpre-trainingmolecularrepresentation\nlearning(MRL)methods(TableR2,whichisalsoattachedtotherevisedSupplementaryInformationasTableS2)to\nhelpmoreclearlydemonstratethevariousadvantagesofM2UMolcomparedtoexistingmethods. (2)Inresponsetothe\nconfusionraisedbyReviewer#1abouthowthemodelhandlesmoleculeswithmissingmodalities(Comment1.2),we\nhaverevisedthe\"OverviewofM2UMol\"sectiontomoreexplicitlystatethatourmethodhasthecapabilitytoretain\nmoleculeswithmissingmodalitiesforpre-training,alongwiththerationalewhyourmodelpossessesthiscapability. We\nhaveaddedaVenndiagram(FigureR2a,whichhasalsobeenincorporatedintoFigure1aoftherevisedmanuscript)to\nillustratethatM2UMolcanbeextendedtolarger-scalemoleculardatasetsthatincludemoleculeswithmissingmodalities\nforpre-training.\n• Inthe\"Method\"section. (1)InresponsetotheconcernraisedbyReviewer#1aboutwhetheritisessentialtoconsider\nthebiochemicalfeaturesasaseparatemodality(Comment1.6),wefirstelaboratethatthebiochemicalfeatureswe\nusedaresignificantandrarelytakenintoconsiderationinexistingmethods. Thenwehavepointedoutthedrawbacksof\ntreatingbiochemicalfeaturesaspartofothermodalitiesratherthanconsideringthemasaseparatemodality(thecontent\nisaddedtoSection1.1oftherevisedSupplementaryInformation). (2)InresponsetothequestionraisedbyReviewer#1\nregardingtherationalebehindtheadapterdesign(Comment1.5),wehaveexplainedthemotivationandrationalebehind\ntheadapterdesign(thecontentisaddedtoSection2.2oftherevisedSupplementaryInformation). (3)Moredetailsabout\nthepre-trainingdataset(Comment1.3,1.6)andthemodalencoders(Comment1.4)areaddedtoSections1.1and2.1of\ntherevisedSupplementaryInformation,respectively.\n• Inthe\"Results\"section. (1)AssuggestedbyReviewer#2(Comment2.1)and#3(Comment3.1),wehavecompared\nM2UMolwithMOLEBLEND,Uni-Mol,ChemRL-GEM,andMOLFORMER-XLonMPPclassificationtasks,and\nM2UMolcanstillachievetheoptimalperformanceonmostdatasets. TheresultsofMOLEBLENDandMOLFORMER-\nXLareaddedtoTable1intherevisedmanuscript. SinceUni-MolandChemRL-GEMarefine-tunedon3Dconformers,\nweaddedtheresultsofthemtoTableS8oftherevisedSupplementaryInformationasadditionalcomparisonresults. We\nhavealsoevaluatedtheperformanceofM2UMolonMPPregressiontaskssuggestedby#3(Comment3.2). Theresults\n(TableR7)areaddedtotherevisedSupplementaryInformationasTableS9.(2)Weinvestigatetheimportanceofdifferent\nmodalitiesbycomparingM2UMolwithitsvariantspre-trainedwithallpossiblesingleandpairwisecombinationsofthe\n3D,Text,andBiomodalities. TheresultsinFigureR4showthatallthemodalitiesaresignificantforM2UMol,and\nM2UMolcaneffectivelylearndiversemultimodalinformation. Besides,consideringtheBiomodalityasaseparate\nmodalityandintroducingitintothepre-trainingcanleadtoperformanceimprovementon7outof8datasets,which\nindicatestheBiomodalitycanprovideuniquebiologicalinsightsthatenhancethemolecularrepresentationlearning\n(Comment 1.6). (3) We have also discussed the impact of our designed modal-specific adapters on our model by\ncomparingM2UMolwithitsvariantsthatremoveadaptersinthepre-trainingorfine-tuningphase. TheresultsinFigure\nR3verifythecrucialroleofthemodal-specificadaptersinM2UMol. (4)InresponsetotheconcernraisedbyReviewer#1\nregardingthe\"performancediscrepancy\"onMPPclassificationtasks(Comment1.1),wehaveexplainedthatthesample\nquantityandthenumberoflabelscanjointlyaffecttheperformanceofthemodelontheMPPtasks(theexplaination\nare added to Section 1.2 of the revised Supplementary Information). We have also summarized the statistics of the\nMPPdatasets(TableR1,whichisalsoaddedtotherevisedSupplementaryInformationasTableS1)tosupportthis\nexplanation. (5)AssuggestedbyReviewer#1(Comment1.9),wehaveconductedcasestudiesontwoMPPdatasets\nthatM2UMolachievessuboptimalperformance(FigureR5),whichshowthenegativeeffectsofmissingstereochemical\ninformationinbondinitializationattributes. ForimprovingthemodelperformanceonMPPtasks,weconsidermore\ncomplete bond attributes as the initialization features of edges in 2D molecular graphs, which enables M2UMol to\nachievebetterperformanceonmostMPPdatasets. TheresultsofM2UMol,alongwiththeresultsofablationstudieson\ntheMPPdataset,areallupdatedintherevisedmanuscriptandSupplementaryInformation.\n1/18\nReviewer #1 (Remarks to the Author):\nThispaperintroducesM2UMol,anovelmethodforpre-trainingmolecularrepresentationsbytransferringknowledgefrom\nmultiplemodalitiestoa2Dmolecularencoder. Theauthorsalsopresentanewmultimodaldatasetofdrug-likemolecules\nwithfourmodalities: 2Dtopologicalgraphs, 3Dconformergraphs, textualdescriptions, andbiochemicalfeatures. They\ndemonstratetheeffectivenessofM2UMolonvariousmoleculartasks,includingpropertypredictionandinteractionprediction.\nComment1.1 ThepaperlacksadetailedanalysisofM2UMol’sperformancevariationacrossdifferentdatasets,particularly\nthereasonsforsuboptimalresultsoncertaintasks.\nResponse1.1: TheperformancevariationsofM2UMolandbaselinesacrossdifferentdatasetsshowasimilartrend(as\nshowninTable1oftheoriginalmanuscript). Specifically,allthemethodstendtoperformwellondatasetssuchasClintox,\nBACE, and HIV, but achieve relatively lower performance on datasets like SIDER, ToxCast, and Tox21. To explain this\nsituation,weanalyzethecharacteristicsoftheeightdatasetsusedinourexperiments,assummarizedinTableR1. Wefindthat\nSIDER,ToxCast,andTox21containalargernumberofpredictiontasksbutrelativelylimiteddata,whichposeschallengesfor\nlearningeffectiverepresentationstomakeaccuratepredictions. Inthesecases,theToxCastdatasetstandsoutasanexception.\nDespiteitslargenumberoftasks(617tasks),mostofthemethodscanachievebetteraccuracyonToxcastcomparedtoSIDER\n(27tasks). SowefurtherinvestigatethedifferencesbetweenSIDER,ToxCast,andTox21,andconductacorrelationanalysisof\nthetasklabelsinthesedatasets. Specifically,weuseCramér’sVtoquantifytheinter-taskdependenciesandvisualizethetask\nlabelcorrelationsviaaheatmap(FigureR1). ItisgenerallybelievedthattwotasklabelsaresignificantlycorrelatedifCramér’s\nVvaluebetweenthemisgreaterthan0.5. WecalculatetheproportionofelementswithCramér’sVvaluesgreaterthan0.5in\nthetasklabelcorrelationmatrix,andfindthatToxCasthasmoretaskpairsthathavesignificantcorrelation(theproportion=\n0.1414). TheresultsindicatethatToxCastexhibitsstrongerinter-tasklabelcorrelationcomparedtoSIDERandTox21,which\nmayhelpthemodelbettergeneralizeacrosstasksandalleviatethechallengeraisedbythegreatnumberoftasks.\nRegardingtheissueraisedbythereviewersthatM2UMoldidnotachievetheoptimalaccuracyonthetwodatasets(Tox21\nandSIDER),wewouldliketoprovidethefollowingclarification.AsTableR1showsthattheMPPclassificationdatasetsvaryby\ndatadistribution,reflectingthatitisinherentlydifficultforasingleMRLmodeltosimultaneouslyaccommodatesuchdiverse\npropertypredictiontaskswithoutencounteringtrade-offsinperformance. Therefore,achievingoptimalaccuracyacrossall\ndatasetsisextremelychallenging. Moreover,thisphenomenonisnotuniquetoM2UMol. Recentstate-of-the-artMRLmodels\npublishedinleadingjournals 2,3,4 alsodonotachievethebestaccuracyoneverydataset. Notably,despitebeingpre-trained\nononly11Kmolecules,M2UMolachievesthebestperformanceon6outof8datasets,whichsufficientlydemonstratesthe\nrobustnessandsuperiorityofourproposedmodel.\nThe analysis of MPP datasets has been added to Section 1.2 of the revised Supplementary Information. Table\nR1hasbeenaddedtotherevisedSupplementaryInformationasTableS1. FigureR1hasbeenaddedtotherevised\nSupplementaryInformationasFigureS1.\nComment1.2 Howdidtheyhandlemoleculeswithmissingmodalities? Werethesemoleculesdiscarded,orwereimputation\ntechniquesused?\nTableR1. Summaryforthemolecularpropertypredictiondatasets.\nDataset #Tasks #Compounds #Description1\nMUV 17 93087 SubsetofPubChemBioAssaybyapplyingarefinednearestneighbor\nanalysis,designedforvalidationofvirtualscreeningtechniques.\nHIV 1 41127 ExperimentallymeasuredabilitiestoinhibitHIVreplication.\nBACE 1 1513 Quantitative(IC50)andqualitative(binarylabel)bindingresultsfora\nsetofinhibitorsofhumanβ-secretase1(BACE-1).\nBBBP 1 2039 Binarylabelsofblood-brainbarrierpenetration(permeability).\nTox21 12 7831 Qualitativetoxicitymeasurementson12biologicaltargets, including\nnuclearreceptorsandstressresponsepathways.\nToxCast 617 8575 Toxicologydataforalargelibraryofcompoundsbasedoninvitrohigh-\nthroughputscreening,includingexperimentsonover600tasks.\nSIDER 27 1427 Databaseofmarketeddrugsandadversedrugreactions(ADR),grouped\ninto27systemorganclasses.\nClinTox 2 1478 QualitativedataofdrugsapprovedbytheFDAandthosethathavefailed\nclinicaltrialsfortoxicityreasons.\n2/18\nFigureR1. ThetasklabelcorrelationmatricesfortheToxCast,Tox21,andSIDERdatasetsviaheatmaps.\nResponse 1.2: To provide a comprehensive explanation of the noted question, we first systematically review and\nsummarizetheexistingmultimodalpre-trainingmolecularrepresentationlearning(MRL)methods,highlightingmultiple\nchallengessuchastheinabilitytohandlemoleculeswithmissingmodalities. Then,wereviewtheframeworkofourmethod\nandexplaintheprincipleoftheabilityofM2UMoltohandlemoleculeswithmissingmodalitiesandthereasonfornotusing\nimputationtechniques. Finally,weverifythatourabilitytohandlemoleculeswithmissingmodalitiescanleadtoperformance\nimprovement.\nAssummarizedinTableR2,theexistingmultimodalpre-trainingMRLmethodscanbedividedintotwocategories. The\nfirstcategorycanbesummarizedasa\"one-one\"paradigm,whichaimstomodeltherelationshipbetweentwomodalities\ninpre-trainingtograspmodalinformation. Typicalpre-trainingtasksinthiscategoryincludealigningtherepresentations\noftwomolecularmodalitiesusingcontrastivelearning5,6,7,8 orgeneratingamolecularmodalityfromanother9,10,8. Insuch\nframeworks,downstreamfine-tuningusuallyreliesonasinglepre-trainedencoder,typicallya2Dmolecularencoder. Asa\nresult,thesinglepre-trainedencodercannotfullyexploitthemultimodalknowledgelearnedinpre-training. Moreover,this\ncategoryisinherentlylimitedtotwomolecularmodalitiesandprovidesaninsufficientunderstandingofmoleculesduetothe\nlackofknowledgefrommoremodalities. Itischallengingtodesignpre-trainingstrategiesinvolvingthreeormoremolecular\nmodalities. Thesecondcategoryincludesrecentefforts11,12whichattempttoaddressthesechallengesthrougha\"one-others\"\nparadigm. These methods select one modality as the anchor and integrate representations of all other modalities through\nconcatenationorsummation. Theythenadoptcontrastivelearningtoaligntherepresentationsoftheanchormodalitywiththe\nintegratedmultimodalrepresentations. Intheirfine-tuningphase,alltheencodersfordistinctmodalitiesserveaspre-trained\nmodules,andtheirlearnedrepresentationsareintegratedfordownstreamtasks. However,thesemethodsoftennecessitatethat\nthemoleculeshavecompletemodalitiesduringboththepre-trainingandfine-tuningphases,resultingintwoflaws. Ontheone\nhand,theyfailtoexploitlarger-scalemoleculeswithmissingmodalitiesformoresufficientpre-training. Ontheotherhand,\ntheyarenotcompetentinpracticaldownstreamapplications,whereallmodalitiesareoftenunavailableexceptmolecular2D\ntopologicalgraphs(usuallyconvertedfromSMILES).\nUnliketheexistingmultimodalpre-trainingMRLmethods, M2UMolcanuseabroaderrangeofmoleculesforpre-\ntraining(thered-borderedareainFigureR2a)thatcontainmoleculeswithmissingmodalities,andalsobecompetent\nintheabovementionedpracticalapplicationscenarioswhereonlymolecular2Dstructuresareavailable. Specifically,\ninthepre-trainingphase,M2UMollearns2Drepresentationsfrommolecular2Dstructuresbya2Dgraphencoder,which\nsubsequentlyareconvertedintomultipletypesofpseudorepresentationscorrespondingtotheotherthreemodalities(3D,\nText,andBio)bythemodal-specificadapters. Combiningtheactualmultimodalrepresentationslearnedbythecorresponding\nmodal encoders, we conduct two pre-training objectives: the modality classification and the generated-actual multimodal\ncontrastivelearning.Themodalityclassificationiscarriedoutonthegeneratedpseudorepresentationstoenhancetheirmodality\ndistinguishabilityandspecificity. Hence,itonlyrequiresthatthe2Dgraphstructuresoftrainingmoleculesamplesareavailable.\nThegenerated-actualmultimodalcontrastivelearningcontainsthreecontrastivelearningobjectivesthataimtoseparatelyalign\nthe pseudo representations for each modality from {3D, Text and Bio} with the corresponding actual representations, i.e.,\n2D-3D,2D-Textand2D-Biocross-modalcontrastive. Hence,themoleculeswithboth2Dgraphstructuresandatleastone\n3/18\nTableR2. Summaryoftheexistingmultimodalpre-trainingMRLmethods\nPre-trainingon Requiredmodalities\nCategory Model Pre-trainingmodule Pre-trainingstrategy Fine-tuningmodule\nincompletemodalities infine-tuning\n3D-Infomax5 2D&3Dencoders Cross-modalcon. × 2Dencoder 2D\nGraphMVP8 2D&3Dencoders Cross-modalcon.&gen. × 2Dencoder 2D\nMoleculeSDE10 2D&3Dencoders Cross-modalgen. × 2Dencoder 2D\none-one MoleculeSTM6 2D&Textencoders Cross-modalcon. × 2Dencoder 2D\nA↔B MoMu7 2D&Textencoders Cross-modalcon. × 2Dencoder 2D\nKV-PLM9 SMILES&Textencoders Cross-modalgen. × SMILESencoder SMILES\nMOLEBLEND13 2D&3Dencoders Cross-modalgen. × 2D&3Dencoders 2D+3D\none-others MolFM11 2D&Text&KGencoders Cross-modalcon. × 2D&Text&KGencoders 2D+Text+KG\nA↔[B+C+...] MEMO12 SMILES&FP&2D&3Dencoders Cross-modalcon. × SMILES&FP&2D&3Dencoders SMILES+FP+2D+3D\nM2UMol 2D&3D&Text&Bioencoders Multi-to-unimodal ✓ 2Dencoder 2D\n+modal-specificadapters knowledgetranfer +modal-specificadapters\nThemeaningof\"one-others\"referstoaligningonemodalitywithanintegratedmodalityofallothermodalitiesinpre-training.The\"con.\"and\"gen.\"arethe\nabbreviationsfor\"contrastivealignment\"and\"generation\",respectively. The\"×\"indicatesmethodsthatrequiremoleculeswithcompletemodalitiesfor\npre-training.Incontrast,the\"✓\"indicatesmethodscanadditionallyincorporatemoleculeswithincompletemodalitiesforpre-training,therebyenabling\npre-trainingonabroaderrangeofmoleculardata.\nFigureR2. a. TheVenndiagramrepresentsthedistributionofmultimodaldatainthedataset. Theareaborderedbytheblack\ndashedlinemeansthemoleculeswithallmodalities,whichareusuallyusedbyexistingmethods. Theareaborderedbythered\nlinemeansmoleculeswithincompletemodalities,coveringabroaderrangeofmolecules,whichcanbeusedforpre-trainingin\nourmethod. b. Theresultsoftheablationstudytoevaluatetheinfluenceofthemoleculeswithincompletemodalities. The\nvariantM2UMol(discardincompletedata)removesmoleculeswithincompletemodalitiesandispre-trainedonlyonmolecules\nthatsimultaneouslyhaveallmodalities.\nofthethreemodalitiescanbeusedforourpre-training, withoutrequiringmodalitycompletenessforalltrainingmolecules.\nTherefore,theabilityofourmodeltoembracethemoleculeswithmissingmodalitiesinpre-trainingstemsfromourwell-\ndesignedpre-trainingstrategy. Inthefine-tuningphase,M2UMolfuses2Drepresentationswiththepseudorepresentations\ngeneratedbyadapters,realizingthesimulationofamultimodalfusionprocesswithsolely2Dstructuresandhenceenhancing\ntheperformanceindownstreamtaskswithonly2Dmodalityavailable.\nM2UMoladdressesthechallengeofIncompleteMultimodalLearning,whichisanemergingresearchdirectionthataims\ntofacilitateeffectivelearningfromincompletemultimodaldata14. M2UMolretainmoleculeswithmissingmodalitiesfor\npre-traininginsteadofdiscardingthem. Wedidnotuseanyimputationtechniquestofillinthemissingmodalitiesfor\nthesemoleculesinpre-training. Inourpre-trainingdataset,thereare18%,49%and9%ofmoleculesthatlack3D,Text,and\nBiodata,respectively. Webelievethatfillinginthemultimodaldataforsuchagreatproportionofmoleculesthroughsome\nimputationmethodswouldintroducetoomuchnoise,whichwoulddisruptthemodel’slearningfrommultimodalinformation.\nTheabilitytoincorporatemoleculeswithmissingmodalitiesduringpre-trainingisakeyadvantageofourmethodoverexisting\nmethods. Furthermore,weconductanablationstudythatcompareM2UMolwiththevariantofM2UMolthatispre-trained\nonlyonmoleculesthathavecompletemodalities. TheresultinFigureR2bshowthatpre-trainingonmoremoleculeswith\nmissingmodalitiescanachievebetterperformancethanonlyonmoleculesthathavecompletemodalities,whichprovesthe\neffectivenessofourdesign.\nWehaverevisedthethirdparagraphofthe\"Introduction\"sectionintherevisedmanuscript,clarifyingthechallenge\nmoreclearlyandmakingtheinnovationsandcontributionsofM2UMolmoreprominent. Thedescriptionsaboutthe\nabilityofM2UMoltohandlemoleculeswithmissingmodalitiesandtherationalewhyM2UMolpossessesthisability\nhavebeenaddedtothe\"OverviewofM2UMol\"sectionintherevisedmanuscript. TableR2hasbeenaddedtothe\n4/18\nrevisedSupplementaryInformationasTableS2. FigureR2ahasbeenaddedtoFigure1aoftherevisedmanuscript.\nFigureR2bhasbeenincorperatedtoFigureS3oftherevisedSupplementaryInformation.\nComment1.3 Howwerethetextualdescriptionsandbiochemicalfeaturespreprocessedandencodedformodelinput?\nResponse1.3: Asmentionedinourresponse1.2,themostsignificantadvantageofM2UMolovertheexistingmethodsis\nthatitrequiresactualmultimodaldataonlyinpre-training,andcansolelyutilize2Ddatatogeneratemultimodalrepresentations\nformultimodalfusioninfine-tuning. Hence,boththetextualdescriptionsandbiochemicalfeaturesarepreprocessedand\nencodedformodelinputonlyinpre-training.\nInthepre-trainingphase,forthetextualdescriptions,weextracttwocategoriesofdruginformationfromDrugBank:\n(1)Concisesummariescontainingkeyproperties,mechanismsofaction,andusageguidelines;(2)comprehensivebackground\ninformationcoveringhistoricaldevelopment,discoveryprocesses,andclinicaltrialoutcomes.Thesetextsaretokenizedbyusing\nPubMedBERT’spre-trainedtokenizer15. Then,thesetokensarefedintoPubMedBERT,resultingin768-dimensionalvectors\nfromthepoolerlayerofPubMedBERT,whichareservedastheTextrepresentations. Alltheabovestepsareimplementedusing\nthetransformerspackage16 ofHuggingface. Forthebiochemicalfeatures,wederivethreetypesofbiochemicalfeatures:\ndrugcategories,targetsinteractedwiththedrug,andenzymesassociatedwiththedrug. Eachtypeoffeatureisencodedas\nabinaryvectorwhoseelement(0or1)indicatesthepresenceorabsenceofthecorrespondingdrugcategory/target/enzyme.\nThesevectorsarethenprojectedintoaunifieddimensionusingthreeseparatelinearlayers,followedbyameanpoolingto\nobtainthefinalrepresentationoftheBiomodality.\nWehaveaddedthedetailsaboutpreprocessingandencodingofmultimodaldatatothe\"Multimodalmoleculedata\nconstruction\"sectionandthe\"Modalencoders\"sectionintherevisedmanuscript,respectively.\nComment1.4 Thepaperemploysdifferentencodersforeachmodality(GraphGPSfor2D,ComENetfor3D,PubMedBERT\nforText). Theauthorsshouldjustifytheirselectionofthesespecificencodersanddiscusspotentialalternatives. Theyshould\nalsoprovidemoredetailsonthearchitectureandhyperparametersusedforeachencoder.\nResponse1.4: Inourwork,weleveragemultiplemolecularmodalities,including2Dtopologygraphs,3Dconformers,\nandtextualdescriptions. Eachmodalityrequiresaspecificencodertotransformitintoameaningfulrepresentation. TableR3\nsystematicallysummarizestheencodersusedforeachmodalityintheexistingmultimodalpre-trainingMRLmethods.Itreveals\nthatGIN17, SchNet18, andSci-BERT19 haveemergedasthemostwidelyusedencodersfor2D,3D,andTextmodalities,\nrespectively. Whiletheseencodershavedemonstratedeffectivenessingeneralmultimodalpre-trainingMRLmethods,they\nmaybesuboptimalforourspecificmulti-to-uniknowledgetransferpre-trainingframework. Wenextelaborateontheencoders\nselectedforourframework,alongwiththerationalebehindeachchoice.\nTableR3. Summaryforthemultimodalencodersselectedbytheexistingmultimodalpre-trainingMRLmethods\nModel Modalitiesused 2Dencoder 3Dencoder Textencoder\n3D-Infomax5 2D+3D PNA SMP -\nGraphMVP8 2D+3D GIN SchNet -\nMoleculeSDE10 2D+3D GIN SchNet -\nMoleculeSTM6 2D+Text GIN - Sci-BERT\nMoMu7 2D+Text GIN - Sci-BERT\nKV-PLM9 SMILES+Text - - Sci-BERT\nMOLEBLEND13 2D+3D ASharedEncoder - -\nMolFM11 2D+Text+KG GIN - Sci-BERT\nMEMO12 SMILES+FP+2D+3D GIN SchNet -\nM2UMol 2D+3D+Text+Bio GraphGPS ComENet PubMedBERT\n• Forthe2Dencoder,wechoseGraphGPS,agraphtransformer-basedencoder. Ontheonehand,thegraphtransformer-\nbasedencodersarecurrentlythemostadvancedinmodelingmoleculartopology. Theycanprovidepowerfulexpressivity\nandinterpretabilitybasedontheirattentionmechanism13,20. Ontheotherhand,inourmulti-to-uniknowledgetransfer\nscenario,weexpectour2Dencodertobeabletosufficientlyreceivemultimodalknowledgefrommoleculardatainother\nmodalitiesthroughpre-training. Consideringthecomplexheterogeneityamongthosemodalities,wecontendthata2D\nencoderwithasimplearchitecture(e.g.,GIN)wouldnotfittheexpectation. Incontrast,GraphGPSwithfine-grained\nandadaptivemodelingforinteratomicinteractionmaybemorecompetentinourpre-training. Moreover,theempirical\n5/18\nexperimentalresultsinGraphGPS21 demonstrateitssuperiorperformanceinvariousgraphrepresentationtasksover\nGIN17.\n• For the 3D encoder, we chose ComENet22. Compared with commonly used 3D conformation encoders, such as\nSchNet18,DimeNet23,andSphereNet24,ComENethasbeendemonstratedtocapturemorecomprehensivemolecular\n3Dconformationalinformation22. Moreimportantly,itcanachievelowertimecomplexityandhighercomputational\nefficiency22,whichismoreadaptedtoourcomplexmultimodallearningscenario.\n• FortheTextencoder,wechosePubMedBERT15,whichhasprovenitsstrongpoweroverBERT25,BioBERT26and\nSci-BERT19inhandlingbiomedicalapplications15. Duetousingpurelyin-domainbiomedicaldatatopre-trainfrom\nscratch,PubMedBERThasanin-domainvocabularytohelpnaturallanguageunderstanding(e.g.,’chloramphenicol’\nwillbebrokenintowordpieces’ch-lor-amp-hen-ico-l’bySci-BERT,whilePubMedBERThas’chloramphenicol’in\nitsvocabulary). Inaddition,Sci-BERTneedstobalanceoptimizingforpre-trainingonbiomedicaltextandcomputer\nsciencetext,thelatterofwhichisunlikelytobebeneficialforbiomedicalapplications,leadingtoworseperformance\nthanPubMedBERTinbiomedicalapplications.\nWemustemphasizethatourmodelisdedicatedtodevisinganovelpre-trainingframeworkthatcanpre-trainonthree\nor more modalities and can generate multimodal representations solely from 2D molecular structure when fine-tuned on\ndownstreamtasks. Intheproposedframework,thecoredesignistheinnovativepre-trainingstrategy,multi-to-uniknowledge\ntransferpre-training,whichtransfersmultimodalknowledgelearnedthroughpre-trainingintodownstreamtasks. Toshow\nthesuperiorityofthispre-trainingstrategy,wehaveconductedablationstudiesonmodalities(FigureR4),whichshowthat\nremoving any modality in pre-training leads to a decline in model performance. On the one hand, it indicates that all the\nmodalitiesweusedarecrucial,andusingmoremodalitiescanhelpthemodelmorecomprehensivelyunderstandmolecules.\nOntheotherhand, itprovesthatM2UMolcaneffectivelyintegratemultimodalinformationandenhancethedownstream\ntasks. Moreover,asmentionedinResponse1.2,adefiniteadvantageofM2UMol,bornfromourwell-designedpre-training\nobjectives,isitsabilitytousemoleculeswithmissingmodalitiesforpre-training,whichresultsinasignificantimprovement\nover the pre-training on complete modalities, as shown in Figure R2 b. And as mentioned in Response 1.5, our designed\nmodal-specificadaptersplayacrucialroleinbothpre-trainingandfine-tuning,whichisdemonstratedbytheablationstudy\nshowninFigureR3. Insummary,theseresultscollectivelydemonstratethatthestrengthofM2UMolarisesfromourunique\nand innovative pre-training strategy, rather than from the choice of specific encoders. In molecular representation learning,\nmanyencodersforvariousmolecularmodalitieshavebeenproposed,andmoreadvancedencoderswillemergeinthefuture.\nThoughtheremightbebetterencodersforourmodel,ourfocusisnotonfindingthebestencoders,butratheronproposinga\nnovelpre-trainingframeworkthatcanaddressthepracticalapplicationchallenges: incompletemodalitiesinpre-trainingand\nunimodaldownstreamtasks.\nWehaveaddedtherationaleforselectingtheseencoders,alongwithdetailsabouttheirarchitectures,toSection2.1\noftherevisedSupplementaryInformation. ThedetailsabouttheirhyperparametershavebeenaddedtoSection1.5of\ntherevisedSupplementaryInformation. TableR3hasbeenaddedtotherevisedSupplementaryInformationasTable\nS7.\nComment1.5 Themodal-specificadaptersconstituteacrucialcomponentofM2UMol,butitsarchitectureandtraining\nprocesscouldbedescribedinmoredetail. Whatistherationalebehindthedesignchoicesfortheadapter,theadapteris\ntrainedtogeneratepseudorepresentationsfordifferentmodalities,Potentiallimitationsoftheadapteranditsimpactonthe\noverallperformanceofM2UMol?\nResponse1.5: AsmentionedinthesecondparagraphofResponse1.2,inthepre-training,M2UMolaimstoalign2D\nmodalitywithmultiplemodalitiesintherepresentationspace,enablingthemodal-specificadapterstoacquiretheabilityto\nconvert 2D representations into pseudo multimodal representations. When undergoing fine-tuning, M2UMol retains the\nadapters’abilitytogeneratethepseudomultimodalrepresentationsfrom2Drepresentationstoenhancethedownstreamtasks.\nInthefollowing,weexplaintherationaleandthemotivationfortheadapters’designandshowtheimportanceoftheadapters\nthroughanablationstudy.\nInthefieldofmultimodallearning,adaptersarewidelyusedtobridgethegapbetweendifferentmodalitiesandtoenablean\nencoderpre-trainedononemodalitytoeffectivelyprocessandunderstandinformationfromanothermodality27. Typically,\nadaptersaredesignedaslightweightmodules,suchasatwo-layerMLP,andareappendedafterthepre-trainedencoders28,29,30.\nDuringthefine-tuningphase,theparametersofthepre-trainedencodersareusuallyfrozen,whileonlytheadapterparameters\nareupdated. Thisdesignallowsthepre-trainedencoderstoretaintheirstrongcapabilityinhandlingtheiroriginalmodality,and\ntheadaptersfacilitatetheencoders’abilitytoincorporateandcomprehendknowledgefromadditionalmodalities. Moreover,\ntheoutputsoftheadaptersaretypicallycombinedwiththeoutputsofthepre-trainedencodersthrougharesidualconnection,\ntherebyenrichingthemodal-specificrepresentationsusedfordownstreamtasks28,29,31.\n6/18\nInspiredbytheseworks,wedesignedthemodal-specificadaptersforenablingthe2Dencodertoeffectivelyunderstand\nmultimodalinformationfrom3D,Text,andBiomodalitiessimultaneously.Ourdesignedmodal-specificadapterssimplyconsist\nofalinearlayerandalayernormlayer,andaredeployedbetweenthe2Dencoderandmultimodalencoders,andtrainedinboth\nthepre-trainingandfine-tuningphases. Inthepre-training,theseadapterscanlearntheabilitytoconvert2Drepresentations\nofmoleculesintopseudomultimodalrepresentations,helpingthe2Dencoderunderstandothermodalities. Inthefine-tuning,\nmodal-specific adapters can serve as generators of corresponding modalities, whose outputs are considered as the pseudo\nmultimodalrepresentations. Thegeneratedpseudomultimodalrepresentationsarethenfusedwith2Drepresentations,realizing\nthesimulationofamultimodalfusionprocesswithsolely2Dstructuresandhenceenhancingtheperformanceindownstream\ntaskswithonlya2Dmodalityavailable. Toinvestigatetheimpactofthemodal-specificadaptersonM2UMol,weconductan\nablationstudybycomparingM2UMolwithitsvariantsinwhichtheadaptersareremovedeitherinthepre-trainingphaseorthe\nfine-tuningphase. TheresultsinFigureR3showthatremovingadaptersinpre-trainingandfine-tuningleadstoperformance\ndegradationon6and5datasets,respectively,underscoringthecriticalroleofmodal-specificadaptersinenablingeffective\nmultimodalinformationlearningduringpre-trainingandingeneratingreliablemolecularrepresentationsduringfine-tuning.\nFigureR3. Theresultsoftheablationstudyonthemodal-specificadapters. Weconstructedtwovariants,M2UMol(w/o\nadaptersinpre-training)andM2UMol(w/oadaptersinfine-tuning). Theformerisconstructedbyremovingthemodal-specific\nadapterinthepre-trainingphaseandconductscontrastivelearningbetween2Drepresentationsandrepresentationsofother\nmodalitiesinthesamevectorspace. Thelatterisconstructedbyremovingthemodal-specificadapterinthefine-tuningphase,\nandonlythe2Drepresentationsareusedforpredictionwithoutgeneratedrepresentationsofdifferentmodalities.\nAlthoughthemodal-specificadaptershelpM2UMolachievebetterperformance,thereisstillapotentiallimitation. While\nthegeneratedmultimodalrepresentationscanprovidemultimodalinformation,theycannotreplacetherealisticmultimodal\nrepresentations. Inaddition,therearevariousadapterstructuresavailable,andtheremaybebetteroptionstochoosefrom.\nHowever,consideringtheimportanceofscalabilityandefficiencyfortheapplicationofM2UMolindownstreamtasks,we\nultimatelychosearelativelysimplebuteffectiveadapterarchitecture.\nMoredetailsaboutthearchitectureandtrainingprocessofthemodal-specificadapters,alongwiththerationale\nbehindthedesign,havebeenaddedtoSection2.2oftherevisedSupplementaryInformation. FigureR3hasbeenadded\ntoTableS3intherevisedSupplementaryInformation.\nComment1.6 Whilethepaperclaimsthatbiochemicalfeaturesareoftenignoredasamodalityinpreviousworks,this\nstatementneedsfurthersupport. Severalstudieshaveexploredincorporatingrichdomainknowledge,includingbiochemical\nfeatures,asamodalityinmolecularrepresentationlearning. It’sdebatablewhethertreatingbiochemicalfeaturesasaseparate\nmodalityisessential,astheycouldpotentiallybeintegratedwithotherfeaturesets. Tojustifytheirchoice,theauthorsshould\nincludeanablationstudydemonstratingthespecificcontributionofbiochemicalfeaturesasanindependentmodality.\nResponse1.6: Therearesomemultimodalpre-trainingMRLmethodsthatconsiderbiochemicalfeatures,whichpotentially\nexistintextualdescriptions6,9,7andareconsideredaspartoftheTextmodality. However,itcanleadtothelossofbiochemical\ninformationandweakentheiradvantagesinunderstandingtheroleofmoleculesinbiologicalsystems. Soitisessentialto\nconsider biochemical features as a separate modality. Moreover, in our work, we considered the drug’s categories and\nassociationswithtargetsandenzymesastheBiomodality. Thecategoriesofdrugsinvolveoneormoretermsthatsignify\nthe class to which a drug belongs, with each term referring to a specific aspect of drug properties, use, or mechanism of\naction,offeringacomprehensiveunderstandingofdrugs. Theassociationswithtargetsandenzymesplayacriticalrolein\n7/18\ncomprehendingthemolecular-levelmechanismsofdrugaction. Itisinstrumentalinelucidatingtheinvivopathwaysthrough\nwhichdrugsexerttheireffects,whichholdssignificantguidingimplicationsfordrugdesign,research,development,andclinical\napplications. Thesebiochemicalfeatureshavenotbeendirectlyconsideredbytheexistingmultimodalpre-trainingMRL\nmethods\nToinvestigatethecontributionofeachmodality,weconstructedseveralvariantsofM2UMolbyaligningthe2Dmodality\nwithallpossiblesingleandpairwisecombinationsofthe3D,Text,andBiomodalities. Wethencomparedtheperformanceof\nthesevariantswiththefullversionofM2UMol. Besides,wechooseMolCLR,whichisthestrongest2Dunimodalpre-training\nMRLmethodinTable1ofthemanuscript,asareferencebaseline. TheresultspresentedinFigureR4showthatM2UMol\nachievesthebestperformanceon6outof8datasetsandthesecond-beston1dataset,demonstratingthatallthemodalities\nare significant for M2UMol and M2UMol can effectively learn diverse multimodal information. In addition, the variants\nofM2UMolthatincorporateonlyoneortwoadditionalmodalitiesstillconsistentlyoutperformMolCLR,highlightingthe\nsignificanceofusingmultiplemodalitiesforpre-training. Moreover,M2UMol(2D+Bio)achievesbetterperformancethan\nM2UMol(2D+3D)andM2UMol(2D+Text)across4outof8datasets,indicatingthedistinctiveandsubstantialcontribution\noftheBiomodalitytomolecularrepresentationlearning. ByexplicitlyincorporatingBiomodalityasaseparatesourceof\nknowledgeinourframework,M2UMolisabletocaptureuniquebiologicalinsightsthatcomplementmultimodalinformation,\ntherebyenhancingtheexpressivenessandutilityofthelearnedrepresentations.\nFigure R4. The results of the ablation study on the modalities used for pre-training. We use each modality or modality\ncombinationsforpre-trainingM2UMoltoformseveralvariantsofM2UMoltoinvestigatetheinfluencesofmodalitiesonour\nmodel. Barsmarkedwithastarrepresentthehighestperformanceonthecorrespondingdataset,whereasbarswithatriangle\nindicatethesecond-bestperformance. DashedhorizontallinesindicatetheperformanceofMolCLR,thestrongestunimodal\npre-trainingMRLmethodbasedon2Dmoleculargraphsinourbaselines,usedasareferencebaselineacrossdatasets.\nWe have added the details about the construction of Bio modality to Section 1.1 of the revised Supplementary\nInformation. TheanalysisoftheablationstudyhasbeenaddedtoSection3.2oftherevisedSupplementaryInformation.\nFigureR4hasbeenaddedtoFigureS3oftherevisedSupplementaryInformation.\nComment1.7 Thepaper’sclaimthatM2UMolcanperformpredictionsusingonly2Dmodaldata,despitebeingpre-trained\non4modalities(2D,3D,Text,Bio),needsclearerexplanation. Presumably,thepre-trainingphaseonthe4Ddataenablesthe\nmodeltoextractandencodecomprehensivemolecularinformation,allowingittogeneralizewellevenwhenonly2Ddatais\navailableduringprediction. Theauthorsshouldexplicitlyclarifythispoint.\nResponse1.7: AsmentionedinthesecondparagraphofResponse1.2,M2UMolrequiresrealmultimodaldataonly\nduringpre-training,whileinthefine-tuning,differentfromthe’generalize’mentionedbythereviewer,M2UMolcan\ngeneratepseudomultimodalrepresentations,whicharethenfusedwith2Drepresentations. Thisdesigneffectivelyrealizes\nthesimulationofamultimodalfusionprocesswithsolely2Dstructures,therebyenhancingtheperformanceindownstream\ntaskswhereonlythe2Dmodalityisavailable.\nComment1.8 Thepapermentionsprojecting2Dinformationusingcorrespondingencoderstoobtain3D,Text,andBio\nembeddingsforcomparisonwithactualdataembeddings. Thisprocess,termed\"modal-specificadapters\",generatespseudo\nrepresentationsfortheothermodalitiesbasedonthe2Dmolecularrepresentation. TheadaptersarecentraltoM2UMol’s\nabilitytobridgeinformationacrossdifferentmodalities,evenwhensomearemissing.\nResponse1.8: Thankyouforpointingoutoneofourkeycontributionsaboutourwell-designedmodal-specificadapters.\n8/18\nComment1.9 Table1showsthatM2UMolachievessuperiorperformanceon6outof8datasets. Theauthorsshouldaddress\ntheperformancediscrepancyontheremaining2datasets. Analyzingthesecasescouldreveallimitationsoftheapproach\norhighlightspecificmolecularpropertieswherethemulti-to-unimodalknowledgetransferislesseffective. Providingsuch\ninsightswouldstrengthentheevaluationandcontributetoamorenuancedunderstandingofM2UMol’scapabilities.\nResponse1.9: ToanalyzethereasonsforM2UMol’sperformancediscrepancyontheTox21andSIDER,weconducta\nvisualizationanalysisofthemolecularrepresentationslearnedbyM2UMolusingUMAPdimensionalityreductionand\nplotthedecisionboundaries(FigureR5). SinceTox21andSIDERcontainmultipletasks,weconductvisualizationanalysisof\nallthetasksinbothdatasetsbutonlypresentthe\"Musculoskeletalandconnectivetissuedisorders\"taskfromSIDERandthe\n\"NR-AR\"taskfromTox21,becausethemisclassifiedmoleculesinthesetasksexhibitmoreobviouscommonalities. Through\ntheresults,wehavethefollowingobservations:\n• For the SIDER dataset, we choose the misclassified molecules around the decision boundary (the orange box in\nFigure R5a) and present their structures. We find that these molecules are predominantly macrocyclic compounds\n(blue-highlightedstructuresinFigureR5a). Macrocyclicmolecules,characterizedbycomplex3Dconformations,often\nrelyonsubtlestereochemicalinteractionsforbiologicalactivity. ItdemonstratesthatM2UMolmaystruggletocapture\nstereochemicalfeatures,potentiallylimitingitsperformance.\n• FortheTox21dataset,wealsochoosethemisclassifiedmoleculesaroundthedecisionboundary(theorangeboxin\nFigureR5c)andpresenttheirstructures.Weobservethatthesemoleculescontainbothsteroidandnon-steroidcompounds\n(green-highlightedregionsrepresentthesteroidnuclei). Thisindicatesthatthemodelfailstoeffectivelydiscriminate\nbetweensteroidsandnon-steroids. Thesteroidnucleus,composedofthreesix-memberedringsandonefive-membered\nring, exhibitscomplexspatialgeometry. Themodel’sinabilitytodistinguishthesestructuressuggestsalimitationin\nlearningintricatestereochemicalfeatures.\nTheseobservationsindicatethatstereochemicalinformationiscrucialforunderstandingcomplexspatialmolecularstructures,\nandignoringitcanleadtosuboptimalperformanceondatasetsthatarehighlyrelatedtothemolecularstructure. Thelackof\nstereochemicalinformationmayalsobethereasonwhyourmulti-to-uniknowledgetransferpre-trainingislesseffective(the\ndegradedversionofM2UMolthathasnotundergonemultimodalpre-trainingachievedbetterperformancethanM2UMol)on\ntheTox21dataset.\nToaddresstheperformancediscrepancyonMPPtasks,weconsiderthemorecomprehensivebondattributesfromOGB32\nastheinitializationfeaturesofedgesin2Dmoleculargraphs. ThebondattributesfromOGB32haveanadditionalbond_stereo\nattribute,whichcontainsbondstereochemistryinformationandcanprovidecriticalinsightsintothespatialarrangement\nofatomsarounddoublebondsorchiralcenters,enhancingthemodel’sabilitytocapturestereoisomericdifferences\ninmolecules. Ithasbeenintroducedbymanymultimodalpre-trainedMRLmethods8,6,11,10,12andweoverlookeditbefore.\nAfterupdatingthebondattributes,thevisualizationanalysisresultsonM2UMolshowclearerUMAP-separatedclustersand\nsharperdecisionboundariesonbothSIDERandTox21datasets(FigureR5bandFigureR5d). Andtheclassificationerrorsfor\nmacrocyclicmoleculesintheSIDERandthefeaturespaceoverlapbetweensteroidsandnon-steroidsintheTox21dataset\narereduced. Moreover,wealsocomparedtheresultsofM2UMolwithM2UMolwithoutaddingthebond_stereoattribute\n(M2UMolw/obond_stereo)onMMPclassificationtasks. Thedegradedversionsofthem(M2UMol andM2UMol\nunpre unpre\nw/obond_stereo)arealsoincluded. AstheresultsshowinTableR4,M2UMolachievessignificantlybetterperformanceson\nfivedatasets,includingTox21andSIDER,thanM2UMolw/obond_stereo. Inaddition,M2UMoloutperformsM2UMol\nunpre\nonalldatasets. Theseobservationsindicatethattheaddedbond_stereoattributeisbeneficialforimprovingtheperformanceof\nM2UMol. Andaddingthebond_stereoattributecanmakeourmulti-to-unimodalknowledgetransfereffectiveonalldatasets,\nwhichmaybebecauseitcanenhancetheinformationofthe2Dgraphitself,therebyhelpingeliminatethegapbetweenthe\nmodalitiesandfacilitatingbetterknowledgetransfer.\nTableR4. ResultsoftheM2UMolbeforeaddingbond_stereoattributeandafteraddingbond_stereoattributeontheMPP\nclassificationtasksunderthescaffoldsplitsetting.\nMethods BBBP Tox21 ToxCast SIDER Clintox MUV HIV BACE Avg\nM2UMol w/obond_stereo 63.8(0.6) 75.8(0.4) 64.2(0.3) 59.7(0.5) 90.2(0.2) 75.9(1.7) 74.7(0.7) 78.7(1.1) 72.88\nunpre\nM2UMolw/obond_stereo 73.4(0.5) 74.5(0.0) 66.5(0.0) 63.1(0.9) 94.4(0.3) 78.7(0.8) 77.6(0.8) 83.4(0.2) 76.45\nM2UMol 66.9(1.2) 74.9(0.3) 64.3(0.8) 61.4(0.5) 90.6(1.9) 75.8(3.9) 76.2(0.3) 80.1(2.2) 73.78\nunpre\nM2UMol 68.2(0.6) 76.2(0.1) 66.3(0.5) 66.0(0.8) 92.2(0.5) 80.5(0.8) 79.0(1.0) 84.3(1.4) 76.59\nThebest-performingresultsaremarkedinbold.\n9/18\n12xoT\nLabel 0 Label 1\nREDIS\noerets_dnob\no/w\nloMU2M\nloMU2M\nMolecular representation visualization\nsesaC\ngnorW\nsesaC\n1\nlebaL\nsesaC\n0\nlebaL\na\nb\nc\nd\nsesaC\ngnorW\nsesaC\n1 lebaL\nsesaC\n0 lebaL\nO\nO OH\nHO HO N HO HO OH HO O OH O OH O O OH\nO O HO O O HO O N O ...\nHO HO OH O\nOH NH2\nOHO N\nO O O O O O HO O O O\nO\nHO HO O S\nN HO HO O\nCl\nHO Cl\nO O\nN N N + O−\nN N N N H HO HN O S ...\nO N N\nN H2N S O S O O O O HO O H\nO O\nO\nN O\nO HO\n...\nO O O HO\nN N S O S N N N H HN O S\nN O O HO O O N HO O O O N O N OH\nO O\nO O O O\n...\nO\nO O O O O HO\nO O\nO OH O\nO O O O\nHO ...\nO\nO O O O O HO O O O O HO O\nO\nO O\nOH\nNH\n...\nO\nO\nNH\noerets_dnob\no/w\nloMU2M\nloMU2M\nFigure R5. Left of a,b,c,d: Visualization of molecular representation and decision boundary. M2UMol: consider bond\nattributes with bond_stereo attribute; M2UMol w/o bond_stereo: consider bond attributes without bond_stereo attribute;\nRight of a,b,c,d: Molecular structures corresponding to the scatter points in the selected regions of the scatterplot. (The\nblue-highlightedstructureismacrocyclic;thegreen-highlightedstructureissteroidcore.)\nTheresultsofM2UMolonMPPtasksareupdatedinTable1oftherevisedmanuscript. Theresultsoftheablation\nstudiesonMPPtasksareallupdated.\nComment1.10 Whilethepaperusesstandardevaluationmetricsformoleculartasks(AUROC,ACC,Macro-F1,etc.),it\ncouldbenefitfromincludingadditionalmetricsthatcapturedifferentaspectsofmodelperformance. Forexample,metricsfor\nevaluatingthequalityofgeneratedmultimodalrepresentationscouldprovidefurtherinsightsintothemodel’scapabilities.\nResponse1.10: WehadevaluatedtheperformanceofM2UMolacrossthreekeytasks: molecularpropertyprediction,\ndrug-druginteractionprediction,anddrug-targetinteractionprediction. Foreachtask,weemployedallstandardevaluation\nmetricsthatarewidelyadopted,ensuringafairandcomprehensivecomparisonwithexistingmethods.\n10/18\n• Molecularpropertyprediction(binary/multi-labelclassification): AUC\n• Drug-druginteractionprediction(multi-classclassification): ACC,Macro-F1,Macro-Recall,Macro-Precision\n• Drug-targetinteractionprediction(binaryclassification): AUC,AUPR\nAsforevaluatingthequalityofthegeneratedmultimodalrepresentationsmentionedbythereviewers,wehadconducteda\nmultimodalretrievaltask(Figure3aintheoriginalmanuscript). Therationaleisthathigh-qualitygeneratedrepresentations\nshouldlieclosetotheircorrespondingactualrepresentationsinthevectorspace. Specifically,weconsideredallthegenerated\nandactualrepresentationsinacommonEuclideanspace,andcheckedwhethertheactualrepresentationiswithinthetop-k\nclosestrepresentationstothecorrespondinggeneratedrepresentation. WeadoptedthewidelyusedmetricRecall@K33,34,7,11\ntoevaluatethequalityofgeneratedmultimodalrepresentationsinthemultimodalretrievaltask. Specifically,Recall@K\nmeasuresthefractionofgeneratedrepresentationqueriesforwhichthecorrespondingactualrepresentationappearsamongthe\ntop-Kclosestrepresentations. Formally,letQbethesetofallgeneratedrepresentationsandlett(q)denotetheground-truth\nactualrepresentationforqueryq∈Q. Recall@KisdefinedasRecall@K= 1 (cid:80) 1 (cid:2) t(q)isamongthetop-Kclosesttoq (cid:3) ,\n|Q| q∈Q\nwhere1[·]istheindicatorfunction,equalto1iftheconditionholdsand0otherwise. TheresultsinFigure3ainthemanuscript\nshow that the generated and actual representations are matched well in the space, which demonstrates that M2UMol can\ngeneratereliablemultimodalrepresentationsonlyusing2Dmoleculegraphs.\nWehaveaddedthedetailsabouttheRecall@KtoSection1.4oftherevisedSupplementaryInformation.\nComment1.11 Thepaperbrieflymentionssomelimitationsbutcouldexpandthisdiscussiontoincludeamorein-depth\nanalysisofpotentialweaknessesandareasforfutureimprovement.\nResponse1.11:Benefitingfromourwell-designedmulti-to-unimodalknowledgetransferpre-trainingframework,M2UMol\ncanlearnhigh-quality molecularrepresentationsandachievesuperiorperformance onimportantmoleculartasks, suchas\nmolecular property prediction and molecular interaction prediction. In recent years, molecular generation has garnered\nsignificantattentionduetoitspotentialtoacceleratedenovodrugdesignandexplorenovelchemicalspace,andhasbecomea\ncriticalfrontierincomputationalchemistryanddrugdiscovery.Consideringthatlearninghigh-qualitymolecularrepresentations\nisalsothefundamentalbasisformoleculargeneration,ourfutureworkwillfocusonextendingthesuperiorcapabilitiesof\nourmodelinmolecularrepresentationlearningtomoleculegeneration. Weaimtoinvestigatetheadvantagesofmultimodal\nrepresentationlearninginmoleculegeneration,anddesignamultimodalmoleculegenerationmodelcapableofgenerating\nnovelmoleculesguidedbydiversemultimodalinputs,suchasthe3Dstructuresoftargets’pocketsandtextualdescriptions\naboutdemandedmolecularproperties.\nWehaveaddedtheabove-mentionedlimitationsandareasforfutureimprovementtothe\"Conclusion\"sectionin\ntherevisedmanuscript.\nReviewer #2 (Remarks to the Author):\nInthispaper,anewpre-trainingmolecularrepresentationlearningframeworkcalledM2UMolispresented.M2UMolgenerates\npseudomultimodalrepresentationssolelyfrom2Dmolecularrepresentationsthankstoamulti-modalpre-trainingphasewhere\ncontrastivelearningisleveraged. Thepaperiswellstructured,presentstheideaclearlyandtheincludedresultsandstudies\nindicatethatthemodelcangeneratehigh-qualitymolecularrepresentationsandthatishighlycompetitiveintheselected\nfine-tuningtasks,outperformingthebaselinesinmostofcases.\nComment 2.1 Some parts of this work appear similar to another study accepted at a different venue earlier this year\n(https://openreview.net/forum?id=oM7Jbxdk6Z).AcomparisonofTable1frombothpapersrevealsthattheperformanceofthe\ntwomodelsisquitesimilaracrossmostofthetaskspresented. Giventhis,itwouldbehelpfultoexplainthedifferencesbetween\nthetwoworksinthepaperandincludethismodelasanadditionalbaseline. Furthermore,pleaseclarifyhowthebaseline\nscoreswereobtained,astherearediscrepanciesinthereportedscoresforthesamebaselinesbetweenthetwopapers.\nResponse2.1: Wehavesummarizedmultimodalpre-trainingMRLmethods,includingMOLEBLEND13 inTableR2,\nthroughwhichwecanobservethatM2UMoldifferssubstantiallyfromMOLEBLENDinmultipleaspects. Thesedifferences\nareprimarilymanifestinthepre-trainingandfine-tuningphases:\n• In the pre-training phase, MOLEBLEND is specifically designed for learning the intrinsic correlation between 2D\nand 3D modalities, which is hard to extend to other modalities (such as Text and Bio modalities), and limiting the\ncomprehensiveunderstandingofmolecules. Incontrast,ourmethodM2UMolispre-trainedonfourmodalitiesto\nlearnmorecomprehensivemolecularknowledge,whichismorechallenging. (1)Thefourmodalitiesweemploy\nexhibitsignificantlywidermodalitygapscomparedtotherelativelynarrowgapbetween2Dand3Dmodalities(bothare\nrelatedtomolecularstructure). Theseheterogeneousmodalitiesdiffersubstantiallyintermsofrepresentationformats,\n11/18\nsemanticgranularity,andinformationdensity,posingconsiderablechallengesforeffectivemultimodalalignment.(2)The\navailabilityofmodalitiesvariesacrossmolecules,withsomemoleculescontainingonlyasubsetofthefourmodalities\nratherthanallofthem,whichisamorecomplexdatascenariothanthefullypaired2Dand3Ddata.\n• In the fine-tuning phase, MOLEBLEND implicitly injects 3D knowledge into a 2D unimodal encoder to learn rep-\nresentations for downstream tasks, lacking multimodal complementary and interactive information. However, our\nmethodcandirectlygeneratepseudomultimodalrepresentationswithonly2Dinputs,whicharethenintegratedwith\n2Drepresentationsbyanattentionmechanismforenablingthesimulationofmultimodalfusionandsupplementing\nmultimodal information to enhance the downstream tasks that only give 2D data. Besides, the generated pseudo\nmultimodalrepresentationcanalsobeusedformultimodalretrieval,whichcanprovidevaluablesupportformolecular\nresearch.\nAssuggestedbythereviewer,wehaveincludedtheMOLEBLENDasabaselinemethodforcomparison. Wereproduced\nMOLEBLEDNandemployeditonourdatasplitforafaircomparison. Thedetailsaboutthereproductioncanbefoundin\nSection1.3oftherevisedSupplementaryInformation. WeadditionallyincludetheMoleculeSDE10,whichisalsoamultimodal\npre-trainingMRLmethodthatutilizes2Dand3Dmodalitiesforpre-training,asabaselinemethod. Thecomparisonresults\nare shown in Table R5. Despite being pre-trained on only 11K molecules (significantly fewer than the 3.37M molecules\nrequiredbyMOLEBLEND),M2UMolstilloutperformsMOLEBLENDandMoleculeSDEonmostdatasets. Itdemonstrates\nthat, benefiting from our designed multi-to-uni knowledge transfer pre-training strategy, M2UMol can efficiently acquire\nmore comprehensive multimodal knowledge, consequently achieving superior performance with lower cost compared to\nMOLEBLEND.\nTableR5. ResultsofM2UMol,MoleculeSDE,andMOLEBLENDoneightmolecularpropertydatasetsunderthescaffold\nsplitsetting.\nMethods #Molecules BBBP Tox21 ToxCast SIDER Clintox MUV HIV BACE Avg\nMoleculeSDE10 3.4M 73.2(0.5) 76.5(0.3) 65.2(0.3) 59.6(0.8) 86.6(3.7) 79.9(0.2) 78.5(0.3) 80.4(0.9) 74.98\nMOLEBLEND13 3.37M 66.5(0.7) 76.9(0.5) 64.6(0.4) 61.6(1.5) 85.2(4.2) 76.7(1.0) 74.7(2.7) 76.3(0.3) 72.80\nM2UMol 11K 68.2(0.6) 76.2(0.1) 66.3(0.5) 66.0(0.8) 92.2(0.5) 80.5(0.8) 79.0(1.0) 84.3(1.4) 76.59\nThebest-performingresultsaremarkedinboldandthesuboptimalresultsareunderlined.\nRegardingthesourceoftheresultsinourpaper,theresultsofChemBERTa35,Momu7,3D-Infomax5,MoleculeSDE10,\nMolFM11,andMEMO12aretakendirectlyfromtheoriginalpaper. TheresultsofMegaMolBART36,MolCLR37,KV-PLM9\nand MoleculeSTM6 are taken from MoleculeSTM6. The results of EdgePred38, AttrMask39, GPT-GNN40, InfoGraph41,\nContextPred39,GraphLoG42,G-Contextal43,G-Motif43,GraphCL44,JOAO45andGraphMVP8aretakenfromGraphMVP8.\nSincethedatasplittingsofMOLEBLEND13,ChemRL-GEM46,Uni-Mol47,andMOLFORMER-XL48haveminorvariations\ncomparedtoours,wereproducedthesemethodsandemployedthemonourdatasplitforafaircomparison. Wedownloadthe\nofficialprogramofthesemethods,andretrainandtestthemonourdatasplits. Specifically,wereconstructedthetrain/val/test\ndatasetsaccordingtooursplitsandtheirdataformat. Thenwereplacetheiroriginaldatasetwithourconstructedones,avoiding\nany modifications to their model’s source code. We strictly adhere to their provided hyperparameters for training. Since\nthesemethodsdidnotspecifyrandomseeds,weuserandomseeds0,1,and2,whicharethesameasourownmethod. The\ndiscrepanciesinsomebaselineresultsoccurbecauseGraphMVPre-implementedcertainmethodsforcomparison. Sincewe\nadoptedidenticaldatasplitstoGraphMVP,somebaselineresultsinourpaperweretakenfromGraphMVP’sre-implementation\nratherthandirectlytakenfromoriginalpaperslikeMOLEBLENDdid.\nWehaveaddedthedetailsaboutthereproducingofMOLEBLENDtoSection1.3oftherevisedSupplementary\nInformation. TheresultsofMOLEBLENDandMoleculeSDEhavebeenaddedtoTable1oftherevisedmanuscript.\nThe source of the baseline results and the reason for the discrepancy in the results of the same baselines between\nMOLEBLENDandourshavebeenaddedtoSection1.3oftherevisedSupplementaryInformation.\nComment 2.2 Secondly, as it concerns the modality classification component. Interestingly, its inclusion enhances the\nmodel’sperformanceasitcanbeseenfromthemainresultsandtheprovidedsupplementarymaterial. Itwouldbeuseful\ntoreporttheselectedvalueofthehyperparameterα thatbalancesthecontributionofthemodalityclassifierasthisisakey\nparameterofthemodel. Additionalexperimentsclarifyingthechoiceofα wouldbebeneficial.\nResponse2.2: Thankyouforyourrecognitionofourdesignedmodalityclassificationcomponent. Itisouroversightfor\nnotreportingthevalueofα. Weconductparametertuninganalysisforα bycomparingM2UMolunderdifferentvaluesof\nthehyperparameterα onmolecularpropertypredictiontasks. TheresultsinFigureR6showthatM2UMolpre-trainedwith\nα =0.5canachievebetterperformancesonmostdatasets,sowechoosethehyperparameterα =0.5.\n12/18\nFigureR6. PerformanceofM2UMoloneightmolecularpropertypredictiondatasetsunderdifferentvaluesofthehyperpa-\nrameterα.\nWehaveaddedtheselectedvalueofhyperparameterα,alongwiththeresultsofparametertuninganalysis,toSection\n1.5oftherevisedSupplementaryInformation. FigureR6hasbeenaddedtotherevisedSupplementaryInformationas\nFigureS2.\nReviewer #3 (Remarks to the Author):\nThemanuscriptintroducesM2UMol,amulti-to-unimodalknowledgetransferframeworkformolecularrepresentationlearning\n(MRL).Thisapproachisnovelbecauseitleveragesmultimodaldata(2D,3D,text,biochemicalfeatures)duringpre-training\nbutrequiresonly2Dgraphsatinference,addressingacriticalreal-worldchallengewheremultimodaldataisoftenincomplete.\nM2UMoldistillsitsfinalrepresentationsintoasingle2D-basedmodelsuitablefordownstreamtaskswhereonly2Ddatais\navailableatinference. Thisapproachisbothtimelyandpractical,giventhatmultimodaldatacanbeincompleteinreal-world\ndrugdiscoveryyetcanstillprovidevaluablesignalsattrainingtimefordownstreamtasks. Althoughtheideaispractical\nandwell-motivated,thecurrentevaluationomitskeybenchmarksandstate-of-the-art(SOTA)baselines,limitingtheabilityto\nvalidateitsclaimedadvantagesfully.\nComment 3.1 While the paper compares to some strong methods (like GraphMVP, 3D-Infomax, etc.), it omits or only\nbrieflymentionsrecentleadingmethods,suchasUni-Mol,ChemRL-GEM,MOLFORMER-XL,andmolCLR,allofwhichhave\nreportedexcellentresultsonsimilartasks.\nResponse3.1: Thanksforthereviewer’ssuggestion. MolCLR37hadbeenincludedasabaselinemethodinTable1ofour\ninitiallysubmittedmanuscript. ThereasonthatwedidnotinitiallyincludecomparisonswithUni-Mol47,ChemRL-GEM46,\nandMOLFORMER-XL48isthat(1)wemainlycompareourmethod(amultimodalpre-trainingMRLmethod)withexisting\nmultimodalpre-trainingMRLmethods,butUni-Mol47,ChemRL-GEM46,andMOLFORMER-XL48areunimodalpre-training\nMRLmethods; (2)ourworkprimarilyfocusesonmethodsthatonlyrequireeasilyavailablemodalitiesofmolecules(i.e.,\nSMILESor2Dmolecularstructures)asinputindownstreamtasks,while,Uni-MolandChemRL-GEMutilize3Dmolecular\nstructuresasinputindownstreamtasks.\nTableR6. Resultsofdifferentmolecularpropertypredictionmethodsoneightmolecularpropertydatasetsunderthescaffold\nsplitsetting.\nMethods Pre-train #Molecules Fine-tune BBBP Tox21 ToxCast SIDER Clintox MUV HIV BACE Avg\nMolformer-XL48 SMILES 1.1B SMILES 69.7(0.5) 77.3(1.1) - 62.5(0.7) 88.8(3.2) - 78.7(1.1) 82.1(0.6) -\nMolCLR37 2D 10M 2D 67.8(0.5) 75.6(0.4) 64.6(0.1) 58.7(0.1) 84.2(1.5) 72.8(0.7) 75.9(0.2) 71.1(1.2) 71.34\nChemRL-GEM46 3D 20M 3D 67.9(1.3) 76.7(0.6) 67.3(0.4) 60.3(0.8) 88.2(0.9) - - 81.3(1.1) -\nUni-Mol47 3D 209M 3D 70.0(0.7) 78.5(3.1) 68.9(0.2) 62.9(1.1) 86.2(1.6) 75.3(1.2) 77.0(1.9) 77.6(5.9) 75.13\nM2UMol 2D+3D+Text+Bio 11K 2D 68.2(0.6) 76.2(0.1) 66.3(0.5) 66.0(0.8) 92.2(0.5) 80.5(0.8) 79.0(1.0) 84.3(1.4) 76.59\nThebest-performingresultsaremarkedinbold.\nWeacknowledgetheconstructivesuggestionofthereviewerandarealsocuriousabouthowourM2UMolmethodwould\nperformrelativetothesemethods. SowecompareM2UMolwithMOLFORMER-XL,MolCLR,ChemRL-GEM,andUni-Mol\nforamorecomprehensiveevaluationoftheperformanceofM2UMol. SincethedatasplittingsofChemRL-GEM,Uni-Mol,\n13/18\nandMOLFORMER-XLhaveminorvariationscomparedtoours,wereproducedthesemethodsandemployedthemonour\ndata split for a fair comparison. We downloaded the official program of these methods, and retrained and tested them on\noursplitdataset. Specifically,wereconstructedthetrain/val/testdatasetsaccordingtooursplitsandtheirdataformat. Then\nwereplacedtheiroriginaldatasetwithourconstructedones,avoidinganymodificationstotheirmodel’ssourcecode. We\nstrictly adhere to the provided hyperparameters for training. Since these methods did not specify random seeds, we use\nrandomseeds0, 1, and2, whicharethesameasourownmethod. ThecomparisonresultsofM2UMolwiththemethods\nmentionedbythereviewerareshowninTableR6. TheresultsshowthatM2UMolproducesthebestperformanceon5outof8\ndatasetswithaverageimprovementsof8.02%,6.85%,8.71%,2.33%,and8.04%onAUC,respectively,whichdemonstrates\nthesuperiorperformanceofM2UMol. Additionally,wehaveseveralkeyfindings: (1)M2UMoloutperformsUni-Moland\nChemRL-GEMonthemajorityofdatasets,benefitingfromitsabilitytointegraterichermultimodalknowledge. Incontrast,\nUni-MolandChemRL-GEMarebothunimodalpre-trainingmethodsfocusedprimarilyonmolecular3Dstructures,which\nlimitstheirgeneralizationacrossdiversemolecularpropertytasks. Uni-MolandChemRL-GEMachievebetterperformanceon\nTox21andToxCast,likelyduetotheirfine-tuningonmoleculeswith3Dconformers. Thisallowsthemtomoreeffectively\ncapturespatialstructuralinformation,whichmaybeparticularlyrelevanttothemolecularinteractionsrepresentedinthese\ntoxicity-relateddatasets. (2)M2UMolachievessuperiorperformanceacrossdiverseMPPdatasetswhilebeingpre-trainedon\nonly11Kmolecules,whichis1,000timesfewerthanMolCLR(whichusesthefewestmoleculesamongcomparablemethods).\nThishighlightsthehighscalabilityandefficiencyofourapproach. (3)M2UMolexhibitsrelativelylowerstandarddeviations\nacrossalldatasets,particularlyonTox21(0.1),Clintox(0.5),andMUV(0.8). ThissuggeststhatM2UMolnotonlyachieves\nstrongaverageperformancebutalsodemonstratesstabilityandconsistencyacrossdifferentruns,whichiscrucialforpractical\napplicationswherereliablepredictionsareneeded.\nWehaveaddedthedetailsaboutreproducingChemRL-GEM,Uni-Mol,andMOLFORMER-XLtoSection1.3of\ntherevisedSupplementaryInformation. TheresultsofMOLFORMER-XLandMolCLRhavebeenaddedtoFigure1\noftherevisedmanuscript. TheresultsofChemRL-GEMandUni-Mol,alongwiththecorrespondinganalysis,have\nbeenaddedtoSection3.1oftherevisedSupplementaryInformation.\nComment3.2 Criticaldrugdiscoverybenchmarks(e.g.,ESOL,FreeSolv,QM9forsolubility/energyprediction)arenot\nevaluated,leavinggapsinassessingM2UMol’sversatility. Similarly,nodirectexperimentsconfirmthepseudo-3Dorpseudo-\ntextembeddingscanhandlepurely3D-drivenortext-driventasksatnear-SOTAlevels—thoughthepapersuggestsstrong\nretrievalalignment,thatalonemightnotprovedomain-specificprowess.\nResponse3.2: WeinitiallydidnotevaluateM2UMolonregressiontasks,asmostexistingmultimodalmolecularpre-\ntrainingmethodsfocusprimarilyonclassificationtasks9,6,7,11. Uponfurtherinvestigation,wefoundthatregressiontasksare\ntypicallyusedinstudiesinvolving3Dmolecularmodalities5,8. ThisislikelybecauseregressionbenchmarkssuchasESOL\n(aqueoussolubility),FreeSolv(solvationfreeenergy),andLipophilicity(octanol-waterpartitioncoefficient)fundamentally\nevaluatemolecule-H2Ointeractions. Theseinteractionsaretypicallydeterminedbyconformation-specificpolarfunctional\ngroupsforminghydrogenbondswithH2O.Hence,theseregressiondatasetsarehighlyrelevanttothemolecular3Dconformers.\nInresponsetothereviewer’ssuggestion,wehavenowevaluatedourmethod’sperformanceonregressiontasks,including\nFreeSolv,Lipo,ESOL,Malaria,andCEP.TheQM9datasetsarenotincludedbecauseQM9datasetsexistintheformof3D\nmolecularconformationsandaretypicallyutilizedforevaluatingtheabilityofthemodeltohandlemolecular3Dconformers.\nSinceourmethodisspecificallydesignedtoenhance2Dmolecularrepresentations,evaluationonthisdatasetmightbeunfair\ntoM2UMol. AsshowninTableR7,M2UMolachievesoptimalperformanceon4outof5datasets,whichdemonstratesthe\neffectivenessandsuperiorityofM2UMol. Furthermore,wehavethefollowingobservations: (1)Themultimodalpre-training\nMRLmethods(3DInformax,GraphMVP,andMEMO)tendtoachievebetterperformancesthanunimodalmethods(AttrMask,\nContextPred,andJOAO),indicatingthatincorporatingmultiplemodalitiesduringpre-trainingenablesthemodeltocapture\nricherandmoreinformativerepresentations. Thisleadstoimprovedperformanceacrossvariousdatasets. (2)M2UMolachieves\ncompetitiveorsuperiorperformancecomparedtoothermultimodalpre-trainingMRLmethods. Thisadvantagearisesfrom\nM2UMol’sintegrationofmorediversemodalities. WhileMEMOalsoutilizesfourmodalities,theyareallfocusedonmolecular\nstructuralinformation. Incontrast,M2UMolincorporatesTextandBiomodalitiesinadditionto2Dand3Ddata. Although\nthe3Dconformationofmoleculesiswidelyconsideredcrucialformolecularregressiontasks,theinclusionofTextandBio\ndataallowsM2UMoltobetterunderstandmoleculesfrommultipleperspectives,resultinginsuperiorregressionperformance.\n(3)M2UMolsignificantlyoutperformsM2UMol ,whichdemonstratesthatourdesignedpre-trainingframeworkcanalso\nunpre\nsignificantlyimprovethequalityofmolecularrepresentationlearningintheregressiontask,therebyachievingbetterregression\nperformance.\nMoreover, in response to the reviewers’ concerns regarding whether the pseudo-3D or pseudo-text embeddings can\nhandlepurely3D-drivenortext-driventasks. Wewouldliketoemphasizethatthepurposeofgeneratingpseudomultimodal\nrepresentationsisnottohandlepurely3Dortext-driventasks. M2UMolaimstogeneratepseudomultimodalrepresentations\nforenablingthesimulationofmultimodalfusionevenwhenonlythe2Dmodalityisavailable. Itcanhelpthemodelcapture\n14/18\nTableR7. Resultsofdifferentmolecularpropertypredictionmethodsonfiveregressionmolecularpropertydatasetsunderthe\nscaffoldsplitsetting. Thebest-performingresultsaremarkedinboldandthesuboptimalresultsareunderlined.\nMethods Pre-train Fine-tune ESOL↓ FreeSolv↓ Lipo↓ Malaria↓ CEP↓\nAttrMask 2D 2D 1.112 - 0.730 1.119 1.256\nContextPred 2D 2D 1.196 - 0.702 1.101 1.243\nJOAO 2D 2D 1.120 - 0.708 1.145 1.293\n3DInformax 2D+3D 2D 0.894 2.337 0.695 - -\nGraphMVP 2D+3D 2D 1.029 - 0.681 1.097 1.244\nMEMO SMILES+2D+3D+FP 2D 0.984 - 0.707 1.093 1.101\nM2UMol - 2D 1.010 2.338 0.712 1.110 1.046\nunpre\nM2UMol 2D+3D+Text+Bio 2D 0.914 1.894 0.667 1.093 1.038\nmultimodalcomplementaryandinteractiveinformationamongmodalities,enhancingtheperformanceondownstreamtasks\nwhereonly2Ddataisgiven.\nInaddition,assuggestedbythereviewerintheSideNote,weconductedSMILES-to-textmoleculecaptioningonChEBI-20\nfollowingtheMolFM11. Specifically,weremovedthemoleculesthatappearintheChEBI-20testingsetfromourpre-training\ndatasetandre-trainedourpre-trainedmodel. FollowingMolFM,weconcatenatedtheatomicfeatureslearnedbyM2UMol\nwiththeencoderoutputsofMolT5formolecularcaptiongeneration. AstheresultsshowninTableR8,M2UMolcanachieve\ncompetitiveperformanceonthemoleculecaptioningtask. Thepossiblereasonourmethoddidnotachieveoptimalperformance\nisthatthemolecularcaptioningtasktypicallyrequiresatomic-levelrepresentationsasinput,whereasourmethodisdesignedto\nenhancemolecule-levelrepresentationsthroughmultimodaldata. Thisdiscrepancymaylimitthesuitabilityofourmethodfor\nsuchgenerativetasks. Addressingthislimitationrepresentsapromisingdirectionforfuturework,forinstance,bydevelopinga\nmorefine-grainedmulti-to-unimodalknowledgetransferstrategytobetteraccommodateawiderrangeofapplicationscenarios.\nTableR8. ThemoleculecaptioningresultsonthetestsplitofChEBI-20.\nDecoder Encoder BLEU-2 BLEU-4 ROUGE-1 ROUGE-2 ROUGE-L METEOR\nMolT5-base 0.540 0.457 0.634 0.485 0.578 0.569\nMoMu 0.549 0.462 0.630 0.479 0.575 0.576\nMolT5-base GraphMVP 0.577 0.491 0.651 0.505 0.592 0.599\nMolFM 0.585 0.498 0.653 0.508 0.594 0.607\nM2UMol 0.577 0.488 0.640 0.492 0.581 0.598\nWe have added the above analysis on the performances of M2UMol and baselines in Section 3.1 of the revised\nSupplementaryInformation. TableR7hasbeenaddedtotherevisedSupplementaryInformationasTableS9.\nComment 3.3 Ensure consistent usage of method names and dataset references - e.g., \"DrugBank\" vs. \"Drugbank\"\n(Inconsistency);\"multimodal\"vs. \"multimodal.\"\nResponse3.3: Wehaveaddressedtheinconsistentusageofmethodnamesanddatasetreferences,andwedouble-checked\nourmanuscripttomakesuretherewerenosimilarmistakes.\nComment3.4 Relyingonasinglegenerated3Dconformercanbeproblematic,especiallyforlargeorflexiblemolecules\nwithmultiplebiologicallyrelevantconformations.\nResponse 3.4: We collect 3D conformers from DrugBank, which provides only a single low-energy 3D conformation\npermolecule. Althoughmoleculestypicallypossessmultiplelow-energy3Dconformations,priorresearch49hasshownthat\nmostmoleculesinDrugBanktendtoexhibitlinearandplanarstructures,withlimitedconformationaldiversity. Furthermore,\nwecomputethemeanmolecularweightandmeanrotatablebondsofmoleculesinourpre-trainingdatasets,andrevealthat\nthesemoleculesarerelativelysmall(meanmolecularweight=382Da)andexhibitlimitedflexibility(meanrotatablebonds=\n6). Thissuggeststhattheconformationalspaceofthesemoleculesisquiteconstrained,andthegeometricdifferencesamong\nlow-energyconformersaretypicallyminor. Therefore,usingasinglerepresentativeconformerissufficientforourmodelto\ncapturetheessential3Dstructuralfeatures. Inaddition,unlikesomeworksfocusingsolelyon2Dand3Dstructureintegration5,\nM2UMolincorporatesrichermodalities. Tobalancemodelcomplexityandcomputationalcost,wedidnotcollectadditional\n3Dconformationsformoleculesinourpre-trainingdatasets.\n15/18\nWehaveaddedthereasonforusingasingle3Dconformerformolecularpre-trainingtoSection1.1oftherevised\nSupplementaryInformation.\nReviewer #4 (Remarks to the Author):\nIco-reviewedthismanuscriptwithoneofthereviewerswhoprovidedthelistedreports. ThisispartoftheNatureCommunica-\ntionsinitiativetofacilitatetraininginpeerreviewandtoprovideappropriaterecognitionforEarlyCareerResearcherswho\nco-reviewmanuscripts. Iwasabletoeasilyinstalltheenvironmentandsuccessfullyusethepretrainedmodelfordownstream\ntasksasdescribedintheREADMEfile,includingDrug-DrugInteraction(DDI)andMolecularPropertyPrediction. Addition-\nally,thecodewasstraightforwardtoapplytocustomdatasets. TheREADMEprovidesclearinstructionsforinstallationand\nusage,makingitausableresourceforthecommunity. Theimplementationiswell-structured,andIencounterednomajor\nissueswhilerunningthecode.\nResponse4: Wesincerelyappreciateyourtimeandeffortinreviewingourmanuscript,aswellasyourvaluablefeedback.\nWearedelightedtohearthatyoufoundourcodeeasytoinstallanduse. Itisalsoencouragingtoknowthattheimplementation\nwasstraightforwardforcustomdatasetsandthattheREADMEprovidedclearinstructions. Wethankyouforyourpositive\nassessmentandaregladthatourworkcanserveasausefulresourcefortheresearchcommunity.\nReferences\n1. Wu,Z.etal. Moleculenet: abenchmarkformolecularmachinelearning. Chem.science9,513–530(2018).\n2. Zheng,Y.etal. Largelanguagemodelsforscientificdiscoveryinmolecularpropertyprediction. Nat.Mach.Intell.1–11\n(2025).\n3. Wan, Y., Wu, J., Hou, T., Hsieh, C.-Y. & Jia, X. Multi-channel learning for integrating structural hierarchies into\ncontext-dependentmolecularrepresentation. Nat.Commun.16,413(2025).\n4. Boulougouri,M.,Vandergheynst,P.&Probst,D. Molecularsetrepresentationlearning. Nat.Mach.Intell.6,754–763\n(2024).\n5. Stärk,H.etal. 3dinfomaximprovesgnnsformolecularpropertyprediction. InInternationalConferenceonMachine\nLearning,20479–20502(PMLR,2022).\n6. Liu, S. et al. Multi-modal molecule structure–text model for text-based retrieval and editing. Nat. Mach. Intell. 5,\n1447–1457(2023).\n7. Su,B.etal. Amolecularmultimodalfoundationmodelassociatingmoleculegraphswithnaturallanguage. arXivpreprint\narXiv:2209.05481(2022).\n8. Liu,S.etal. Pre-trainingmoleculargraphrepresentationwith3dgeometry. InInternationalConferenceonLearning\nRepresentations(2021).\n9. Zeng, Z., Yao, Y., Liu, Z. & Sun, M. A deep-learning system bridging molecule structure and biomedical text with\ncomprehensioncomparabletohumanprofessionals. Nat.communications13,862(2022).\n10. Liu,S.,Du,W.,Ma,Z.-M.,Guo,H.&Tang,J. Agroupsymmetricstochasticdifferentialequationmodelformolecule\nmulti-modalpretraining. InInternationalConferenceonMachineLearning,21497–21526(PMLR,2023).\n11. Luo, Y., Yang, K., Hong, M., Liu, X. & Nie, Z. Molfm: A multimodal molecular foundation model. arXiv preprint\narXiv:2307.09484(2023).\n12. Zhu,Y.etal. Featurizationsmatter: amultiviewcontrastivelearningapproachtomolecularpretraining. InICML20222nd\nAIforScienceWorkshop(2022).\n13. Yu,Q.etal.Multimodalmolecularpretrainingviamodalityblending.InTheTwelfthInternationalConferenceonLearning\nRepresentations.\n14. Wu,R.,Wang,H.,Chen,H.-T.&Carneiro,G. Deepmultimodallearningwithmissingmodality: Asurvey. arXivpreprint\narXiv:2409.07825(2024).\n15. Gu,Y.etal. Domain-specificlanguagemodelpretrainingforbiomedicalnaturallanguageprocessing. ACMTransactions\nonComput.forHealthc.(HEALTH)3,1–23(2021).\n16. Wolf,T.etal. Transformers: State-of-the-artnaturallanguageprocessing. InProceedingsofthe2020Conferenceon\nEmpiricalMethodsinNaturalLanguageProcessing: SystemDemonstrations, 38–45(AssociationforComputational\nLinguistics,Online,2020).\n17. Xu,K.,Hu,W.,Leskovec,J.&Jegelka,S. Howpowerfularegraphneuralnetworks? arXivpreprintarXiv:1810.00826\n(2018).\n18. Schütt,K.etal. Schnet: Acontinuous-filterconvolutionalneuralnetworkformodelingquantuminteractions. Adv.neural\ninformationprocessingsystems30(2017).\n19. Beltagy, I., Lo, K.&Cohan, A. Scibert: Apretrainedlanguagemodelforscientifictext. InProceedingsofthe2019\n16/18\nConferenceonEmpiricalMethodsinNaturalLanguageProcessingandthe9thInternationalJointConferenceonNatural\nLanguageProcessing(EMNLP-IJCNLP),3615–3620(2019).\n20. Luo,S.etal. Onetransformercanunderstandboth2d&3dmoleculardata. InTheEleventhInternationalConferenceon\nLearningRepresentations.\n21. Rampášek, L. et al. Recipe for a general, powerful, scalable graph transformer. Adv. Neural Inf. Process. Syst. 35,\n14501–14515(2022).\n22. Wang,L.,Liu,Y.,Lin,Y.,Liu,H.&Ji,S. Comenet: Towardscompleteandefficientmessagepassingfor3dmolecular\ngraphs.InKoyejo,S.etal.(eds.)AdvancesinNeuralInformationProcessingSystems,vol.35,650–664(CurranAssociates,\nInc.,2022).\n23. Gasteiger, J., Groß, J. & Günnemann, S. Directional message passing for molecular graphs. arXiv preprint\narXiv:2003.03123(2020).\n24. Liu,Y.etal. Sphericalmessagepassingfor3dmoleculargraphs. InInternationalConferenceonLearningRepresentations\n(ICLR)(2022).\n25. Devlin,J.,Chang,M.-W.,Lee,K.&Toutanova,K. Bert: Pre-trainingofdeepbidirectionaltransformersforlanguage\nunderstanding. arXivpreprintarXiv:1810.04805(2018).\n26. Lee,J.etal. Biobert: apre-trainedbiomedicallanguagerepresentationmodelforbiomedicaltextmining. Bioinformatics\n36,1234–1240(2020).\n27. Song,S.etal. Howtobridgethegapbetweenmodalities: Surveyonmultimodallargelanguagemodel. IEEETransactions\nonKnowl.DataEng.(2025).\n28. Chen,S.etal. Adaptformer: Adaptingvisiontransformersforscalablevisualrecognition. Adv.NeuralInf.Process.Syst.\n35,16664–16678(2022).\n29. Gao,P.etal. Clip-adapter: Bettervision-languagemodelswithfeatureadapters. Int.J.Comput.Vis.132,581–595(2024).\n30. Yang,L.,Zhang,R.-Y.,Wang,Y.&Xie,X. Mma: Multi-modaladapterforvision-languagemodels. InProceedingsofthe\nIEEE/CVFConferenceonComputerVisionandPatternRecognition,23826–23837(2024).\n31. Chen,G.,Shen,L.,Shao,R.,Deng,X.&Nie,L. Lion: Empoweringmultimodallargelanguagemodelwithdual-level\nvisualknowledge.InProceedingsoftheIEEE/CVFConferenceonComputerVisionandPatternRecognition,26540–26550\n(2024).\n32. Hu, W. et al. Open graph benchmark: Datasets for machine learning on graphs. Adv. neural information processing\nsystems33,22118–22133(2020).\n33. Long,Q.,Wang,M.&Li,L.Generativeimaginationelevatesmachinetranslation.InProceedingsofthe2021Conferenceof\ntheNorthAmericanChapteroftheAssociationforComputationalLinguistics:HumanLanguageTechnologies,5738–5748\n(2021).\n34. Zeng, Z., Yao, Y., Liu, Z. & Sun, M. A deep-learning system bridging molecule structure and biomedical text with\ncomprehensioncomparabletohumanprofessionals. Nat.communications13,862(2022).\n35. Chithrananda,S.,Grand,G.&Ramsundar,B. Chemberta: large-scaleself-supervisedpretrainingformolecularproperty\nprediction. arXivpreprintarXiv:2010.09885(2020).\n36. Irwin,R.,Dimitriadis,S.,He,J.&Bjerrum,E.J. Chemformer: apre-trainedtransformerforcomputationalchemistry.\nMach.Learn.Sci.Technol.3,015022(2022).\n37. Wang,Y.,Wang,J.,Cao,Z.&BaratiFarimani,A. Molecularcontrastivelearningofrepresentationsviagraphneural\nnetworks. Nat.Mach.Intell.4,279–287(2022).\n38. Hamilton, W., Ying, Z. & Leskovec, J. Inductive representation learning on large graphs. Adv. neural information\nprocessingsystems30(2017).\n39. Hu,W.etal. Strategiesforpre-traininggraphneuralnetworks. arXivpreprintarXiv:1905.12265(2019).\n40. Hu,Z.,Dong,Y.,Wang,K.,Chang,K.-W.&Sun,Y. Gpt-gnn: Generativepre-trainingofgraphneuralnetworks. In\nProceedingsofthe26thACMSIGKDDInternationalConferenceonKnowledgeDiscovery&DataMining,1857–1867\n(2020).\n41. Sun,F.-Y.,Hoffmann,J.,Verma,V.&Tang,J. Infograph: Unsupervisedandsemi-supervisedgraph-levelrepresentation\nlearningviamutualinformationmaximization. arXivpreprintarXiv:1908.01000(2019).\n42. Xu,M.,Wang,H.,Ni,B.,Guo,H.&Tang,J. Self-supervisedgraph-levelrepresentationlearningwithlocalandglobal\nstructure. InInternationalConferenceonMachineLearning,11548–11558(PMLR,2021).\n43. Rong, Y. et al. Self-supervised graph transformer on large-scale molecular data. Adv. Neural Inf. Process. Syst. 33,\n12559–12571(2020).\n44. You,Y.etal. Graphcontrastivelearningwithaugmentations. Adv.neuralinformationprocessingsystems33,5812–5823\n(2020).\n45. You,Y.,Chen,T.,Shen,Y.&Wang,Z. Graphcontrastivelearningautomated. InInternationalConferenceonMachine\nLearning,12121–12132(PMLR,2021).\n46. Fang,X.etal. Geometry-enhancedmolecularrepresentationlearningforpropertyprediction. Nat.Mach.Intell.4,127–134\n(2022).\n47. Zhu,J.etal. Unified2dand3dpre-trainingofmolecularrepresentations. InProceedingsofthe28thACMSIGKDD\nConferenceonKnowledgeDiscoveryandDataMining,2626–2636(2022).\n48. Ross,J.etal. Large-scalechemicallanguagerepresentationscapturemolecularstructureandproperties. Nat.Mach.Intell.\n4,1256–1264(2022).\n49. Prosser,K.E.,Stokes,R.W.&Cohen,S.M. Evaluationof3-dimensionalityinapprovedandexperimentaldrugspace.\nACSmedicinalchemistryletters11,1292–1298(2020).\n18/18"}, {"round": 2, "reviewer_comments": {"reviewer_1": "I appreciate the response by authors and the response has been detailed except i have few comments to add:\n1. The use of single linear layers as modal-specific adapters imposes a strong inductive bias, assuming a near-linear relationship between the embedding spaces of diferent modalities. This may be an insufficient architectural choice for bridging the gap between the topological space of 2D graphs and the high-dimensional, semantic spaces of text or biochemical features. An analysis of the adapters representational capacity, perhaps by examining the svd of the weight matrices or comparing against a lightweight non-linear alternative, would be necessary to justify this design choice beyond just performance improvement. The resulting \"pseudo-representations\" are just linear projections of the 2D embedding, not genuine multimodal features. The paper fails to demonstrate that this \"fusion\" provides any advantage over simply using a deeper or more expressive 2D-only encoder that could learn similarly complex features without the multimodal pretext.\n2. The ablation studies show that adding modalities improves performance, but they fail to demonstrate synergy. There is no evidence that learning from the \"Bio\" modality improves the model's ability to learn from the \"Text\" modality, or vice-versa. The improvements could be purely additive, with each modality independently providing a separate learning signal. A critical missing experiment is a quantitative analysis showing that the quality of one pseudo-representation (e.g., pseudo-Text Recall@K) is improved by the presence of another modality (e.g., Bio) during pre-training. Without this, the framework is just an ensemble of independent 2D-to-X mappings.\n3. multi-to-uni knowledge transfer framing is questionable. The architecture is isomorphic to a multi-task learning setup where a shared 2D encoder is regularized by multiple modality-prediction heads (the adapters). The core claim of transferring knowledge lacks a crucial baseline: a simpler model where the 2D encoder is trained simultaneously on the downstream task and a generative or contrastive loss for each auxiliary modality, without the explicit fine-tuning fusion step. Without this comparison, the necessity of the proposed complex framework over a standard multi-task regularization scheme is bit doubtful in my undersytanding.\n4. Biochemical feature space is constructed from fixed vocabulary of known targets, enzymes, and drug categories from DrugBank. This closed-type world assumption limits the model's utility in real-world discovery pipelines where novel biological entities are common. The authors should address the framework's robustness to out-of-vocabulary entities and clarify whether the model can generalize or would require complete retraining when encountering new targets or enzyme classes not present in the pre-training dataset.", "reviewer_2": "I have reviewed the revised version of the manuscript and appreciate the authors’ efforts in addressing the previous comments. The revisions are appropriate and have improved the clarity and quality of the work. I have no further concerns and recommend acceptance.", "reviewer_3": "The manuscript is very well written, with the content presented clearly and accurately. No further revisions are required.", "reviewer_4": "I co-reviewed this manuscript with one of the reviewers who provided the listed reports. This is part of the Nature Communications initiative to facilitate training in peer review and to provide appropriate recognition for Early Career Researchers who co-review manuscripts.\n(Remarks on code availability)\nAs stated previously, I was able to install the environment and utilize the pretrained model. The implementation is well-structured, and I encountered no major issues while running the code."}, "author_response": "Reviewer #1 (Remarks to the Author):\nIappreciatetheresponsebyauthorsandtheresponsehasbeendetailedexceptIhaveafewcommentstoadd:\nComment 1.1 The use of single linear layers as modal-specific adapters imposes a strong inductive bias, assuming a\nnear-linear relationship between the embedding spaces of different modalities. This may be an insufficient architectural\nchoiceforbridgingthegapbetweenthetopologicalspaceof2Dgraphsandthehigh-dimensional,semanticspacesoftextor\nbiochemicalfeatures. Ananalysisoftheadapter’srepresentationalcapacity,perhapsbyexaminingtheSVDoftheweight\nmatricesorcomparingagainstalightweightnon-linearalternative,wouldbenecessarytojustifythisdesignchoicebeyondjust\nperformanceimprovement. Theresulting\"pseudo-representations\"arejustlinearprojectionsofthe2Dembedding,notgenuine\nmultimodalfeatures. Thepaperfailstodemonstratethatthis\"fusion\"providesanyadvantageoversimplyusingadeeperor\nmoreexpressive2D-onlyencoderthatcouldlearnsimilarlycomplexfeatureswithoutthemultimodalpretext.\nResponse1.1: Wethankthereviewerforraisingtheconcernregardingtheinductivebiasintroducedbyusingasinglelinear\nlayerasmodal-specificadapters. Notably,manyexistingmultimodallearningmethodshaveemployedasinglelinearlayerasa\nlightweightbridgebetweendifferentmodalities1,2,3,4,suchasCLIP1andLLaVA2. Theempiricalsuccessofthesemethods\ndemonstrates that a single linear layer can be sufficient to bridge modality embedding spaces. Building on this evidence,\nourexperimentsinthesection\"M2UMolcangeneratereliablemultimodalrepresentationsfrom2Dmoleculegraphs\"ofthe\nmaintextfurthervalidatetheeffectivenessoftheproposedadapters. AsshowninFigure3a,throughtheadapters,M2UMol\ncansolelyutilize2Drepresentationstogeneratepseudo-representationsfordifferentmodalities,whicharewelldistributedin\nmodality-specificregionsandcloselyresembletheactualmodalitydistributions(thepseudo-representationscansuccessfully\nretrievethecorrectrealmodalityrepresentationswithhighRecall@k). Theseresultsindicatethattheadapterisindeed\ncapable of establishing meaningful cross-modal correspondences. In addition, the ablation study on adapters further\nconfirmsthattheyplayacrucialrolebothinbridgingmodalitiesduringpre-trainingandingeneratingeffectivemultimodal\npseudo-representationsfordownstreamtasks. Together,theseresultssupportthesoundnessofourdesignchoiceforthe\nmodal-specificadapters.\nAssuggestedbythereviewer,wealsoexploredavariantofM2UMol,termedM2UMol(non-linearadapters),inwhicha\nnon-linearactivationlayerwasaddedaftereachadapterintheoriginalM2UMol. Wefirstevaluatedthisvariantonmolecular\npropertypredictiontasks(FigureR1)andfoundthatitsperformancewasgenerallyinferiortothatoftheoriginalM2UMol. To\nbetterunderstandthisdegradation,wefurtherassessedM2UMol(non-linearadapters)onthemultimodalretrievaltask(defined\ninthefirstparagraphofthesection“M2UMolcangeneratereliablemultimodalrepresentationsfrom2Dmoleculegraphs”\nofthemaintext). AsshowninTableR1,theintroductionofnon-linearityledtoreducedperformanceacrossmostmetrics.\nTheseconsistentresultsindicatethat,despitetheirapparentsimplicity,linearadaptersareinfactmoresuitableforour\ncurrentmultimodalpre-trainingframework. Inaddition,therearevariousadapterstructuresavailable,andtheremaybe\nbetteroptionstochoosefrom. However,weultimatelychosearelativelysimplebuteffectiveandsuitableadapterarchitecture.\nRegardingtheconcernthat“thepseudo-representationsarejustlinearprojectionsofthe2Dembedding,notgenuinemulti-\nmodal features,” we would like to clarify that our multi-to-uni modal knowledge transfer pre-training explicitly employs\nFigure R1. The results of the ablation study to evaluate the influence of the non-linear adapters. The variant M2UMol\n(non-linearadapters)isconstructedbyaddinganon-linearactivationlayeraftereachadapteroftheoriginalM2UMol.\n1/5\nTableR1. ComparisonofM2UMolandM2UMol(non-linearadapters)onthemultimodalretrievaltask.\nRecall@1 Recall@3 Recall@5 Recall@10\nMethod\n3D Text Bio 3D Text Bio 3D Text Bio 3D Text Bio\nM2UMol(non-linearadapters) 0.75 0.65 0.30 0.92 0.84 0.48 0.96 0.90 0.58 0.99 0.96 0.71\nM2UMol 0.73 0.66 0.32 0.91 0.86 0.51 0.95 0.92 0.60 0.99 0.97 0.73\ncontrastive learning to align the distribution of generated pseudo-representations as close as possible to that of the real\nmultimodalrepresentations. Throughthisprocess,thepseudo-representationscancapturemodality-specificinformationfrom\ntherealmodalities,ratherthanbeingmerelylinearprojectionsofthe2Dembeddings. Wehavedemonstratedexperimentally\nthatthegeneratedpseudo-representationsderivedsolelyfrom2Ddatacanaccurately(i)retrieverealmultimodalrepresentations\ninthemultimodalretrievaltasks(Figure3a),and(ii)retrieverealmultimodaldatainthecasestudies(Figure3b),aspresented\ninthesection“M2UMolcangeneratereliablemultimodalrepresentationsfrom2Dmoleculegraphs”. confirmingthatthe\ngeneratedpseudo-representationsembodyreliablemultimodalinformation. Thoughweacknowledgethatthegeneratedmulti-\nmodalrepresentationscannotreplacetherealisticmultimodalrepresentations,theycanprovidemultimodalinformation\nwhenonlya2Dmodalityisavailable,whichistypicalindownstreamtasks,andachievebettergeneralizationperformance\nandpredictionaccuracy.\nRegardingtheconcernthattheproposed“fusion”maynotprovideadvantagesoveradeeperormoreexpressive2D-only\nencoder,wefirstemphasizethatourworkisprimarilymotivatedbytherecenttrendtowardmultimodalpre-traininginmolecular\nrepresentationlearning(MRL).Ourframeworkisdesignedtoaddresstwokeypracticalchallengesinthissetting: incomplete\nmodalitiesduringpre-trainingandunimodaldownstreamtasks. Thus,anexhaustivecomparisonwithpurely2Dencoders\nis not the main focus of our study, given their inherent limitations in generalization and their reliance on a single\nsourceofinformation. Nevertheless,wehaveprovideddirectevidencethatourapproachoutperforms2D-onlyencoders.\nAsshowninTable1ofthemaintextandTableS8oftheSupplementaryInformation,M2UMolsignificantlyoutperforms\nM2UMol ,whichisequivalenttothepure2DencoderGraphGPS5(itselfalreadyhasbeenprovenstrongerthanmost2D\nunpre\nencoderssuchasGIN6,PNA7,andGraphormer8). Inaddition,M2UMolalsosurpassesbaselinemethodsthathaveemployed\nhighlyexpressiveanddeepmolecularencoders(e.g.,MOLEBLEND9,12graphtransformerlayers; Molformer-XL10,12\ntransformerlayers;Uni-Mol11,15graphtransformerlayers). Thesehighlighttheadvantageofourmultimodalpre-training\nframeworkover2D-onlyrepresentationlearning,andweexpectthatfutureadvancesin2Dencoderscanalsofurtherbenefit\nfrombeingintegratedintoourframework. Regardingtheeffectivenessofthe\"fusion\"mentionedbytheauthor,FigureS3bin\ntheSupplementaryInformationshowsthatM2UMoloutperformsthevariantM2UMol(w/oadaptersinfine-tuning),which\nremovesthemodal-specificadaptersinthefine-tuningphase,andonlythe2Dencoderrepresentationsareusedwithoutfusing\nthegeneratedmultimodalpseudo-representations. Thisfurtherconfirmsthatthepseudo-representationscapturecomplementary\nmodality-specificinformationandthattheirfusionwith2Drepresentationsenablesthemodeltoleveragemultimodal\ninteractions,ultimatelyimprovingdownstreamperformance.\nTheanalysisoftheablationstudyhasbeenaddedtoSection3.2oftherevisedSupplementaryInformation. Figure\nR1hasbeenaddedtotherevisedSupplementaryInformationasFigureS3.\nComment1.2 Theablationstudiesshowthataddingmodalitiesimprovesperformance,buttheyfailtodemonstratesynergy.\nThereisnoevidencethatlearningfromthe\"Bio\"modalityimprovesthemodel’sabilitytolearnfromthe\"Text\"modality,or\nvice-versa. Theimprovementscouldbepurelyadditive,witheachmodalityindependentlyprovidingaseparatelearningsignal.\nAcriticalmissingexperimentisaquantitativeanalysisshowingthatthequalityofonepseudo-representation(e.g.,pseudo-Text\nRecall@K)isimprovedbythepresenceofanothermodality(e.g.,Bio)duringpre-training. Withoutthis,theframeworkisjust\nanensembleofindependent2D-to-Xmappings.\nResponse1.2: Wethankthereviewerforraisingthisimportantpointregardingmultimodalsynergy. Wewanttoclarifythat\ntheprimaryobjectiveofM2UMolistoenable2Dmolecularrepresentationsalongwithmodality-specificadapterstoabsorb\nandretainknowledgefromdiversemodalities. Thisdesignallowsdownstreamtasks,whereonly2DinputssuchasSMILES\naretypicallyavailable,tobenefitfromtherichermultimodalknowledge. Fromthisperspective,themultimodalsynergywe\nmainlypursueliesinenhancingthe2Dmodalityitselfbyusingvariousmodalities(3D,Text,Bioenhances2D),rather\nthaninimprovingmutualunderstandingacrossmodalities(e.g.,BioenhancesText,3DenhancesText).\nNevertheless,motivatedbythereviewer’ssuggestion,wefurtherinvestigatedwhetherincorporatingadditionalmodalities\ncouldfacilitatemutualunderstandingacrossmodalities. Tothisend,weconductamultimodalretrievalanalysis,whichis\nanextensionofthemultimodalretrievaltaskintroducedinSection“M2UMolacquiresdiverseknowledgefrommultimodal\n2/5\ndatainpre-training”ofthemaintext. Specifically, wedividedtheretrievaltasksintothreegroupsaccordingtothetarget\nmodality(Text,Bio,and3D)andcomparedthepseudo-representationsgeneratedbythefullpre-trainedM2UMolwiththose\nfromablatedvariantsthatonlyalign2Dwiththetargetmodality(e.g.,intheTextgroup,M2UMolandM2UMol(2D+Text)\narecompared). TheevaluationisalsobasedonRecall@KwithoptionalKin{1,3,5,10}. WealsotrackedtheRecall@K\nperformancethroughoutthepre-trainingprocesstobettercapturethedynamicsofmultimodallearning.\nFigureR2. MultimodalretrievalanalysisofM2UMolandablatedvariants. Thequalityofthegeneratedpseudorepresentation\nisassessedbyRecall@KonretrievaltaskstargetingText,Bio,and3Dmodalities. Eachpanelcomparesthefullpre-trained\nM2UMolwiththecorrespondingablatedmodel(2D+Text,2D+Bio,or2D+3D)acrosstrainingepochs.\nThe results in Figure R2 show that for the Text and Bio modalities, the pseudo representations generated by the full\npre-trainedM2UMolareofhigherqualitythanthoseproducedbytheablatedvariants,acrossalmostallRecall@Kvalues\nandtrainingepochs. Thisprovidesdirectquantitativeevidencethatlearningfrommoremodalitiescanalsoenhance\nthemodel’sabilitytolearnfromonemodalityandgeneratebetterpseudo-representations. Itconfirmsthepresenceof\nmultimodalsynergymentionedbythereviewer. Forthe3Dmodality,wecanobserveaslightdecreaseinRecall@Kunder\nthefullpre-trainedM2UMol. Weattributethistothefactthat2Dand3Dmodalitieshavecarriedhighlysimilarstructural\ninformation,soaddingothermodalitiesyieldslimitedroomforimprovementandmayinsteadinducenoise,whichcancause\nsmallfluctuations. However,theabsoluteRecallvaluesremainveryhigh(Recall@5andRecall@10arebothgreaterthan0.9).\nThisindicatesthatthefullpre-trainedM2UMolstillgeneratesreliable3Dpseudo-representationsfrom2Dinputs,andthe\nminorfluctuationhaslittlepracticalimpactonourmodel.\nWehaveaddedtheresultsandanalysistoSection3oftherevisedSupplementaryInformation. FigureR2hasbeen\naddedtotherevisedSupplementaryInformationasFigureS4.\nComment 1.3 Multi-to-uni knowledge transfer framing is questionable. The architecture is isomorphic to a multi-task\nlearningsetupwhereashared2Dencoderisregularizedbymultiplemodality-predictionheads(theadapters). Thecoreclaim\noftransferringknowledgelacksacrucialbaseline: asimplermodelwherethe2Dencoderistrainedsimultaneouslyonthe\ndownstreamtaskandagenerativeorcontrastivelossforeachauxiliarymodality,withouttheexplicitfine-tuningfusionstep.\nWithoutthiscomparison,thenecessityoftheproposedcomplexframeworkoverastandardmulti-taskregularizationschemeis\nabitdoubtfulinmyunderstanding.\nResponse 1.3: Thanks for the reviewer’s comment. A key contribution of M2UMol lies in addressing the critical\nlimitationinreal-worldapplications: indownstreamtasks, typicallyonlySMILESor2Dmolecularstructuresare\navailable,whilemultimodalinformationsuchas3Dconformations,textualdescriptions,orbiochemicalfeaturesisrarely\n3/5\naccessible. To overcome this, our framework M2UMol aligns 2D modality with multiple modalities in the representation\nspaceinthepre-trainingphase,enablingthemodal-specificadapterstoacquiretheabilitytoconvert2Drepresentationsinto\npseudomultimodalrepresentations. Asaresult,fordownstreamtasksthatonlySMILESor2Dmolecularstructuresaregiven,\nthepre-trainedadapterscangeneratepseudomultimodalrepresentationsfrom2Drepresentationstosupplementmultimodal\ninformationandenhancethedownstreamtasks.\nCrucially,M2UMolisfundamentallydifferentfromthe“multi-taskregularizationscheme”mentionedbythereviewer.\nMulti-task regularization requires multimodal supervision during downstream learning, which is impractical in real-world\ndrugdiscoverywheremultimodalannotationsarescarceandcostlytoobtain. Bycontrast, M2UMoltransfersmultimodal\nknowledge entirely during pre-training, and downstream tasks operate solely on unimodal data. Thus, M2UMol is not a\nregularizedmulti-tasksetup,butratheraknowledgetransfermechanismthatbridgesmultimodalpre-trainingwithunimodal\nfine-tuning,enablingdownstreamtaskstobenefitfrommultimodalinsightswithoutrequiringmultimodalinputsthemselves.\nFinally, while we appreciate the reviewer’s suggestion, we note that the baseline suggested by the reviewer implicitly\nassumestheavailabilityofmultimodaldataindownstreamtasks. However,suchmultimodalfeaturesaretypicallyunavailable\nin standard benchmarks (e.g., MoleculeNet) and would require additional manual collection. Therefore, we believe this\ncomparisonmaynotprovideafairassessmentofourmethod,whichisexplicitlydesignedtoaddressthelackofmultimodal\nsupervisionindownstreamtasks.\nComment 1.4 Biochemical feature space is constructed from a fixed vocabulary of known targets, enzymes, and drug\ncategoriesfromDrugBank. Thisclosed-typeworldassumptionlimitsthemodel’sutilityinreal-worlddiscoverypipelineswhere\nnovelbiologicalentitiesarecommon. Theauthorsshouldaddresstheframework’srobustnesstoout-of-vocabularyentities\nandclarifywhetherthemodelcangeneralizeorwouldrequirecompleteretrainingwhenencounteringnewtargetsorenzyme\nclassesnotpresentinthepre-trainingdataset.\nResponse1.4:Wethankthereviewerforthisthoughtfulcomment.WewouldliketoclarifythatinourframeworkM2UMol,\nthevocabularyoftargets,enzymes,anddrugcategoriesfromDrugBankisusedsolelyduringthepre-trainingphasetoconstruct\nbiochemicalfeatures,andisnotrequiredduringdownstreaminference. Consequently,thepresenceofout-of-vocabulary\n(OOV)biologicalentitiesdoesnotimpedeinferencefordownstreamtasks,butonlyinfluencesthebreadthofbiochemical\nknowledgecapturedduringpre-training.\nFigure R3. Comparisonofthenumbersofsmallmolecules, theirassociateddrugtargets, anddrugenzymesindifferent\nversionsofDrugBank.\nFurthermore,the“targets”and“enzymes”consideredinoursettingcorrespondtodrug-relatedentities,whosediscovery\nandvalidationtypicallydemandextensivepharmacologicalandbiochemicalstudies,resultinginrelativelyslowexpansion\novertime. Wehavecomparedthenumbersofsmallmolecules,targets,andenzymesindifferentversionsofDrugBank12,13,\nwhich are shown in Figure R3. We can find that although the number of small molecules in the DrugBank has increased\nrapidlyacrossversions,thegrowthinassociatedtargetsandenzymeshasbeenrelativelymodest,typicallybyonlytenstoa\nfewhundrednewentriesperversioninDrugBank. Consequently,M2UMolcanremainbroadlyapplicablewithoutrequiring\n4/5\nfrequentre-trainingtocapturenewlyaddedentries.\nFinally,thepre-trainingprocedureofM2UMoliscomputationallyefficientandcanbecompletedwithin11hours,making\nre-training with an expanded vocabulary straightforward. These factors together suggest that our approach is robust and\npracticalforreal-worlddrugdiscoveryscenarios. Weappreciatethereviewer’sforward-lookingperspective,andinfuturework,\nwemayconsiderdynamicpre-trainingstrategies(e.g.,vocabularyexpansionorcontinualpre-training)tobetterhandlethe\ngradualexpansionofbiologicalentities.\nReferences\n1. Radford,A.etal. Learningtransferablevisualmodelsfromnaturallanguagesupervision. InInternationalconferenceon\nmachinelearning,8748–8763(PmLR,2021).\n2. Liu,H.,Li,C.,Wu,Q.&Lee,Y.J. Visualinstructiontuning. Adv.neuralinformationprocessingsystems36,34892–34916\n(2023).\n3. Chen,K.etal. Shikra: Unleashingmultimodalllm’sreferentialdialoguemagic. arXivpreprintarXiv:2306.15195(2023).\n4. Koh,J.Y.,Salakhutdinov,R.&Fried,D. Groundinglanguagemodelstoimagesformultimodalinputsandoutputs. In\nInternationalConferenceonMachineLearning,17283–17300(PMLR,2023).\n5. Rampášek, L. et al. Recipe for a general, powerful, scalable graph transformer. Adv. Neural Inf. Process. Syst. 35,\n14501–14515(2022).\n6. Xu,K.,Hu,W.,Leskovec,J.&Jegelka,S. Howpowerfularegraphneuralnetworks? arXivpreprintarXiv:1810.00826\n(2018).\n7. Corso,G.,Cavalleri,L.,Beaini,D.,Liò,P.&Velicˇkovic´,P. Principalneighbourhoodaggregationforgraphnets. Adv.\nneuralinformationprocessingsystems33,13260–13271(2020).\n8. Ying, C. et al. Do transformers really perform badly for graph representation? Adv. Neural Inf. Process. Syst. 34,\n28877–28888(2021).\n9. Yu,Q.etal.Multimodalmolecularpretrainingviamodalityblending.InTheTwelfthInternationalConferenceonLearning\nRepresentations.\n10. Ross,J.etal. Large-scalechemicallanguagerepresentationscapturemolecularstructureandproperties. Nat.Mach.Intell.\n4,1256–1264(2022).\n11. Zhu,J.etal. Unified2dand3dpre-trainingofmolecularrepresentations. InProceedingsofthe28thACMSIGKDD\nConferenceonKnowledgeDiscoveryandDataMining,2626–2636(2022).\n12. Wishart, D. S. et al. Drugbank 5.0: a major update to the drugbank database for 2018. Nucleic acids research 46,\nD1074–D1082(2018).\n13. Knox,C.etal. Drugbank6.0: thedrugbankknowledgebasefor2024. Nucleicacidsresearch52,D1265–D1275(2024).\n5/5"}, {"round": 3, "reviewer_comments": {"reviewer_1": "Thanks to the authors for responding to my comments and i dont have any further comments."}, "author_response": ""}]

Decision letter