Self-supervised Blending Structural Context of Visual Molecules for Robust Drug Interaction Prediction
Authors: Tengfei Ma
Kun Chen
Yongsheng Zang
Yujie Chen
Xuanbai Ren
Bosheng Song
hongxin xiang
Yiping Liu
xiangxiang Zeng
Venue: NeurIPS 2025 poster ()
Keywords: Drug Interaction Prediction
Drug Discovery
Molecule Representation Learning
Primary area: machine_learning_for_sciences
TL;DR: S²VM self-supervised learning from large unlabeled drug pairs, achieving SOTA DDI prediction and superior generalization/interpretability on novel/few-shot drugs.

Abstract: Identifying drug-drug interactions (DDIs) is critical for ensuring drug safety and advancing drug development, a topic that has garnered significant research interest. While existing methods have made considerable progress, approaches relying solely on known DDIs face a key challenge when applied to drugs with limited data: insufficient exploration of the space of unlabeled pairwise drugs.  To address these issues, we innovatively introduce S$^2$VM, a Self-supervised Visual pretraining framework for pair-wise Molecules, to fully fuse structural representations and explore the space of drug pairs for DDI prediction. S$^2$VM incorporates the explicit structure and correlations of visual molecules, such as the positional relationships and connectivity between functional substructures. Specifically, we blend the visual fragments of drug pairs into a unified input for joint encoding and then recover molecule-specific visual information for each drug individually. This approach integrates fine-grained structural representations from unlabeled drug pair data. By using visual fragments as anchors, S$^2$VM effectively captures the spatial information of local molecular components within visual molecules, resulting in more comprehensive embeddings of drug pairs. Experimental results show that S$^2$VM achieves state-of-the-art performance on widely used benchmarks, with Macro-F1 score improvements of 4.21% and 3.31%, respectively. Further extensive results and theoretical analysis demonstrate the effectiveness of S$^2$VM for both few-shot and novel drugs.

Self-supervised Blending Structural Context of Visual Molecules for Robust Drug Interaction Prediction
Tengfei Ma" Kun Chen' Yongsheng Zang Yujie Chen
Xuanbai Ren Hongxin Xiang Yiping Liu Xiangxiang Zengi
Bosheng Song
State Key Laboratory of Chemo and Biosensing. College of Computer Science and Electronic Engineering, Hunan University, Changsha 410082, China The Ministry of Education Key Laboratory of Fusion Computing of Supercomputing and Artificial Intelligence, Hunan University, Changsha 4 410082, China. tfma@hnu. edu. cn, xzeng@hnu. edu. cn
Abstract
Identifying drug-drug interactions (DDIs) is critical for ensuring drug safety and advancing drug development, topic that has garnered significant research interest. While existing methods have made considerable progress, approaches relying solely on known DDIs face a key challenge when applied to drugs with limited data (c.g., novel and few-shot drugs): insufficient exploration of the space of unlabeled pairwise drugs. To address these issues, we innovatively introduce S Self- supervised Visual pretraining framework for pair-wise Molecules, to fully fuse structural representations and explore the space of drug pairs for DDI prediction. S²VM incorporates the explicit structure and correlations of visual molecules, such as the positional relationships and connectivity between functional substructures. Specifically, we blend the visual fragments of drug pairs into unified input for joint encoding and then recover molecule-specific visual information for each drug individually. This approach integrates fine-graine structural representations from unlabeled drug pair data. By using visual fragments as anchors, S2VM effectively captures the spatial information of local molecular components within visual molecules, resulting in more comprehensive embeddings of drug pairs. Experimental results show that S2VM achieves state-of-the performance widely used benchmarks, with Macro-F1 score improvements of 4.21% and 3.31 %, respectively. Further extensive results and theoretical analysis demonstrate the effectiveness of S²VM for both few-shot and novel drugs. The code and data are available http::/ggitbbb.coo/iimmmaaaaaa
Introduction
Combinatorial therapy, which involves the simultancous use of multiple drugs, is promising strategy for treating patients with complex diseases [1] []. However, this approach poses challenges due to potential drug-drug interactions (DDIs) that can alter the intended therapeutic outcomes. When patients take multiple drugs at the same time, these interactions
result in unexpected side effects or diminished clinical efficacy [3] 4 Therefore, accurately predicting DDIs is essential to avoid critical task in the common therapeutic field (5). Despite
potential adverse effects, making ongoing efforts, predicting these interactions remains significant challenge.
Numerous computational prediction methods have been developed address these challenges to predict unknown drug-dit -rug interaction (DDI) events [ 7 Many of these methods use handcrafted
Equual contribution
Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

features (e.g., molecule structure 8 9 11, side effects [TO], and phenotypic similarity [TT] to repre- sent each drug for predicting potential DDIs. However, these methods rely heavily on expert domain knowledge to design these features accurately. To address this, some approaches use deep learning models to extract low-dimensional features from molecular sequences, learning representations Additionally, to represent drug structures from functional perspective, several works [14 15 16 extract molecular substructures and employ graph neural networks to model the associations between drug pairs, resulting in promising predictive performance. However, they primarily focus on molecular features, neglecting other biological entities involved in drug interaction events, such proteins, pathways, and diseases, which are crucial to identifying DDIs. Recent works [17 18 19 20
from SMILES in an end-to-end manner [
13
21 22 23 have taken advantage the se- mantic relations and topological structures of biomedical knowledge graphs improve the structural representation of molecules for accurate prediction of DDL. While these methods have achieved some improvements, they primarily predict unknown DDIs by learning drug representations from known DDIs, which are limited to novel and few-shot drugs due to the challenge: limited exploration for the space of drug pairs from huge unlabeled data. As illustrated in Figure previous methods mainly represent drug pairs by concatenating the molecular embeddings from individual drug encoders, which were trained on existing DDIs (Figure [p)) resulting in weak structural fusion and exploration capabilities for broad unknown drug pairs. We provide more detailed discussion in Appendix C.I
To address these limitations, propose self-supervised pretraining framework (called V²VM) to learning from over 200M drug pairs, designed to encode input drugs jointly by capturing both intrinsic struc- tures and extrinsic interactions between molecules. Specifically, S2VM first samples and blends input drugs based on their lo- cal visual fragments for joint encoding by the drug encoder. Then, S introduces a de- coder to reconstruct the orig- inal visual structure of input molecules from the blended rep- resentation. This reconstruction process establishes structural cor-
SVVM
Unexplored Drug Pairs
Known DDIs
ASSS Visulization
DayOnt ImageMid
outperforms Deng's
Previous
Methods
Observed Space of Drug Pairs
ImageMol
SPV
S7VM performs better represontration across 8 types of DDIs
(a) S*VM explores broad space
(b) SPVM shows superior performance
Figure 1: (a) SPVM explores a comprchensive space of drug pairs for existing drugs. (b) The self-supervised V2 shows superior performance and representations.
relations between input drugs using molecular visual information in a self-supervised manner. The pretrained encoder is subsequently adopted for DDI prediction. Empirical observations (Figure [p)) and theoretical (Section 4.2) analysis indicate that S2VM designed for effective structural represen- tation of drug pairs, exhibiting superior exploration capabilities compared the visually pretrained molecule representation model ImageMol [24). Our contributions include: (1) To the best of our knowledge, are the first develop self-supervised pretraining model based large-scale unlabeled drug pairs that jointly encodes the visual structural relations of drug pairs for DDI predic- tion. (2) By representing the blended visual fragments of observed paired molecules and recovering their original visual structures, V²VM effectively captures extrinsic relations and intrinsic structures between molecules from both experimental and theoretical perspectives.
(3) Through theoretical analysis and empirical validation, we demonstrate that S²VM effectively integrates visual structural relationships across diverse drug pairs, achieving state-of-the-ar performance in DDI prediction under various scenarios.
Related Work
Drug Interaction Prediction. Identifying potential drug interaction events is crucial to drug discovery.
Some works mainly adopt handcraft features of molecules to predict unknown DDIs 8 9 However, these handcraft features are limited by reliance on domain knowledge of drugs 25 suffering
from low expressive ability (26). DeepDDI [2] and CASTER [3] utilize deep learning models to

---

mine low-dimensional representations of drugs and predict the interaction associations between imput
drug pairs. Further. SSI-DDI [14] SA-DDI [27], and DSN-DDI [6] proposed substructure-basee
GNN and fused the representation of molecules based on substructures adaptively. However, these
methods overlook the drug-related knowledge from biomedical networks (20 17 To model the
structure of molecules and the interactive information of drugs, MUFFIN [18 and SumGNN [99
adopt GNN (28) to represent the molecular structure and the relational semantics of the biomedical knowledge graph. To further represent the interactive association between drugs, MRCGNN [21] and TIGER [22] utilized a shared encoder with a contrastive learning mechanism to integrate the structural information of molecules and the information of multi-relational DDI events. However, they learn separate drug inputs from known DDIs, limited by modeling the structural relations between them. We innovatively designed supervssd pretraining framework to introduce unified model to represent huge unlabeled drug pairs jointly.
Representation of Visual Molecules. The molecular images are intuitive in representing spatial information such as the positional relations and connectivity between funetional substructures. Some researchers consider representing molecules images and adopting computer vision techniques to extract features for chemical properties prediction [29 30). To effectively understand the structural information of visual molecules, ImageMol [24 proposed pretraining model based on molecular
images to learn representation from 10 million molecules. To enhance the image representation of visual molecules, CGIP [3]] is further proposed to model the molecular graphs and images in a contrastive manner. Although vision-
molecular representation has shown excellent performance, these methods are limited to modeling the paired molecules simultaneously. In this paper, we design a novel architecture to pretrain a unified encoder for representing the paired drugs.
3 Preliminaries
Background. In this paper. we use RDKit to convert molecular SMILES 332 into visual molecules (i.c., 2D images). The molecular images contain more spatial information (c.g., the positional rela- tionships between functional groups and atoms), which are intuitive and informative for representing molecules. Therefore, we consider molecular images as our inputs.
Structure-level Visual Pretraining. In order to explore the structure-level representation fusion of drugs, we design a self-supervised pretraining framework based on visual fragments. Specifically, the framework is in encoder-decoder architecture, which contains transformer-based encoder J and decoder F. Given a pair of drugs (du, dy), where dy E xxxxxxx XC and de E R W xC are the visual molecules converted by RDKit, our goal is learn the structural fusion embedding êu as follows:
z = F(blend( du, do du, d, = F(z);
2 arg min A(dy, d.) A(dr, do),
where (H, C) is the resolution of input images and the pretrained encoder F is adopted to embed inputs for downstream DDI prediction.
Problem Definition. We focus on predicting the potential drug interaction events between drugs. The prediction is achieved by blending input drug pairs and fusing the substructure-based representation based on visual molecules. We formulate the visual-based DDI prediction as a multi-classificatin task, aiming to estimate the probability of corresponding interaction events. Specifically, given pair of drugs (d,, d.), we propose a model to identify the interaction event denoted as ((d, [(ddu F).
4 Method
4.1
Proposed V
Overview. V aims to learn the essential representations of input drugs (i.e., a pair of molecules) that possess inherent connections while appearing distinctive between different molecules. Specifi- cally, S proposes an image-based self-supervised framework to pretrain a unified encoder for representing a pair of molecules. As illustrated in Figure 2 VM mainly consists of four compo- nents: (a) Structure-level encoding module encodes the input drugs into sequence of visual tokens;
3

---

frozen
(d) DDI Prediction
Heads
(c) Drug Reconstruction
Head
Recover
Frageetation
Pmggertaioon
cancedding
Lineer diu Capa
Rexoner
Visun Ensudda Layer
Oupps
Visior Layar
(b) Pairwise Drug Encoder-Decodde
2144
Eaceder Module
Deceder Module
*
Vision Encoder Layer
(llnding
Vision Encoder Layer
Visie Decoda Layer
Semetture level Lincoding
Tokesiees
Faraderr I ayer
Vision ayer
Vision Eioooeer Layer
Vision Decoder Layer
(n) Structure-level Encoding
Iapul
Figure 2: VM consists of four components: (a) To fuse the drug pairs into unified input, sample and blend them into structural tokens (i.e., fragments of visual molecules); (b) We feed the structural tokens into a vision-based Imcoder-Decodee
model the semantic relations of molecular fragments; (c) To promote the structural fusion of drug pairs, we set a reconstruction operation to recover the input drugs; (d) The pretrained encoder is adopted to predict potential drug interactions.
(b) The Pairwise Drug Encoder-decoder architecture embeds the sequence of visual tokens; (c) We introduce self-supervised objective to reconstruct original molecular images; (d) The pretrained encoder is used to represent pair of drugs for DDI prediction.
Structure-level Encoding. In the context of molecules, the local substructures are the common intrinsic attributes across different molecules. Based on this, we leverage the substructures as anchors, represent a pair of drugs in fine-grained manner, blending molecular local structures in the early stage. Specifically, we propose a Structure-level Encoding module to blend molecules at the structure level (Figure 24). Given the input drugs (d.,,d)), to focus on the local structure of molecules, them into a matrix of visual fragments M, E IR m and M,. E IRmx*. where m = H/P, HW/P indicates the number of fragments.
split
W/P,
(P, P. C)C is the resolution of each fragment, and To deeply fuse the structures of input drugs, we design a sampling strategy to blend M, and M, into one fused matrix Muke R.V, which then fed into single image encoder for molecule representation. We define a binomial distribution S with probability vector p (PI, P2). For each fragment of M u&e (0 < i 0 j < n) in the fused matrix, we sample 3 E {1, following the probability distribution p, determining the corresponding element of the blended matrix:
MM
if =1
M udcu
MY
otherwise,
where each element in position (i,j) are randomly selected from M and M According to this process, the extrinsic and intrinsic relations of local structures across molecules are blended into a single structure-blended matrix Maka. We then inject the blended matrix Muxe into sequence of visual tokens 2t R NRPP-O)) where C denotes the number of channels. The tokenized sequence se is represented by a transformer-based encoder that mines the semantics between the local structures of blended molecules.
Decoder. We utilize an encoder-decoder architecture to embed the visual tokens 2, into hidden space and decode the latent embedding for the reconstruction of molecular images. To effectively model the semantic relations of local structures within Ht, we apply standard encoder F (i.e., 12 blocks of VIT). Following ViT, prepend learnable embedding Tels IR [gxx(P2CC) the sequence of embedded tokens 2t, whose state at the output of the encoder JF serves as the representation of input drugs. Specifically, the forward process of the encoder is as follows:
Pairwise Drug
ViT 33
Zo = [Teta W; x W; WW
; :... W] + Wenc pos
z MSA(LayorNorm((211 + 2i-1,
Zi = MLP(LayerNorm(Z)) + z//

---

where the W E (PP.. xd and W E [[++1)xdmm are the trainable parameters and positional embedding. MSA represents the multiheaded self-attention. After L layers of iterations, we obtain the fused latent embedding 21,, To further the structural fusion of input drugs, we feed 2r, into lightweight decoder F (i.e.. 4 blocks of ViT) for molecule reconstruction. Similar to the encoder, the reasoning process of decoder F is defined as follows:
Cy Z Wdee
post
e MSA (LayerNorm(e1-1)) et-1,
C₁ MMP((LayeNNorm(c)) + c/2
where Wdss E [I(++))xd denotes the positional embedding of the decoder and e, E [NN++)xdimm pos is the decoded representation by F. Then the decoded embedding e, is input into two heads for image reconstruction, described in the next section. The decoder is only used during pretraining reconstruct the original molecular images, and the encoder is adopted for the downstream DDI prediction task.
Drug Reconstruction. V introduces reconstruction objective, recovering the original molecular images dy and de from ej by predicting the pixel value each missing patch within the target molecule. Specifically, introduce two linear projections to scale the latent embedding e,, defined as follows:
h, = e [1: JJE + b1. h 1: E2 + b2,
(5)
where E, E (Rdmm((PA.) and E2 E [Rdimx(P2.C) represent learnable parameters. h, E RN
and h E (x((..))) denote the constructed latent embedding, which then is reshaped to form the reconstructed molecular images du and dy. As depicted in Eq. (5) to emphasize the visual information in molecules, we remove the global [class] token (i.e., e (0) as e;[1; Subsequently, introduce mean squared eπror (MSE) our loss function to optimize the reconstruction process:
Erec MSE d.) + d.).
(6)
By minimizing free during pretraining,
obtain a unified encoder to represent the paired drugs.
Downstream DDI Prediction and Optimization, We consider the DDI prediction multi-class classification task. Given a predicted drug pair (du, d. we use the structure- encoding module to convert it into sequence x, of visual tokens. We then feed them into the pretrained encoder F and obtain their latent embedding z. To focus on the global representation of the drug pair, we adopt the embedding [class] token Z00, predict the interaction probability of the given drug pair as follows:
(dd, = =o (MLP(2((0::)),
where (-) is the softmax activation function. We then utilize the cross-entropy loss:
l ppre
log(üt'., W
(8)
where K. is the number of DDI event types and V(d.,d.) represent the ground truth.
4.2 Theoretical Analysis
In this section, we present a theoretical perspective based on mutual information maximization 34 35
to understand better the effectiveness of S²VM. Given pair of drugs (du, de), as described in Structure-level Encoding module, they are randomly partitioned into two parts, represented as d, = [A,, A2] and d, [B, B2). A, shares identical indexes of visual fragments with B,, i (1,2) and the blended matrix is denoted as Mu&v [A1, Bz].
Proposition (Mutual Information Maximization) VM represents input molecules with structure- level encoding into latent space and then recovers them, maximizing the lower bound of the mutual information: EsT (Ag; A1, B2) I(B;; B2). The proof is detailed in Appendix A.
Proposition 2 (Objectives of Pretraining Process) The mutual information T(Az; A1, B2) I [BB; A1, B2) can be decomposed into extrinsic and intrinsic objectives: (I) contrastive and
5

---

generative between input drugs, and (2) recover missing visual visual fragments for each molecule (i.e., Eq. The proof refers to Appendix A.2
9
BI) + (A2; Ba) + [AA;; Bi + (A2; B2|A1)
contrastive and generative
conditional contrastive and generative
(9)
A2) [BB;
A2 (2) [B;;; (2(AA)
recovery
conditional recovery
Based on the above propositions, conclude that V has two strengths in embedding paired molecules: (i) VM using contrastive and generative objectives can learn finer- seer-graieed associations (e.g., structural interactions between different molecules) from large-scale paired drugs, which improves the generability of molecular representations (i.e., Extrinsic Relations); (ii) S can effectively model the relationship between local structures within a molecule through the recovery of missing visual fragments, which helps to enhance the structural representation of the molecule (i.e., Intrinsic Structure). In conclusion, S effectively enhances downstream DDI prediction by modeling paired molecular representation from both external and internal perspectives.
5 Experiments
In this section, to evaluate the effectivvneess of VM, we carefully consider the following key research questions: Q1: Does S outperform SOTA baselines on DDI prediction across various scenarios? Q2: Are the designed supe-vupe pretraining architecture and unified encoder effective? Q3: Can S achieve superior performance in new drugs and explore structural mechanisms for DDIs?
5.1 Experimental Settings
Datasets. To evaluate our S we adopt widely-used datasets: (1) Deng's 's dataset 336 contains 65 types of DDI events with total of 37,264 DDIs among 570 drugs, (2) Ryu 's dataset (i.e., DrugBank) [12] includes 86 types of DDI events with a total of 191,570 DDIs between 1,700 drugs, and (3) TWOSIDES (37 has 604 drugs and 252, 11 for 200 event types. Further, following MRCGNN (2I) we count the number of DDI instances involving each DDI event as event frequency and split these DDI events into two groups for few-shot settings (Few and Rare). We present event types and corresponding proportions in each group in Appendix B.I The TWOSIDES is adopted to evaluate the performance of S VM in emerging drugs. Specifically, we adopt two strategies: S/ setting, determining the interaction type between an emerging drug and an existing drug, and S2 setting, predicting the interaction type between two new drugs. For pretraining, we adopt 200,000 molecules from PubChem to construct ~ 200M pairs of drugs. Refer to Appendix [B. for details. Each molecule transformed into a molecular image through a standardized and reproducible pipeline, which serves as the visual imput to model. detailed in B.I
Evaluation. Following MRCGNN (21). we split Deng's 's and Ryu's datasets into training, validation, and test sets with a ratio of 7:1:2, ensuring that each set contains DDI events from all interaction types. We treat the prediction on Deng's and Ryu's datasets as a multi-class classification task, employing Accuracy, Macro-F1, Macro-Recaa both common and few-shot scenarios. In the TWOSIDES dataset under the inductive setting, a drug pair may exhibit multiple interaction types. The task here is to predict whether specific type of interaction would occur between the paired drugs using a binary classification setting. So we utilize the Accuracy and ROC-AUC metrics on the TWOSIDES datasets. In addition, we select the best model on the validation set based on Macro-Fi for the multi-class classification task and ROC-AUC for the multi-type classification task. Table I reports the average results from five runs the test set.
and Macro-Precisio
our evaluation metrics in
Implementation Details. For the pretraining process, we set the learning rate number of iterations as 2, 000, the size of the molecular image is 224 x 224 x 3, the size of the visual fragment 16 16 x 3, and the numbers of transformer layers in the encoder and decoder are 12 and 4. respectively. For the downstream DDI prediction task, we set the learning rate Ir = x 10 and the number of iterations as 100. The image and fragment sizes remain consistent with the pretraining process. The encoder's weights are frozen and utilized to model the representation of DDI pairs. We provide hyperparameter analysis in Appendix C All experiments are conducted on the Linux server with one RTX 3090 (24GB RAM) RTX 2080Ti (12GB RAM) (refer to Appendix B.2
1.5 10-4 the

---

Table Results of S"VM and baselines for drug interaction prediction on two datasets. We mark the best score with bold font and the second best with underline.
Deng's s dataset
Ryu's dataset Macro-Rec.
Method
ACC.
Macro-F1 Macro-Rec. Macro-Pre.
ACC.
Macro-F1
Macro-Pre
DeepDDI
78.07
60.55
58.39
66.11
93.23
86.43
85.12
89.28
SSI-DDI
78.66
42.16
38.96
51.39
90.08
66.63
62.87
75.07
MUFFIN
82.69
$2.45
48.44
62.04
95.10
85.66
83.39
89,80
KGNN
85.57
72.62
69.87
77.14
92.31
83.77
83.91
89.81
GoGNNN
87.66
69.38
68.41
73.16
94.24
85.89
84.51
89.49
MRCGNN
89.79
77.91
76.88
81.01
95.67
88.94
87.27
92.21
CGIP
87.57
76.33
76.41
81.72
93.35
85.72
87.65
88.47
ImageMol
88.75
77.83
76.13
82.72
91.74
87.57
86.62
89.93
CSSE-DDI
82.90
63.46
61.19
70.05
90.90
87.21
85.64
89.82
S"VM
91.05
82.12
79.31
85.42
95.86
92.07
91.48
94.31
Impr: (%)
11.26
13.83
12.43
12.70
10.19
13.13
14.21
12.10
Table 2: Results comparison on the few-shot Table 3: Performance comparison the few-shot
Deng's 's dataset.
Ryu's 's dataset.
Few Setting Macro-F1
Rare Setting Macro-FI
Few Setting Macro-F1
Rare Setting
Method
Method
ACC.
ACC.
ACC.
ACC.
Macro-F1
DeepDDI
47.18
41.91
36.36
31.86
DeepDDI
65.17
62.32
42.43
37.23
SSI-DDI
64.40
61.73
41.17
38.04
SSI-DDI
72.21
71.15
56.17
52.37
META-DDIE
76.85
74.12
55.13
51.01
META-DDIE
84.06
79.25
69.58
64.21
MRCGNN
81.89
79.92
47.27
43.75
MRCGNN
90.16
89.06
66.67
61.21
ImageMol
87.12
89.76
63.69
66.67
ImageMol
95.21
91.56
92.72
92.03
S'VM
91.53
91.76
68.54
73.33
S
99.51
95.37
99.23
98.44
Impr. (%)
?2.91
12.0
13.6
16.66
Impr. (%)
14.30
f3.81
16.51
16.41
Baselines. To evaluate the performance of S VM, compare it with several SOTA methods: the descriptor-based DeepDDI [12], the molecular structure-based SSI-DDI [14], the biomedical knowledge graph based KGNN [77, the molecular substructure together with DDI-related biomedical knowledge MUFFIN [88, GoGNN 38), MRCGNN (2I, and CSSE-DD [2] and the image-based
molecule representation methods ImageMol 24 and CGIP [3I] Additionally,
include scenario-
specific methods: META-DDIE (39] for the few-shot scenario and STNN-DDI [40] together with CSMDDI 25 for the inductive scenario. Refer to Appendix B.3 for more details.
5.2
Main Results (Q1)
In response to Q1, we design various experiments evaluate V in different scenarios.
Comparison with Baselines. We present the absolute performance gains of S and baselines for predicting DDIs in Table [ As shown in Table [ we observe that S² achieves the best results in the DDI prediction task on both Deng's s and Ryu's datasets. Specifically, VM improves the Macro- F1 and Macro-Rec. by least 4.21 % and 2.43% respectively 3.13% and 4.21% absolute increase over the best baseline on Ryu's dataset. Furthermore, we have the following observations: (1) Compared with DeepDDI and SSI-DDI, which focus solely on modeling molecular structures, KGNN, which utilizes local semantic relations of drug entities, performs better. This suggests that DDI-related semanties are
Deng's dataset, and achieves the
effective than molecular structures alone in predicting potential DDIs. (2) Compared with KGNN, MRCGNN, which leverages both the semantic relations of drug interaction networks and molecular structures. achieves better performance, This indicates that integrating DDI-related semantics with molecular structures enhances the prediction task. (3) Compared with MRCGNN, CGIP, and ImageMol, which mine visual information (c.g.. positional relations of functional substructures) from separate molecular images, show comparable results on both Deng's 's and Ryu's datasets. This demonstrates the potential of visual molecular information in predicting unknown DDIs. (4) S²VM, which considers paired drugs as unified input for joint encoding and explores wide space of drug pairs using self-supervised pretraining, outperforms all other methods, especially the methods based on semantic relations together with molecular structure. This demonstrates that fine-ggrain structural fusion and exploration of broad drug pairs effectively capture the extrinsic and intrinsic associations between drugs.
7

---

Few-shot Scenario. To investigate the effectiveness of VM on the fho- DDI prediction task, we design two subsets (See Table in Appendix B.I) with Few Setting and Rare Setting from Deng's 's and Ryu's datasets, respectively. The results of VM on these few-shot scenarios are presented in Table and Table 3 VM consistently outperforms the baselines, achieving a Macro-F1 improvement 2% and 6.66% in the Few and Rare settings on Deng's s dataset, respectively. Similarly, VM increases the Accuracy score by 4.3% and 5.5 the Few and Rare settings on Ryu's dataset. Besides, we observe that (1) MRCGNN, which incorporates drug-related semantic relations and molecular structures, outperforms the structure-basee methods SSI-DDI and DeepDDI, indicating that the inclusion of biomedical information from knowledge graphs enhances the few-shot DDI prediction task; (2) Image-based methods CGIP and ImageMol perform comparably to methods like MRCGNN, demonstrating that visual information from molecular images is effective for predicting DDIs under limited supervision: (3) V which unifies paired drugs through structural fusion, achieves the best performance, highlighting that finer-grained structural representations of visual molecules crucial for identifying unknown DDIs, even with limited interaction information. These findings suggest that VM unifying input drugs through structural fusion and self-supervised learning offers a novel and effective perspective for few-shot DDI prediction.
5.3 Ablation Study (Q2)
To investigate the impact of each mod- ule in V perform an ablation study on Deng's s and Ryu's datasets by (1) removing the pretraining process (called w/o pretrain) and (2) consid- ering the pretrained encoder shared for encoding imput drugs separately (called w/ shared). We can observe that all variants perform worse than the VM in Figure 3 verifying the effectiveness of VM.
Results on Deng's dataset
Results on Ryu's dataset
Macro-F1
0.90
ACC.
ACC.
Metric
Metric
0.91
0.85
w/o pretrain. We observe signifi- cant reduction in performance across all datasets for DDI prediction after removing the self-supervised learn- ing process for paired drugs. Sim- ilarly, we can see that the perfor- mance of VM without pretraining shows worse results than the variant w/ shared, which implies that self- supervised pretraining has positive im- pacts on representing DDIs for shared encoders. This is because V²MM's use of a self-supervised objective for jointly encoding paired drugs effec- tively extracts both extrinsic and in- trinsic structural correlations between them.
pretrain
SPVM
pectrain
shared
SPVM
Figure 3: The results on different variants of S²
Sotting
Setting
0.70
ROC AUC
ROC-AUC
ACC:
ACC:
0.34
0.54
0.68
0.58
0.55
0.52
CSMOD!
MRCONN
Imagehiel
SYM
0.52
S85-DDi
CSNODI
MECONN
ImageMaa
SV
Figure 4: The performance of S2 based on TWOSIDES on inductive scenarios.
w/ shared, From the results reported in Figure 3 we notice a degradation in performance compared with S²VM on both Deng's and Ryu's datasets. This observation demonstrates representing paired fusing structural visual information from molecular images. The
drugs jointly is beneficial performance reductions observed in w/o pretrain and w/ shared underscore the importance of mining structural relationships between drug pairs in a self-supervised manner and jointly encoding paired drugs for DDI prediction.
5.4 Effectiveness and Interpretability of V (Q3)
Inductive Scenario. Predicting potential DDIs for new drugs remains significant challenge. SPVM introduces self-supervised framework that mines both extrinsic and intrinsie mechanisms of drug interactions, showing potential for predicting unknown DDIs for emerging drugs. To evaluate the

---

prediction ability of VM on new drugs, we design inductive experiments for two settings: SI and S2. As shown in Figure 4 we can see that VZ performs best on both SI and S2 settings. Specifically, the results indicate that MRCGNN outperforms molecular structure-basee methods like SSI-DDI and STNN-DDI, suggesting that incorporating semantic relations can enhance the inductive prediction ability for emerging drugs. Besides, ImageMol shows better performance than previous models, demonstrating that the representations of visual molecules are beneficial DDI prediction in inductive scenarios. Furthermore, S by introducing self-supervised framework that effectively represents paired drugs jointly and explores structural correlations within the broad space of drug pairs, achieves notable improvement'' in both SI and S2 settings. This suggests that S'VM superviee exploration lcrge-scale observed drug pairs can effectively extract the structural relations between new drugs.
Structural Interpretation for DDI Mechanisms. In DDI prediction, per- petrators can alter the pharmacoki- netics (PK) of victim drugs by in- ducing or inhibiting metabolic en- zymes [41] To evaluate the inter- pretability of S²MM, we focused on substructures of perpetrator drugs that are reported in the literature to in- hibit metabolic enzymes. We utilize a manually curated dataset compris- ing multiple chemicals known to in- hibit metabolic enzymes via specific substructures. Figure demonstrates how SM identifies the most salient structural motifs in the drug Paroxe- tine across multiple DDI pairs. No- tably, the highlighted substructures in Paroxetine correspond to known inhibitors of CYP2D6, such as 1,3- Benzodioxole 42 These key frag- ments were consistently highlighted in DDIs involving Paroxetine [43]. suggesting mechanistic relevance. To quantitatively assess the ability of V* to focus on key substructures, analyze the predictions that emphasized known inhibitory motifs across 4,543 DDIs involving nine distinct drugs. As shown in Figure 5p, we introduce four metrics to assess the hit rate of the top-weighted substructures by VM. The results indicate that the model's top-attended substructures are well-aligned with domain knowledge, underscoring its strong interpretability in identifying biologically meaningful features for DDI prediction. Additional details on the evaluation metrics, cases highlighted regions, and annotated data are provided in Appendix
(a) example of the focused
drug Paroxetine S'VM
piperidine
1.3Reanndiezole
region 231
region DDI Key for 136 DDI
(b) A quantitative ussessment of S'VM in exploring key substruetures for DDIs
Hiis
Hiis@
Hingge
Hitkgs7
80
Hlisk (%)
60
20
Alkylimidazole
J.-BBezoddoxolee
Functional Group
Figure The structure-based explainability of S²VM.
Limitation and Conclusion
Limitation. While S2VM advances DDI prediction, three considerations warrant attention. Real- world DDI distributions are influenced by temporal emergence patterns, therapeutic classes, and toxicity frofiles-factors not explicitly modeled into the pretraining stage, which could enhance adverse interaction detection. Second, 2D molecular representations ignore 3D conformational effects, which reflect inherent -aabblittyygaaullriiyy trade-off's rather than critical flaws, suggesting future directions. Third, the current framework primarily focuses on structural learning support new or annotated drugs, where biological context is often limited or unavailable, thereby potentially overlooking pharmacological or mechanistic factors underlying DDIs. Compared to knowledge-enhancee models such as MUFFIN, which rely on entity coverage, S achieves up to 32.5% higher accuracy on rare DDIs, demonstrating stronger generalization under low resourcee scenarios.

---

Conclusion. Predicting drug-drup interactions (DDIs) is essential for ensuring patient safety and optimizing therapeutic strategies. However, existing models are often limited by insufficient represen- tation of structural correlations between paired drugs and inadequate exploration of the vast space of potential drug pairs. To address these issues, we propose S²VM, a self-supervised pretraining frame- work with a pre-fusion strategy that enhances structural modeling and generalization using over 200 million drug pairs. While V demonstrates effectiveness, challenges such as computational costs, data diversity, and limited interpretability remain, presenting opportunities for further improvement. Moving forward, we aim to refine the pretrained encoder as a backbone for drug representation and extend its applications to broad drug discovery.
Acknowledgements
This work was supported by the National Natural Science Foundation of China (625B2067: U22A2037; 62425204; 62122025: 62450002; and 62432011). This work was supported by the Beijing Natural Science Foundation (L.24003)
References
[1] Santiago Vilar, Eugenio Uriarte, Lourdes Santana, Tal Lorberbaum, George Hripesak, Carol Friedman, and Nicholas P Tatonetti. Similarity-based modeling in large-scale prediction of drug-drug interactions. Nature Protocols, 99):2147-2163,, 2014.
[2] Mukesh Bansal, Jichen Yang. Charles Karan, Michael P Menden, James C Costello, Hao Tang, Guanghua Xiao, Yajuan Li, Jeffrey Allen, Rui Zhong, et al. A community computational challenge to predict the activity of pairs of compounds. Nature Biotechnology. 32(12):1213- 1222, 2014.
[3] Bo Jin, Haoyu Yang. Cao Xiao, Ping Zhang. Xiaopeng Wei, and Fei Wang. Multitask dyadic prediction and its application in prediction of adverse trug-drug interaction. In Proceedings of the AAAI conference on artificial intelligence, volume 31, 2017.
[4] Yaqing Wang, Zaifei Yang, and Quanming Yao. Accurate and interpretable drug-drug interaction prediction enabled by knowledge subgraph learning. Communications Medicine, 4(1):59, 2024.
[5] Xuan Lin, Lichang Dai, Yafang Zhou, Zu-Guo Yu, Wen Zhang, Jian-Yo Yu Shi, bong-Sheng Cao, Li Zeng, Haowen Chen, Bosheng Song, et al. Comprehensive evaluation of deep and graph learning on drug-drug interactions prediction. Briefings in Bioinformatics, 24(4):bbad235, 2023.
[6] Yongqi Zhang, Quanming Yao, Ling Yue, Xian Wu, Ziheng Zhang, Zhenxi Lin, and Yefeng Zheng. Emerging drug interaction prediction enabled by flow-based graph neural network with biomedical network. Nature Computational Science, 3(12):0033-103 1033, 2023.
[7] Xinyue Li, Zhankun Xiong, Wen Zhang, and Shichao Liu. Deep learning for drug-drug interaction prediction: A comprehensive review. Quantitative Biology. 12(1):30-52, 2024.
[8] Takako Takeda, Ming Hao, Tiejun Cheng, Stephen Bryant, and Yanli Wang. Predicting drug- drug interactions through drug structural similarities and interaction networks incorporating pharmacokinetics and pharmacodynamics knowledge. Journal of Cheminformatics, 9:1-9, 2017.
[9] Santiago Vilar, Rave Harpaz, Eugenio Uriarte, Lourdes Santana, Raul Rabadan, and Carol Friedman. Drug drug interaction through molecular structure similarity analysis. Journal of the American Medical Informatics Association, (066):066-110 1074, 2012.
[10] Assaf Gottlieb, Gideon Y Stein, Yoram Oron, Eytan Ruppin, and Roded Sharan. Indi: computational framework for inferring drug interactions and their associated recommendations. Molecular Systems Biology, 8(1):592 2012.
[11] Peng Li, Chao Huang, Yingxue Fu, Jinan Wang, Ziyin Wu, Jinlong Ru, Chunli Zheng, Zihu Guo, Xuetong Chen, Wei Zhou, et al. Large-scale exploration and analysis of drug combinations. Bioinformatics, 31(12):2007-2016 2015.
10

---

[12] Jae Yong Ryu, Hyun Uk Kim, and Sang Yup Lee. Deep learning improves prediction of trug-drug drug drug and drug-fort food interactions. Proceedings of the National Academy of Sciences, 115 2018.
(18):F4304-FA4)
[13] Kexin Huang, Cao Xiao, Trong Hoang, Lucas Glass, and Jimeng Sun. Caster: Predicting drug interactions with chemical substructure representation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 702-709, 2020.
[14] Arnold K Nyamabo, Hui Yu, and Jian-Yu Shi. Ssi-ddi: substructure--ubssrruuctre interactions for frug-drus interaction prediction. Briefings in Bioinformatics, 22(6):bbab133 2021.
[15] Shenggeng Lin, Yanjing Wang, Lingfeng Zhang, Yanyi Chu, Yatong Liu, Yitian Fang, Mingming Jiang, Qiankun Wang, Bowen Zhao, Yi Xiong, et al. Mdf-sa-ddi: predicting drug-drug interaction events based multi-source drug fusion, multi-source feature fusion and transformer self-attention mechanism. Briefings Bioinformatics, 23 b3())bbba 2022.
[16] Zimeng Li, Shichao Zhu, Bin Shao, Xiangxiang Zeng. Tong Wang, and Tie- Yan Liu. Dsn- ddi: an accurate and generalized framework for drug-drug interaction prediction by dual-view representation learning. Briefings in Bioinformatics, 24( (A(1):bba,597, 2023.
[17] Xuan Lin, Zhe Quan, Zhi-Jie Wang, Tengfei Ma, and Xiangxiang Zeng. Kgnn: Knowledge graph neural network for drug-drug interaction prediction. In IJCAI, volume 380, pages 2739-2745, 2020.
[18) Yujie Chen, Tengfei Ma, Xixi Yang, Jianmin Wang, Bosheng Song, and Xiangxiang Zeng. Muffin: multi-scale feature fusion for drug-drug interaction prediction. Bioinformatics, 37(17): 2651-2658, 2021.
[19] Yue Yu, Kexin Huang, Chao Zhang, Lueas M Glass, Jimeng Sun, and Cao Xiao. Sumgnn: multi- typed drug interaction prediction via efficient knowledge graph summarization. Bioinformatics, 37(18):2988 2995, 2021.
[20] Tengfei Lyu, Jianliang Gao, Ling Tian, Zhao Li, Peng Zhang, and Ji Zhang. Mdnn: A multimodal deep neural network for predicting drug-drug interaction events. In Proceedings of the International Joint Conference on Artificial Intelligence, volume 2021, pages 3536-3542, 2021.
[21] Zhankun Xiong, Shichao Liu, Feng Huang, Ziyan Wang. Xuan Liu, Zhongfei Zhang, and Wen Zhang. Multi-relational contrastive learning graph neural network for drug-drug interaction event prediction. Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 1339-5347, 2023.
[22] Xiaorui Su, Pengwei Hu, Zhu-Hong You. S Yu Philip, and Lun Hu. Dual-channel learning framework for drug-drug interaction prediction via relation-aware hetcrogeneous graph trans- former. In Proceedings the AAAI Conference on Artificial Intelligence, volume 38, pages 249-256, 2024.
[23] Haotong Du, Quamming Yao, Juzheng Zhang, Yang Liu, and Zhen Wang. Customized subgraph selection and encoding for drug-drug interaction prediction. Advances in Neural Information Processing Systems, 37:109982-109608, 2024.
[24] Xiangxiang Zeng, Hongxin Xiang, Linhui Yu, Jianmin Wang, Kenli Li, Ruth Nussinov, and Feixiong Cheng. Accurate prediction of molecular properties and drug targets using a self- supervised image representation learning framework. Nature Machine Intelligence, 4(11): 1004-1016, 2022.
[25] Zun Liu, Xing-Nan Wang, Hui Yu, lian-Yu Shi, and Wen-Min Dong. Predict multi-type drug-drug interactions in cold start scenario. BMC Bioinformatics, 23(1):75, 2022.
[26] Xiaorui Su, Zhuhong You, Deshuang Huang, Lei Wang, Leon Wong, Boya Ji, and Bowei Zhao. Biomedical knowledge graph embedding with capsule network for multi-label drug- drug interaction prediction. IEEE Transactions on Knowledge and Data Engineering, 35(6):
5640-5651, 2022.
11

---

[27] Ziduo Yang, Weihe Zhong, Qiujie Lv, and Calvin Yu-Chian Chen. Learning size-adaptive molecular substructures for explainable graph neural network. Chemical Science, 13(29):8693-8700 2022.
interaction prediction by substructure-awee
[28]
Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations, pages 1-14, 2016.
[29] Michael Fernandez, Fugiang Ban, Godwin Woo, Michael Hsing, Takeshi Yamazaki, Eric LeBlane, Paul S Rennie, William J Welch, and Artem Cherkasov. Toxic colors: the use of deep learning for predicting toxicity of compounds merely from their graphic images. Journal of Chemical Information and Modeling, 58(8):1533-1444 2018.
[30] Shifa Zhong, Jiajie Hu, Xiong Yu, and Huichun Zhang. Molecular image-convolutional neural network (enn) assisted qsar models for predicting contaminant reactivity toward oh radicals: Transfer learning, data augmentation and model interpretation. Chemical Engineering Journal, 408: 1279927 2021.
[31] Hongxin Xiang, Shuting Jin, Xiangrong Liu, Xiangxiang Zeng. and Li Zeng. Chemical structure- aware molecular image representation learning. Briefings in Bioinformatics, bbb6)4bb,,404 2023.
[32] David Weininger. Smiles, a chemical language and information system. 1. introduction to methodology and encoding rules. Journal of Chemical Information and Computer Sciences, 28 (1):31-36, 1988.
[33] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, pages 1-21, 2021.
[34] Paul Viola and William M Wells III. Alignment by maximization of mutual information. International journal of computer vision, 24(2):137-154, 1997.
[35] Qiying Yu, Yudi Zhang. Yuyan Ni, Shikun Feng, Yanyan Lan, Hao Zhou, and Jingjing Liu. Multimodal molecular pretraining via modality blending. In International Conference Learning Representations, 2024.
[36] Yifan Deng, Xinran Xu, Yang Qiu, Jingho Xia, Wen Zhang, and Shichao Liu. A multimodal deep learning framework for predicting drug drug interaction events. Bioinformatics, 36(15): 4316 4322, 2020.
[37] Nicholas P Tatonetti, Patrick P Ye, Roxana Daneshjou, and Russ B Altman. Data-driven prediction of drug effects and interactions. Science Translational Medicine, 12525) 125ra31, 2012.
[38] Hanchen Wang, Defu Lian, Ying Zhang, Lu Qin, and Xuemin Lin. Gognn: Graph of graphs neural network for predicting structured entity interactions. In Proceedings of the International Joint Conference on Artificial Intelligence, pages 1317-1323, 2020.
[39] Yifan Deng, Yang Qiu, Xinran Xu, Shichao Liu, Zhongfei Zhang, Shanfeng Zhu, and Wen Zhang. Meta-ddie: predicting drug-dmy interaction events with few-shot learning. Briefings in Bioinformatics, 23(1):bbab51 4, 2022.
[40] Hui Yu, Shi Yu Zhao, and Jian Yu Shi. Stnn-ddi: substructure-awware tensor neural network to predict drug-drug interactions. Briefings in Bioinformatics, 23(4):bbac009, 2022.
[41] Jukka Hakkola, Janne Hukkanen, Miia Turpeinen, and Olavi Pelkonen. Inhibition and induction of cyp enzymes in humans: an update. Archives of Toxicology, 94(11):3671-372 2020.
[42] Zhao Wang, Alisa R Kosheleff, Lilian W Adeojo, Oyinkansola Odebo, Toyin Adewole, Peibing Qin, Vladimir Maletic, Stefan Schwabe, and Azmi Nasser. Impact of paroxetine, strong cyp2d6 inhibitor, on spn-812 (viloxazine extended-release) pharmacokinetics in healthy adults. Clinical Pharmacology in Drug Development, 10(11): 1374, 2021.
12

---

[43] Suvi TM OIT, Sharon L Ripp, T Eric Ballard, Jaclyn L Henderson, Dennis O Scott, R Scott Obach, Hao Sun, and Amit S Kalgutkar. Mechanism- based inactivation (mbi) of cytochrome p450 enzymes: structure-activity relationships and discovery strategies to mitigate drug-drug interaction risks. Journal of Medicinal Chemistry, 55(1 55(11):48964499 2012.
[44] MTCAJ Thomas and A Thomas Joy. Elements of information theory. Wiley-Interscionn 2006.
[45] Tengfei Ma, Cao Xiao, Jiayu Zhou, and Fei Wang. Drug similarity integration through attentive multi-view graph auto-encoders. In Proceedings of International Joint Conference on Artificial Intelligence, pages 3477-3483, 2018.
[46] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, 9(1 1), 2008.
[47] Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael Rabbat, Yann LeCun, and Nicolas Ballas. Self-supervised learning from images with joint- embedding predictive architecture. Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pages 15619-15629, June 2023.
[48] Ramprasaath Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE International Conference on Computer Vision, pages 618-626, 2017.
[49] Yi Zhong, Gaozheng Li, Ji Yang, Houbing Zheng, Yongqiang Yu, Jiheng Zhang, Heng Luo, Biao Wang, and Zuquan Weng. Learning motif- graphs for goug-drug interaction prediction via local-global self-attention. Nature Machine Intelligence, 609):1094-1100 1104, 2024.
Technical Appendices and Supplementary Material
A Theoretical Analysis
In this section, we uppercase to denote the random variables and lowercase represent samples of the random variables, followed by the common notations from 35).
A.1 Lemma (Chain rule of mutual information)
Mutual information with conditions follows the law below, i.e.,
1(X,, = x) 1(2::
(10)
Proof.
T(X1;Y) + T(X;; ((x;; Eppla [log p(11, v)
p(z1)p(u)
En(r1 log p(1z, vlo1) p(12)
p(1z)=1)p(y/zz))
p (pl)) p( (2 log p( p(az.0la1)
- log plu 1)
p(y)p( 11)p
Ep()))22)) log p(u2, y, 11) p(y)p(r2,z1)
1(x1,X2;1)
End Proof.
Based on the above lemma, we can decompose our mutual information Eg A1, B2) I [[B;; A1, B2), described next.
13

---

A.2 Proposition 1 (Objectives of Pretrain Process)
As described in the main paper, the [A2: A1, (B; B2) can be decomposed into following parts:
+ (A2; -
+ [(41 B:) + I(A2: B|)::
(II)
[ 41 B2) 14
2
[B:; (1)
IBB:;
Ba(Ai)]
Proof. We provide the first term in ((42; 4,, Bb) + / [B;; An B2), i.. [(22; A1, B2). Based on the Lemma 1, and let X1 A1, X2 B2, = A2, have:
A1. = I (Ai A2) + I (A2; B2 (1).
(12)
Also use Lemma and let X1 = B2, X2 = A1 Y = A2, then we have:
I(A2; A1; B2) [B2; A2) + [A2; A B2).
(13)
Based on Eq. 12 13.. the T(A2; A1, B2) can be divided into:
// A() + [A2; B2(A1) + [(2;; A2) + [A2; A1 Ba]
(14)
Similarly, we adopt Lemma decompose the second term [(B1; A1, B2):
2 [T(Bi; A1) + [(B2; Ba)Ai) + I(B:; Ra) + T(B: A (Ba)
(15)
End Proof.
14

---

Table 4: The DDIs division of Deng's and Ryu's datasets.
Deng's ss dataset
Few Setting
Rare Setting
Event range
#39 #49
#50 #65
Event Frequency
15 and < 50
S 15
Event Proportion
1.11%
0.35%
Ryu's dataset
Few Setting
Rare Setting
Event range
#64 #75
#76 #86
Event Frequency
>15 and <50
< 15
Event Proportion
7.42%
4.71%
Table 5: The statisties of TWOSIDES for inductive settings.
S1 Setting
S2 Setting
# Drugs in Train
514
514
# Drugs in Valid
30
30
# Drugs in Test
60
60
# training set
185,673
185,673
# valid set
16,113
467
# test set
45,365
2,466
B Experimental Details
All experiments of S and baseline methods were implemented on a Linux Server with vCPU Intel(R) Xeon(R) Platinum 8255C and one RTX 3090/RTX 2080Ti.
B.1 Datasets.
For few-shot settings, we split source data into two groups according to their event frequency. As shown in Table we reported detailed event types and their proportions for few and rare settings. We can observe from Table 4 together with performance in the main paper that S can also achieve superior performance under few supervised signals. For the inductive scenario, we follow (6) and the detailed drugs and DDIs are reported in Table
In the self-supervised pretraining stage, we build sarge-scale drug pairs from a set of base drugs. Specifically, we randomly select 200k molecules from PubChem base set of molecules. Then we randomly sample 2, 0000 molecules from PubChem for each molecule of the base set. Based on this, we construct 200M pairs of drugs for pretraining. The detailed data provided in the anonymous repository: https://noonymous.o hwwprymouusooooesscoooo Siopen.science///SVMM We conduct more experiments to verify S2 on different scales of the base set (Appendix C.2).
We generate molecular images through a standardized and reproducible pipeline designed to ensure visual consistency and structural fidelity. All molecules first canonicalized using RDKit obtain a unique and deterministic SMILES representation, eliminating variations due to atom ordering or tautomers. The 2D molecular structures are then rendered using RDKit's MolsToGridImage function, explicitly depicting atoms and bonds, with each molecule represented as a 224x224 pixel image without stochastic augmentation guarantee deterministic and consistent visual representation across runs. Finally, all layout-related parameters, including sub-image spacing, drawing style, and molecule alignment, are fixed to ensure that chemically identical molecules yield identical image representations.
15

---

Table 6: The hyperparameters of V*.
Pretraining
DDI prediction
learning rate
1.5 x 10
1 x 10-3 3
patch size
16
16
#layers of encoder
12
#layers of decoder
4
scale
200k
batch size
512
64
p embedding dim
(0.5, 0.5)
192
192
Table 7: Performance (Macro-Fl (%) of predicting DDIs on Deng's dataset for different fusion strategies.
Feature Operation Concat
Fusion Strategy
Sum
Post Fusion
56.37
58.94
Pre Fusion
59.73
60.25
B.2 Implementation details of V².M.
In the pretraining stage, we tune the learning rate among (1.5 x 10-1
11.5 x 10-3 10-3,11 1.5 x 1.5 115 1002111210000 2005
10-4, 1.5 10 1.5 10-6), the size of visual fragment/patch in (8, 16, 38, the number of transformer layers in encoder among {4, 8, 12, 16). the number of transformer layers in decoder in {2, 4, 63, the scale of training data in { (50A, 100k; 2004, 300k). Furthermore, we vary the blending probability vector = (P1,P2) into = (0.3, 0.7).
(0.5, 0.5), and (0.7, 0.3). The (0.5. 0.5) is selected finally. For the DDI prediction task, we tune the learning rate in {1 10
10-2.1 2.1x x 10 x 10 4J, the batch size among (32, 64, 128, 256). The final hyperparameters are
shown in Table
B.3 Implementation details of baselines.
In the common prediction scenario, we implemented KGNN CGIFS CSSE-DD 6 and ImageMo using their official code. The results of other methods MRCGNN, DeepDDI, SSI-DDI, MUFFIN, and GoGNN are from MRCGNN 21 In the few-shot scenario, we implemented DeepDD SSI-DD META-DDIE MRCGNN |TI implemented SSI-DDI, MRCGNN, and Image.Mol based on their available sources. The results of STNN-DDI and CSMDDI were from the method [6). Note that, the image-based molecular representation model CGIP and ImageMol, we concat the embeddings of paired drugs and feed it into a 3-layer MI.Ps for classification. The parameters of CGIP or ImageMol are jointly trained with the classifier.
ImageMol using their source code. In the inductive settings, we
16

---

Table 8: The Macro-Fl (%) performance of S and its variant under inductive scenario on TWOSIDES.
S1 setting
S2 setting
VM w/o pretrain
62.33
57.85
VM
78.19
69.34
C Additional Experiments
C.1 Motivation Discussion
Limited representation of structural correlations between paired drugs. A major mechanism of drug interactions results from a few local functional substructures instead of the whole chemical substructure [13 [4] While the remaining substructures are less relevant. Therefore, the structural correlations between drugs are crucial to predict DDIs. To deeply model the structural representation of the whole drug interactions, we adopt a pre-fusion strategy to encode the input drugs jointly. table 7 we conduct a simple experiment to validate the effectiveness of pre-fusion. We introduce two fusion strategies based on molecular morgan fingerprints: (1) Post Fusion, concatenating or summing the latent embeddings of pair of drugs from 3-layer DNN encoder based on their fingerprint features (048-dimenniona. vectors); (2) Pre Fusion, previously concatenating or summing the molecular fingerprints of paired drugs unified input and then encode the input into latent embedding using a 3-layer DNN. The experimental settings of the two strategies are the same. As shown in Table 7 observe that the pre-fusion strategy performs better. This phenomenon suggests that direct joint encoding of inputs helps to model drug interactions.
Limited exploration for the space of drug pairs. Previous methods mainly learned the representations of drug pairs from known DDIs, which are limited by the labeled data and To address this limitation, we propose a self-supervised pretraining framework learning from over 200M drug pairs to extract comprehensive structural correlations between molecules. To validate the effectiveness of the self-supervised objective, we design simple experiment on S for the inductive scenario. Specifically, we perform V and its variant S2 w/o pretrain SI and S2 settings. As shown in Table g we observe that the S has a significant improvement in predicting DDIs compared with S w/o pretrain. This shows that S"VM using self-supervised learning on a broad range of drug pairs has the potential to predict unknown DDIs over emerging drugs. Similarly, in the common scenario depicted in Figure 6 V2VM shows better interaction distribution than others, indicating S²VM is efficient embedding latent space.
generalizability, especially for the new drugs 45 13.
TWOSIDES with
Table 9: The Macro-F1 (%) of S under different probability vector P on Deng's and Ryu's datasets.
Pt P2
Deng's dataset
Ryu's dataset
7:3
81.59
91.76
5:5
82.97
92.53
3:7
80.87
92.08
* http:://jiitbbbccom///ooogxxxxnp
10 "hug:e//gtthh..ooommmmmmnoooooot
17

---

MRCGNN
CGIP
ImageMol
VisualDDI
Figure 6: Distributions of DDI representations from S VM, ImageMol, CGIP, and MRCGNN across
8 event types.
Deng's datasets
Ryu's datasets
0.95
6.5
0.94
Metries
0.86
5.0
Time (min
Metries
0.90
8
Time (min)
0.77
3.5
6
0.86
2.0
0.60
0.82
16
32
48
8
16
32
48
Patch Size P
Putch Size P
Time
Macro-F1
Figure 7: The performance of VM and corresponding running time for one step various patch sizes.
C.2 Representational Distribution drug pairs.
To explore the representation distributions of drug pairs under different methods, we visualize the embeddings of the paired drugs using the T-SNE [46] tool. Specifically, we randomly select 2890 pair of drugs across event types and then extract their embeddings from MRCGNN, CGIP, ImageMol, and S for visualization. As shown in Figure 6 we can find that S can effectively divide the space for different DDI types, performing best representation distributions.
C.3 Performance on various patch sizes.
We investigate the performance of S on different patch sizes (i.c., the value of P and (P,P,C) the resolution of each patch/fragment). We vary across {8, 16, 32, 48). The results and corre- sponding average running time (for on step) are reported in Figure 7 We observe that S²VM achieves best under 8 but suffers expensive time costs. In contrast, VV performs better balance between the predictive capabilities and time costs when P = 16. Mcanwhile, the effect decreases as P increases and is accompanied by low time cost. This is because larger P reduces the number of tokens and brings about inefficient structural fusion, thus exhibiting high time efficiency and low prediction accuracy. Therefore, we finally select 16 our patch size.
C.4 Performance VM on various scales of pretraining data.
To study the performance of S2 on various scales of pretraining data, we select different numbers ([5kk, 100k, 200k, 300k)) of molecules from PubChem our base set of drugs. We then randomly construct drug pairs from the base set by sampling 2000 molecules for each drug. The performance
18

---

Deng's datasets
Ryu's datasets
0.86
0.96
0.94
Macro-FI
0.82
Macro
0.92
0.90
0.78
0.88
50k
100k
200k
300k
50k
100k
200k
300k
Pretrain Data Scale
Pretrain Data Scale
Figure 8: The performance S VM under various scales of pretraining data.
Symbol + extend openation
0.18
0.20
0.83
0.12
Macro-F
0.75
0.30
0.70
0.04
1.00
200
300
400
500
Training Steps
Ours (Random) Ours (B+A) Ours (A+B)
(a) The loss curve various scale
(b) Performance on various position arangeeents
Figure 9: Performance reconstruction and DDI prediction (Deng's 's dataset) under different input
strategies.
of S under different scales is depicted in Figure g As the data size increases, S shows linear growth trend, while the growth slows down when reaches 300k. Meanwhile, the increase in data size bring time costs, finally consider 200h molecules base of drugs.
C.5 Performance of VM different blending ratios.
We study the impact blending ratios for input drugs by varying the probability vector = (P1, P2) for the binomial distribution S. In table 9 show the performance of SM by varying into p= = (0.3, 0.7). = (0.5, 0.5), and p = (0.7, 0.3). S2VM performs best when p = (0.5, 0.5) and we finally select (0.5, 0.5) sampling ratio.
C.6 Performance of Reconstruction.
To investigate the performance of molecular image reconstruction, we illustrate the loss curve on various scales of drugs and show cases of recovery effect. As depicted in Figure Dh we observe that the curve is smooth and the loss is coverage gradually on various seales of training data, indicating V2VM performs well in this pretraining setting. Further, from the reconstruction cases, S²VM can recover most missing regions of the molecular images. These observations demonstrate that V is effective in capturing the local structural correlations between molecules, enhancing its ability to predict missing regions from the visible ones.
that
C.7 Position-independent Structural Fusion.
To verify the effectiveness of V in extracting robust structural relations from visual molecules. we experiment with adjusting the arrangements of blending paired drugs (input). Specifically, we design three strategies: (I) randomly blend the tokens (i.e., visual fragments) of molecular images (called Ours (Random)). (2) the tokens of the first drug extend tokens of the second one (called Ours (A+B)), and (3) the tokens of the second drug extend tokens of the first one (called Ours (B+A)). Note that the Random indicates that the relative positions of tokens in molecule are kept the same as the original molecular structure, and the relative positions tokens between two molecules are random. Refer to Appendix C.8 for more details. We observe that V with the three types of inputs achieves similar results, which demonstrates that V is insensitive the absolute position of the input fragments. This phenomenon indicates that the positional relations between local fragments
19

---

Prachfy
4
Random Blending
Esland openiion (+)
Estend operation (1)
Facddy
Randoms blending betwees drsss, the position of paiches sm cene as M, or.M,
M,
Extend Mr, and
4
Figure 10: Further explanations for different positional strategies.
in visual molecules be efficiently modeled, which is crucial to representing drug pairs for DDI
prediction.
C.8 Details of Position-independent Experiments.
To investigate the impact of different positional arrangements DDI prediction, we experiment
verify the VM. Specifically, we design three strategies: (a) Ours (Random), (b) Ours (A+B), and (c) Ours (B+A). For ease of understanding, we visualize each strategy Figure I0a, Figure 10b. and Figure IOc, respectively. Ours (Random) randomly blends visual fragments (1.c., tokens) of input drugs into a sequence. Ours (A+B) splices the fragments drug before the fragments of drug Similarly, Ours (B+A) concatenates the fragments of drug B before the fragments of drug A. This processed matrix will tokenized into sequence by column.
C.9 Performance of VM under I-JEPA Pretraining
To further validate the robustness and generality of the S2VM framework, we implemented an image- level I-JEPA [7 variant of V², trained under the same settings and scale (50K base molecules, 50M drug pairs) for fair comparison. For the I-JEPA variant, we replaced the VIT backbone in S with the I-JEPA architecture and adapted it to handle paired molecular images. Each drug image is masked and encoded separately to obtain its context features, which are then combined to reconstruct the missing regions of each image following the standard I-JEPA procedure, All default retained, except for those shared with VIT (e.g., embed dim=192). As summarized in Table 10 the I-JEPA variant achieves comparable performance on the Ryu dataset and only slightly underperforms distinet pretraining paradigms demonstrate that S2VM effectively captures transferable molecular representations regardless of the underlying encoder design, while also highlighting the potential of JEPA-style models as a viable alternative for future extensions.
I-JEPA hyperparameters
the Deng dataset. These consistent results across two
Table 10: Performance Macro-Fl (%) comparison between ViT-based and JEPA-based S models under small-scale pretraining
Method
Deng's dataset
Ryu's dataset
I-JPEA SVVM
71.54 77.12
88,78 88.83
Table 11:
Performance comparison between VM and mingle-molecule representation baselines.
Deng's dataset
Ryu's dataset
Method
ACC
Macro-F1
ACC
Macro-Fl
MAE
82.76
69.61
92.15
89.20
S
91.05
82.12
95.86
92.07
20

---

Amáodorosc
Süripentol
Icedndh
I, Al-Benaeliosole
Amine
AS
lentylene
Figure 11:
Examples of visualization by using Grad-CAM. The focused regions of V are mainly
located in key substructures.
C.10 Performance of S under sing-molecule Pretraining
We conducted the control experiment by pretraining a ViT-based masked autoencoder (MAE) us- ing single-molecule reconstruction only. This baseline uses the same architecture, the same 200K PubChem molecules, and identical training epochs as S*, ensuring a fair comparison. During downstream DDI prediction, adopted post-fusion strategy where each drug is encoded individu- ally and the resulting representations are concatenated for classification. As shown Table [T] the single-molecule MAE baseline consistently underperforms S across all metrics and datasets. This demonstrates that joint reconstruction pretraining yields more expressive and interaction-awa awwre representations, beyond what is achievable through standard single-drug encoding.
D Implementation details for interpretable analysis
D.1 Evaluation metries
To quantifiably evaluate the ability of VM in focusing key molecular substructures, introduce the Hits@K K metric. Specifically, the Hits K is computed by filtering Grad-CAM [48] heatmaps using predefined anchor boxes (Figure ITI. Let hx denote the set of top- hot pixels within the entire molecular image (sorted by the pixel value), B indicate the whole pixels in the box, and T is the set of all pixels within the molecular image. The Hits@K is defined follows:
N
Hits Σ
((xx,BT))
(16)
Hits N
Hits@K
where Hits denotes the number of hit samples (i.e., S2VM successfully focuses the key functional
groups), K E (1,3,5,77 7} and fhxx, B,T) is:
{
hg n
B
x 33
f(rg. B,T)
B
T
(17)
others
The core idea to assess whether the V2'''s region interest is concentrated in key substructures.
D.2
Case
As shown in Figure 11 we used Grad-CAM to generate the attention map S2M with the high- attention regions that predominantly influence its predictions. These high-attention regions are mostly concentrated in critical substructures within the molecules, such as Amine, B-Benzodioxxle, and Acetylenc groups, which are key contributors to the occurrence of adverse DDIs.
21

---

Table 12: The labeled key structures for DDI mechanisms.
The evidence is available at 49
Chemical Name
DrugBank ID
Functional Group
Enzyme inhibited
Chemical Formula
Paroxetine
DB00715
1,3- ..3-Benzdioxole
CYP2D6
C19H220
Stiripentol
DB09118
1,3- e
CYP2C19HCYP2266
Cs
Ethinylestradiol
DB00977
Acetylene
CYP3A4
C20
Gestodene
DB06730
Acetylene
CYP3A4
Cn H2
Icotinib
DB11737
Acetylene
CYP344ICYP3AS
22
Midazolam
DB00683
Alkylimidazole
CYP3A4
11
Verapamil
DB00661
Amine
CYP3A4
C27HassN224
Troleandomycin
DB13179
Amine
CYP3A4
Cai 1o15O15
Erythromyein
DB00199
Amine
CYP3A4
C37 Hon N013
Amiodarone
DB01 118
Amine
CYPIA2HCCYP299
C25 22N02
D.3 Labeled Key Structures for DDI Mechanisms
We show the evaluation data used for the structured interpretation of the DDI mechanism in Table T2 For example, for drug Paroxetine and other drugs that produce adverse DDIs, the main reason is that the functional group 1, 1,3-Benzdioxole in Paroxetine inhibits the drug metabolizing enzyme CYP2D6. Therefore, if V can effectively model these key substructures will greatly improve the detection efficiency and discover new structuring mechanisms, We adopt a subset of all the labeled data 449 and are available at this link
E Broader impacts
V²' improved DDI prediction could enhance patient safety by reducing adverse drug interactions and accelerate therapeutic development through efficient preelinical screening. However, overreliance on AI predictions without clinical validation might risk misdiagnosis, while data biases could amplify healthcare disparities for underrepresented populations. Additionally, misuse of the model to design harmful drug combinations poses ethical concerns. To mitigate these risks, rigorous validation with pharmacologists, bias-aware data curation, and ethical governance frameworks are critical to ensure transparent and responsible deployment in real-world healthcare systems.
22

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We clearly state the claims in the abstract and introduction. The claims match theoretical and experimental results and reflect how much the results can be expected to generalize other settings.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We include limitations paragraph the Limitation and Conclusion section.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiitiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
23

---

Answer: [Yes]
The paper includes theoretical analysis of the self-supervised blending of drug pairs in Section 4.2 and its proofs in Appendix A.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Section 5.1 and Appendix detail training settings, datasets, implementation of baselines, architecture parameters, and optimization details. All necessary elements for reproducibility are described. The code and datasets are available.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: A reproducible implementation and preprocessing scripts made available at
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm cc/
oow.aaaboisioooooooooooooooo for more details. public//guides/CdeSSuuiiiisss
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https for more details.
//nips //ipss.c/ cttp://cccpbblccccuiueeeeecs guwdes/dde/ee/ http:///ib///.....ooooo/////aaaa
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: Hyperparameters, data splits, and implementation details are provided in Section 5.1 and Appendices B.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer:
[Yes]
Justification:
We report standard deviation across 5 runs for key metrics (Table I).
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
25

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The compute resources are reported in Section 5.1.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics /tthhh/tttpts//pppppo..o///////t
Answer: [Yes]
Justification:
The research complies with NeurIPS ethical guidelines.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- cration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Appendix discusses the broader impacts of improved DDI detection tools in clinical patients.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
26

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The model and data pose no foresceable risks requiring special safeguards.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All datasets used are properly cited and license-compliant.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URI.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
27

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, batasets.papersstt paperswi
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is available online, the authors encouraged to reach out the asset's s creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The released codebase includes documentation on the training pipeline and dataset preprocessing.
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [Yes]
Justification:
The paper does not involve human subjects or crowdsourcing.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: the paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
28

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: LLMs were used only for editing and formatting purposes.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy battps://neunee http://errripicccoooeeerrreeee/e cc/Coonfereecce//2225/LL. for what should or should be described.
29


Reviews:
[{"review_id": "Ol3JshK246", "reviewer": "Reviewer_yTyP", "summary": "The paper presents a self-supervised model, S$^2$VM, which pre-trains a vision transformer on 200 million drug pairs for downstream drug–drug interaction (DDI) prediction.\n\nThe model processes pairs of 2D molecular images, from their original SMILES representations, through a masking and blending operation that generates a unified image. The pretraining objective is to reconstruct each molecule in the pair, encouraging the encoder to learn useful representations that are later reused for the DDI task; a popular objective in representation learning tasks for other modalities especially images.\n\nThe authors report state-of-the-art performance across multiple DDI benchmarks and similarly argue that the learned latent representations in their self-supervised learning model captures both the intrinsic structure of individual molecules and their extrinsic relational properties.\n\nThis work addresses a critical problem with significant implications for combinatorial therapy, particularly in the treatment of complex diseases.", "questions": "- Could the authors share their thoughts on why a JEPA-based approach was not considered for me to better their architectural choices? I'll consider strengthening my score if I get better understanding of this, and if the authors could add the aforementioned experimental results on scaling pretraining data.", "limitations": "I appreciate that this was communicated within the manuscript: 3D conformational effects. A 2D image of a molecule is not as rich as 3D representation of a molecule, hence a graph based approach is likely the better method to represent the data accurately. Beyond the 3D structure, you can similarly encode other properties which you could not really express in a 2D image. This is a key limitation, and begs the question if this work would be impactful towards advancing our understanding DDI prediction.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "With self-supervised learning approaches, often my main concern lies with demonstrating that the latent representations learned in pre-training tightly capture the intrinsic properties of its data distribution, in this case, two molecular images. Accuracy can sometimes be a misleading statistics and can often not be a robust signal for demonstrating true generalization. Here are some factors that dictate robust latent representations, and I'll use this as a measure of the strengths/weaknesses of the paper:\n\n1) Transfer learning - Out of distribution. Although accuracy can be an ineffective measure of SSL, it is still a valid datapoint:\n - The model was pretrained with data not exposed within their transfer learning tasks, namely the Deng & Ryu datasets. This is a fair evaluation measure and is indicative of a robust latent representation.\n - The authors demonstrate that across benchmarks, S$^2$VM handsomely outperforms other methods. What I found to be notable is that the authors invested time into implementing other approaches to validate their results, this is a high-fidelity result, and code was similarly provided.\n\n2) \"Mutual Information\" - How much information is shared between the latent embedding and the corresponding input distribution:\n- The authors similarly provide some theoretical analysis in support of this in Section 4.2. Although this is encouraging, I found the section to be quite thin. Clarity could be improved in this section, potentially by including a short summary of the proof instead of leading the reader to the Appendix. The proof as well lacks clarity, I imagine $X_1$ and $X_2$ are the images, is $Y$ the latent embedding? All in all, this is a section that could greatly benefit from clarity.\n\n3) Geometry of the representations: E.g. do clusters emerge in the representation:\n-  The authors provide a t-SNE visualization of the representations that emerge from their SSL model in Figure 1b (although small), and the Appendix. Compared to other approaches, S$^2$VM, provides a clear delineation in its representations across DDI event types. This is a promising sign of robustness.\n\n4) An understanding of when & where representational collapse lies:\n- Less trivial, but here, we'd like a measure of the variance of the latent representation, and what are the conditions that enable representational collapse in an SSL architecture. I noticed in the paper that the model employs random masking on the 2D images, one could similarly also apply a learnable masking operation for effective representational learning, although I can imagine a situation in which this could diminish the variance of the latent representation. This is something for the authors to think about.\n- As a follow up to this, I would encourage the authors to consider an experiment in which you measure performance on the Deng & Ryu transfer learning task, as a function of the pretraining dataset size (10K, 1M, 10M, 100M, 200M, ...). This could be a good plot which would demonstrate how important the size of the pretraining dataset is towards the downstream performance of the model. As far as I can tell, this is not communicated in the paper. (**This is, in my view, the most addressable weakness of the paper**). It would be greatly appreciated if the authors could address this.\n\n\nState of the art?:\n\nThe authors should consider moving away from a pixel level reconstruction of the masks of the images and a JEPA-inspired approach in which the SSL objective is within embedding space and not a pixel-level reconstruction. Literature suggests that it will provide more stronger representations: https://arxiv.org/abs/2301.08243 .\n\nUnless there is a convincing argument against using a JEPA-inspired approach, it begs the question if this model is the best model for 2D DDI prediction. Could the authors share their thoughts on why a JEPA-based approach was not considered for me to better understand their architectural choices?", "was_revised": true, "final_justification": "My biggest concern with the manuscript was the lack of comparisons against JEPA-inspired SSL approaches. The authors had run experiments demonstrating that their approach is competitive with JEPA-inspired approaches for DDI prediction.\n\nThe authors similarly addressed other plots that I would have liked to see in the manuscript.", "rebuttal": "Thank you for your detailed review and constructive comments. We address the concerns and questions below.\n\n**Q1:** Could the authors share their thoughts on why a JEPA-based approach was not considered to better understand their architectural choices?\n\n**Response:**\nWe appreciate your insightful suggestion and interest in JEPA-style architectures. Our initial choice of a ViT-based encoder–decoder design was motivated by its demonstrated success in visual molecular representation learning [1] and its compatibility with pixel-level reconstruction, which aligns well with our objective of capturing fine-grained spatial correlations between drug fragments.\n\nThat said, we also agree that JEPA is a compelling alternative for structure-level pretraining. To explore this direction, we implemented an image-level JEPA (I-JEPA [2]) variant of S$^2$VM, trained under the same settings and scale (50K base molecules, 50M drug pairs) for fair comparison. As shown in Table 1, I-JEPA achieves **comparable performance** on the Ryu dataset and slightly underperforms on the Deng dataset. These results suggest that JEPA-style models can capture complementary information and may serve as a promising backbone for future extensions of S$^2$VM.\n\nWe will include these comparative results and release the I-JEPA-based pretrained checkpoints in our revised manuscript. Due to time constraints, we have evaluated I-JEPA only on the small-scale setup, but we plan to further scale up the training to fully assess its potential.\n\n| Method           | Deng's dataset | Ryu's dataset |\n| ---------------- | :---------: | :--------: |\n| I-JEPA (ours)    |    0.7154   |   0.8878   |\n| S$^2$VM (ours) |    0.7712   |   0.8883   |\n\n**Table 1.** Performance (Macro-F1) comparison between ViT-based and JEPA-based S$^2$VM models under small-scale pretraining.\n\n**Q2:** I encourage the authors to report downstream performance as a function of pretraining dataset size. This would help quantify the benefit of large-scale pretraining, which seems underexplored in the current paper.\n\n**Response:**\nWe have conducted additional experiments to evaluate the impact of pretraining dataset size on downstream DDI performance, as detailed in Appendix C.4 (Figures 9 & 10).\n\nAs shown in Table 2, we observe that S$^2$VM benefits consistently from scaling up the number of pretraining drug pairs. The Macro-F1 scores increase steadily from 50M to 300M molecular pairs, with diminishing returns beyond 200M. Based on this trend and computational considerations, we selected 200K base molecules (yielding \\~200M drug pairs) as our default setting.\n\n| #Base Molecules | #Molecular Pairs | Deng's dataset | Ryu's dataset |\n| --------------- | ---------------- | ------------- | ------------ |\n| 50K             | ~50M            | 0.7712        | 0.8883       |\n| 100K            | ~100M           | 0.7963        | 0.9021       |\n| 200K            | ~200M           | 0.8212        | 0.9237       |\n| 300K            | ~300M           | 0.8261        | 0.9295       |\n\n**Table 2.** Macro-F1 scores of S$^2$VM on Deng's and Ryu's datasets across different pretraining data scales.\n\n**W1:** The theoretical analysis in Section 4.2 lacks clarity. A concise summary in the main text and clearer notation in the proof would improve readability.\\\n**Response:** We will add a concise summary of the theoretical insight directly in the main text, rather than deferring entirely to the Appendix. Specifically, we will highlight the core idea as follows:\n> *Our training objective maximizes the mutual information between the blended visible fragments and the original unobserved parts of each molecule, thereby encouraging the model to learn both intra- and inter-molecular dependencies critical for DDI prediction.*\n\nMeanwhile, we will:\n\n* Explicitly clarify that $X_1, X_2$ denote the visible fragments from drug A and drug B (e.g., $A_1, B_2$) used as input;\n* $Y$ denotes the reconstruction target consisting of the missing parts (e.g., $A_2, B_1$);\n* The mutual information term $I(X_1, X_2; Y)$ thus quantifies how well the latent representation captures information necessary to reconstruct the original molecules.\n\nWe will align the theoretical notation with the data flow used in the model (e.g., using $A_1, A_2, B_1, B_2$) and briefly explain how the decomposition supports the design of our fusion-before-encoding framework.\n\n[1] XIANG, Hongxin, et al. A molecular video-derived foundation model for scientific drug discovery. Nature Communications, 2024, 15.1: 9696.\\\n[2] ASSRAN, Mahmoud, et al. Self-supervised learning from images with a joint-embedding predictive architecture. CVPR. 2023. p. 15619-15629.\n\nWe sincerely appreciate your thoughtful suggestions, which will help us further strengthen the impact of our work. We hope our responses have effectively addressed your concerns and provided the necessary evidence to support a more favorable assessment."}, {"review_id": "Nf1KZ8SXdQ", "reviewer": "Reviewer_Wr1x", "summary": "The paper presents S2VM, a self-supervised pretraining scheme for drug–drug interaction modeling. Molecules are rendered as 2-D “visual” images; fragments from two drugs are randomly blended into a single composite image, which an encoder must process so that a decoder can reconstruct the original pair. The resulting pair embeddings, when fine-tuned, deliver state-of-the-art DDI prediction without relying on pre-existing interaction networks or biomedical knowledge graphs.", "questions": "1 How are the molecular images generated and processed? Are they 2D structural diagrams with atoms/bonds drawn, and if so, how do we ensure consistency? Clarifying the image generation pipeline and any augmentation used would help assess robustness. \n\n2  Training on 200M drug pairs is mentioned, which is enormous. What strategies were used to make this feasible?  Also, how does model performance scale with the number of pretraining pairs? If possible, providing insight into the compute time or infrastructure used and the effect of dataset size on performance.\n\n3 The paper presents a theoretical justification for the effectiveness of S2VM. How can this theory guide practical decisions? For instance, does it inform the choice of blending ratio, or the architecture of the decoder?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strengths:\n1 The approach is highly original: representing molecules as images and blending fragments of two drug structure images into one, then training an encoder–decoder to reconstruct the originals from the blend, going beyond traditional single-molecule pretraining.\n\n2 The authors provide both theoretical analysis and extensive empirical validation. S2VM achieves state-of-the-art DDI prediction  performance, with notable Macro-F1 improvements of 4.21% and 3.31% on two benchmark datasets.\n\nWeaknesses:\n1 Images may omit chemical details like atom types or bond orders if not rendered or processed carefully. The paper doesn’t discuss how variations in drawing (different orientations or depiction styles) are handled. Are the molecule images generated via a consistent algorithm and do they ensure invariant features? If the visual representation isn’t canonical, the model might learn image-specific features that don’t generalize. \n\n2 The method focuses purely on structural features of drugs. This neglects other known factors in DDI, such as biological context (targets, pathways) or pharmacological effect data. The authors cite knowledge graph approaches in related work, and indeed S2VM purposely avoids using that information to remain self-supervised. It’s impressive S2VM achieves SOTA without explicit biomedical network data, but it might also miss interactions that are pharmacological. The paper does not discuss this trade-off in depth.\n\n3 Clarity could be improved in describing the blending process: do they simply overlay fragments from two molecule images into one image? How are fragments chosen or positioned? A more detailed algorithm or visual example in the main text would help readers replicate this transformation.", "was_revised": true, "final_justification": "I keep my positive review after the rebuttal.", "rebuttal": "Thank you for your thoughtful and encouraging review. The point-to-point answer is provided below.\n\n**Q1:** How are the molecular images generated and kept consistent? Are they 2D diagrams with atoms and bonds, and is any augmentation used?\\\n**Response:** We generate molecular images through a standardized and reproducible pipeline designed to ensure visual consistency and structural fidelity:\n1. **Canonicalization of SMILES**: All molecules are first canonicalized using RDKit to obtain a unique and deterministic SMILES representation, eliminating variations due to atom ordering or tautomers.\n2. **Image Rendering via RDKit**: The 2D molecular structures are rendered using RDKit’s `MolsToGridImage` function, which explicitly depicts atoms and bonds. Each molecule is rendered as a 224×224 pixel image without any stochastic augmentation to ensure deterministic and consistent visual representation across different runs.\n3. **Layout Standardization**: We fix all layout-related parameters, including sub-image spacing, drawing style, and molecule alignment, to ensure that chemically identical molecules yield identical image representations.\n\nThe full image generation pipeline is available in our codebase for reproducibility. We will clarify this process in the revised manuscript and consider exploring augmentation strategies in future work to enhance generalization.\n\n**Q2:** How was training on 200 million drug pairs made feasible, and how does model performance scale with dataset size?\\\n**Response:** We designed S$^2$VM with scalability in mind.\n + **Pretraining efficiency:**\nTo ensure feasibility, we use a lightweight ViT encoder (12 layers, embedding dim 192) and patch size of 16×16, which significantly reduces the computational cost while maintaining performance. We also apply efficient data loading and batching strategies without extensive augmentations (Appendix B).\n+ **Effect of Dataset Size:**\nWe conducted systematic experiments to evaluate how model performance scales with pretraining data size. Specifically, we varied the number of base molecules from 50K to 300K, which corresponds to approximately 50M to 300M drug pairs. As shown in Table 1 and Appendix C.4 (Figure 9 and Figure 10 in main paper), we observe consistent performance improvements with larger datasets, particularly from 50K to 200K molecules. Beyond 200K, the marginal gains begin to plateau, suggesting diminishing returns at very large scales.\n\n| # Base Molecules | # Molecular pairs | Deng's dataset | Ryu's dataset |\n|------------------|-------------------|----------------|---------------|\n| 50k              | ~50M              | 0.7712         | 0.8883        |\n| 100k             | ~100M             | 0.7963         | 0.9021        |\n| 200k             | ~200M             | 0.8212         | 0.9237        |\n| 300k             | ~300M             | 0.8261         | 0.9295        |\n\n**Table 1.** The performance of S$^2$VM on different pretraining data scales.\n\nWe will include further discussion and clarification in the revised manuscript.\n\n**Q3:** The paper presents a theoretical justification for the effectiveness of S$^2$VM. How can this theory guide practical decisions? For instance, does it inform the choice of blending ratio, or the architecture of the decoder?\\\n**Response:** Our theoretical analysis, which frames S$^2$VM as maximizing mutual information between masked and blended molecular fragments, provides valuable guidance for several design choices.\n+ **Blending ratio:**\nThe theory supports the idea that mutual information is maximized when both source distributions contribute sufficiently to the reconstruction target. This insight informed our choice of a balanced blending ratio (e.g., 0.5:0.5), and we validated this empirically in Appendix C.1 (i.e., Table 9 in the main paper), where extreme ratios (e.g., 0.7:0.3) yielded lower performance as shown in Table 2.\n+ **Decoder architecture:**\nThe theory emphasizes the need to recover both intra- and inter-molecular dependencies. This motivated us to adopt a lightweight but expressive decoder, sufficient to reconstruct fine-grained structural features from the fused latent space without overpowering the encoder’s role in learning meaningful representations.\n\nWe will incorporate a brief discussion of this connection in the revised paper.\n| $(p_1 : p_2)$ | Deng’s dataset | Ryu’s dataset |\n|:------------:|:--------------:|:-------------:|\n| 0.7:0.3          | 81.59          | 91.76         |\n| **0.5:0.5**      | **82.97**      | **92.53**     |\n| 0.3:0.7          | 80.87          | 92.08         |\n|\n\n**Table 2.** The performance of S$^2$VM with various mask ratios.\n\n**W1:** The current method focuses solely on structural features, potentially overlooking pharmacological or biological factors involved in DDIs. Please discuss this trade-off.\\\n**Response:**\nS$^2$VM focuses on structural learning to support new or under-annotated drugs, where biological context is often missing. While knowledge-enhanced models like MUFFIN [1] perform well on common drugs, they rely on entity coverage and may struggle with rare or unseen cases. Our comparison in Table 3 shows that S$^2$VM achieves comparable results on common settings, but significantly outperforms MUFFIN on rare events with up to 32.5% higher accuracy. This highlights that structure-driven self-supervised learning offers stronger generalization under low-resource scenarios. We will elaborate on this trade-off in the revised manuscript and consider integrating knowledge graphs into S$^2$VM in future work.\n|           | Deng's dataset |              |  Ryu's dataset |              |\n|-----------|:--------------:|:------------:|:--------------:|:------------:|\n|           | Common setting | Rare setting | Common setting | Rare setting |\n| MUFFIN    | 0.8269         | 0.4597       | 0.9510          | 0.6428       |\n| S$^2$VM       | 0.9105         | 0.6854       | 0.9586         | 0.9675       |\n| impr. (%) | 8.4%           | 22.6%        | 0.8%           | 32.5%        ||\n|\n\n**Table 3.** The Accuracy of S$^2$VM and MUFFIN on Deng's and Ryu's datasets under different settings.\n\n**W2:** The blending process lacks clarity. How are fragments selected and positioned, and could a visual example help replication?\\\n**Response:** We adopt an anchor-based replacement strategy: one molecule (e.g., $d_u$) serves as the structural anchor, and a fixed proportion of its visual fragments (e.g., 30%) are randomly masked. The corresponding fragments from the second molecule $d_v$ are then inserted at the same positions, forming the blended input. This leads to an effective fragment ratio of 0.7:0.3 between $d_u$ and $d_v$ in the final representation.\nThis strategy ensures spatial alignment while introducing meaningful structural variation across training pairs. We will include a visual illustration of this blending process in the revised paper to enhance clarity and reproducibility.\n\n[1] CHEN, Yujie, et al. MUFFIN: multi-scale feature fusion for drug–drug interaction prediction. Bioinformatics, 2021, 37.17: 2651-2658.\n\nThank you again for your valuable insights, which will help us further strengthen the presentation and impact of our work."}, {"review_id": "0Db0sWbNa4", "reviewer": "Reviewer_pbip", "summary": "The paper propose S2VM, a method for the problem of drug-drug interaction. The key idea is its self-supervised learning framework that learns the presentations of pairs of drugs (2 molecules) from a large collection of drug pairs without label. It shows a high performance in predicting DDI in various datasets.", "questions": "- It would make sense in biology to consider the effect of drugs on proteins and pathways that give rise to DDI, why the paper do not use this data?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "I find the key idea of the paper is interesting. Usual method would have two different parts, learning the presentations of individual drugs and learning the interaction between the presentations to predict the label. Given that there are few labeled and many unlabeled pairs, the method here can exploit a much larger datasets. The choice of learning representation of pairs directly instead of each drug is reasonable, blending both parts of the previous methods into one. \nThe paper is clear and easy to understand for me. Its performance is convincing. \nI could not check the theory in detail.", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed review and constructive comments. We’re glad you found the idea interesting, the methodology sound, and the experiments robust.\n\n**Q1:** It would make sense in biology to consider the effect of drugs on proteins and pathways that give rise to DDI, why the paper do not use this data?\\\n**Response:** We focus on molecular structures from large-scale unlabeled data to address new or under-represented drugs, where protein or pathway data are often missing from biomedical knowledge graphs. Many DDIs, especially those related to drug metabolism such as enzyme inhibition or induction, are may driven by molecular substructures [1] (Nature Machine Intelligence, 2024). This makes structure-level modeling particularly relevant.\nOur results support this view. While S$^2$VM and MUFFIN perform similarly on common drugs (Table 1), S$^2$VM achieves 22.6%  and 32.5% relative gains in rare-event settings on Deng’s and Ryu’s datasets, respectively. This suggests that knowledge-based models may struggle when entity coverage is low, whereas S$^2$VM generalizes more effectively by learning directly from structure.\n\nWe acknowledge that integrating biomedical knowledge into S$^2$VM is a valuable future direction and plan to explore it further.\n|           | Deng's dataset |              |  Ryu's dataset |              |\n|-----------|:--------------:|:------------:|:--------------:|:------------:|\n|           | Common setting | Rare setting | Common setting | Rare setting |\n| MUFFIN    | 0.8269         | 0.4597       | 0.9510          | 0.6428       |\n| S$^2$VM       | 0.9105         | 0.6854       | 0.9586         | 0.9675       |\n| impr. (%) | 8.4%           | 22.6%        | 0.8%           | 32.5%        ||\n|\n\n**Table 1.** The Accuracy of S$^2$VM and MUFFIN on Deng's and Ryu's datasets under different settings.\n\n[1] ZHONG, Yi, et al. Learning motif-based graphs for drug–drug interaction prediction via local–global self-attention. Nature Machine Intelligence, 2024, 6.9: 1094-1105.\n\nThank you once again for your valuable feedback, which has helped us improve our work. We believe these additions will strengthen our model's presentation."}, {"review_id": "o2MxCm7LD2", "reviewer": "Reviewer_Kenp", "summary": "The paper introduces S2VM, a novel self-supervised pretraining framework designed to improve the prediction of drug-drug interactions (DDIs). The core problem it addresses is the poor performance of existing models on new or data-scarce drugs, which stems from an insufficient exploration of unlabeled drug pairs and weak fusion of structural information. To solve this, S2VM employs a \"pre-fusion\" strategy. It represents pairs of drugs as 2D images, splits them into visual fragments, and \"blends\" these fragments into a single, unified input.  This blended input is processed by a Transformer-based encoder-decoder architecture. The model is pretrained on a massive dataset with a self-supervised objective: to reconstruct the original, separate molecular images from the blended representation. The resulting pretrained encoder is then fine-tuned for DDI prediction, where it demonstrates state-of-the-art performance.", "questions": "1. The paper states that 2D images capture spatial relationships of visual molecules. However, functional groups that appear distant in a 2D projection can be very close in 3D space due to molecular folding, which is critical for interactions. How does the model account for this fundamental discrepancy, and what is the justification for accepting this potential loss of critical 3D conformational information compared to using 3D structural data?\n\n2. Why were 2D images chosen as the input modality over molecular graphs, which are a more direct and common representation of molecular structure? Given that graphs explicitly encode atom connectivity and bond types, while an image-based model must learn these rules implicitly, what are the perceived advantages of the visual approach that outweigh the benefits of graph-based methods for this task?\n\n3. The architecture uses two distinct linear projection heads (E1 and E2) to reconstruct d_u and d_v, respectively. This design is inherently asymmetric. Consequently, providing the input pair as (d_u, d_v) versus (d_v, d_u) would result in each drug being processed by a different reconstruction head, potentially leading to different loss values and training dynamics. What is the rationale behind this asymmetric design rather than using a single, shared reconstruction head for both molecules?\n\n4. The structure-level encoding involves a stochastic sampling process to blend the two molecular images. This implies that for a single trained model, the same drug pair can yield different latent representations and potentially different predictions across multiple inferences. How large is the variance brought by sampling?\n\n5. The model is pretrained on a large dataset of molecules from PubChem before being fine-tuned and tested on benchmark datasets. Was a rigorous data leakage check performed to ensure that all molecules present in the validation and test sets of the downstream tasks were explicitly excluded from the large-scale pretraining dataset? If not, the model's reported performance, especially on \"new\" drugs, might be artificially inflated.\n\n6. The paper claims that the joint reconstruction objective enables the model to learn mutual information and structural correlations between the two drugs. To validate this central claim, a crucial control experiment would be to pretrain the model using a standard masked autoencoder objective on single molecules (i.e., masking and reconstructing fragments of only one molecule at a time). If this single-molecule pretraining yields similar or better downstream DDI performance, it would suggest the model's strength derives from learning robust single-molecule features, rather than from learning interaction information between pairs. Was such a control experiment considered?\n\n7. During the downstream DDI prediction task, the model's input is the blended matrix of visual fragments, meaning the encoder is not provided with the complete structure of either molecule at the moment of prediction. This design appears to intentionally sacrifice complete structural information for the sake of maintaining the fusion-before-encoding architecture. What is the rationale for this choice? Why is making a prediction from this partial, mixed representation considered superior to a more conventional approach for the downstream task, such as encoding each complete molecule separately with the pretrained encoder and then combining their representations for a final prediction?\n\nOf all the issues raised, Questions 5 and 6 are the most critical to me. A convincing response to these two concerns would strongly incline me to give a higher rating.", "limitations": "1. Use of 2D Representations: The authors explicitly acknowledge that using 2D molecular images ignores 3D conformational structures, which are critical for understanding physical interactions between molecules.  This represents a fundamental trade-off between computational scalability and representational granularity. \n\n2. Neglect other biological entities: This paper focuses on molecular features, neglecting other biological entities involved in drug interaction events, such as proteins, pathways, and diseases, which are crucial to identifying DDIs.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "strengths:\n1. Large-Scale Self-Supervised Learning: By pretraining on 200 million unlabeled drug pairs, the model learns generalizable structural representations that are not confined to the limited scope of known DDIs.  This directly addresses the key challenge of data scarcity for novel and few-shot drugs. \n2. Strong Empirical Performance: The paper provides robust evidence of the model's superiority. It achieves significant improvements over state-of-the-art baselines on multiple datasets (Deng's, Ryu's, and TWOSIDES) and across various settings, including common, few-shot, and inductive (new drug) scenarios. \n\nweaknesses:\n1. High Computational Cost: Pretraining a Vision Transformer model on 200 million pairs of images is extremely resource-intensive.  This high computational barrier could make it difficult for researchers with limited resources to reproduce the results or build upon the work.\n2. Simplistic Blending Mechanism: The \"blending\" is achieved by randomly sampling patches from each of the two drug images.  While empirically effective, this random approach might not be optimal. A more informed sampling strategy based on preliminary structural analysis could potentially yield even better results.", "was_revised": true, "final_justification": "Most of my concerns are addressed, especially Q5 and Q6. I raise my rating to 4.", "rebuttal": "Thank you for your detailed review and constructive suggestions. We address the concerns and questions below.\n\n**Q1:** What is the justification for accepting this potential loss of critical 3D conformational information compared to using 3D structural data?\\\n**Response:** We acknowledge that 3D conformations play a critical role in drug interactions. While S$^2$VM does not aim to model full 3D structures, it leverages blended visual fragments to approximate pseudo-spatial proximity between functional groups. This design allows efficient and scalable modeling of local structural co-occurrence patterns. As shown in our interpretability analysis (Figure 5), S$^2$VM consistently highlights biologically meaningful substructures (e.g., 1,3-benzodioxole), indicating its ability to capture structure-dependent features. We also explicitly recognize in the Limitation section that extending to 3D or multimodal representations is a promising future direction.\n\n**Q2:** Why were 2D images chosen as the input modality over molecular graphs?\\\n**Response:** We chose 2D molecular images as the input modality because they provide an intuitive and spatially continuous view of molecular structures, making it easier to capture functional group arrangements [1] (Nature Machine Intelligence, 2022). Compared to graphs, images are often more robust—minor errors or missing bonds in molecular graphs may break connectivity, whereas images still preserve chemical context visually. This visual format also facilitates the use of powerful vision models, offering a natural path toward enhancing drug safety analysis through structure-aware attention. Empirically, S$^2$VM outperforms graph-based methods (e.g., SSI-DDI, MRCGNN) across standard and few-shot settings (Table 1–3 and Figure 4 in the main paper), validating the effectiveness of visual representations for DDI prediction.\n\n**Q3:** Why does the model use two separate projection heads for the two drugs instead of a shared one, given the resulting asymmetry?\\\n**Response:** The use of two distinct projection heads reflects a deliberate choice to model the inherent asymmetry in drug interactions. In real-world DDI scenarios, drug pairs often exhibit asymmetric roles. For example, one drug may alter the metabolism or concentration of the other. This asymmetric nature is well-documented in biomedical literature and has been emphasized in prior work [2] (Nature Machine Intelligence, 2024), which frames DDI prediction as a direction-sensitive task. An example of the 72-th Event type is: *“The serum concentration of Drug2 can be increased when combined with Drug1”* where Drug1 acts as the perpetrator and Drug2 as the victim. By assigning separate reconstruction heads to $d_u$ and $d_v$, the model learns **role-specific structural representations**, which are aligned with the biological reality of DDIs and contribute to more faithful pretraining dynamics. This confirms that explicitly modeling role-specific representations brings tangible benefits and better aligns with the asymmetric nature of real-world drug interactions.\n\n**Q4:** Since the model uses stochastic blending during training, does this cause significant variance or inconsistency in predictions for the same drug pair during inference? \\\n**Response:** \nWe clarify that stochastic blending is applied only during pretraining to encourage robust representation learning. For downstream DDI prediction, the model receives deterministic inputs by concatenating the visual tokens of the two drugs in a fixed order, with no randomness involved during inference.\nThis practice is consistent with standard MAE-style reconstruction pretraining, where masking is used only during training, while downstream tasks use the full input [3,4]. Additionally, in Appendix C.7, we report results across different blending strategies for DDI prediction, showing the model’s robustness to fusion variations.\n\n**Q5:** Was there a rigorous check to prevent data leakage from the PubChem pretraining set into the benchmark test sets? \\\n**Response:** We confirm that our pretraining process is entirely self-supervised and task-agnostic, with **no access to DDI-related labels or annotations**. This eliminates any risk of label leakage and follows standard practice in representation learning frameworks such as MAE [3] and MoCo [5], where the pretraining stage is decoupled from downstream tasks. \n\nTo assess potential data overlap, we rigorously analyzed the intersection between the 200K molecules used in pretraining and the test sets of benchmark datasets. We identified no shared drugs in the TWOSIDES dataset under the inductive S1 and S2 settings, and only one shared molecule (DrugBank ID: DB01351) in Deng’s and Ryu’s datasets (<0.001%, settings for both drugs are existing). **This negligible overlap makes structural memorization extremely unlikely**.\n\nTo further ensure strict evaluation, we removed all DDI instances involving the shared molecule from the test sets (34 out of 7,474 in Deng’s and 93 out of 38,349 in Ryu’s). As shown in Table 1, the model’s performance remains virtually unchanged, confirming that our results are unaffected by any potential data leakage.\n|     | Deng's dataset |          | Ryu's dataset |          |\n|-----|:--------------:|:--------:|:-------------:|:--------:|\n|     | ACC.           | Macro-F1 | ACC.          | Macro-F1 |\n| S$^2$VM | 0.9105         | 0.8212   | 0.9586        | 0.9207   |\n| S$^2$VM (removed) |  0.9117        |  0.8193  |  0.9580       |  0.9203  |\n|\n\n**Table 1.** Performance comparison after removing the single shared drug from the test sets.\n\nWe will clarify this verification and filtering process more explicitly in the revised manuscript.\n\n**Q6:** The paper claims that the joint reconstruction objective enables the model to learn structural correlations between the two drugs. Was a single-molecule pretraining control experiment considered to validate this claim?\\\n**Response:** We conducted the suggested control experiment by pretraining a ViT-based masked autoencoder (MAE) using single-molecule reconstruction only. This baseline uses the same architecture, the same 200K PubChem molecules, and identical training epochs as S$^2$VM, ensuring a fair comparison. During downstream DDI prediction, we adopted a post-fusion strategy where each drug is encoded individually and the resulting representations are concatenated for classification.\n\nAs shown in Table 2, the single-molecule MAE baseline consistently underperforms S$^2$VM across all metrics and datasets. This demonstrates that joint reconstruction pretraining yields more expressive and interaction-aware representations, beyond what is achievable through standard single-drug encoding.\n\n|          | Deng's dataset |          | Ryu's dataset |          |\n|----------|:--------------:|:--------:|:-------------:|:--------:|\n|          | ACC.           | Macro-F1 | ACC.          | Macro-F1 |\n| MAE      | 0.8276         | 0.6961   | 0.9215        | 0.8920    |\n| S$^2$VM     |  0.9105        |  0.8212  |  0.9586       |  0.9207  |\n|\n\n**Table 2.** Performance comparison between S$^2$VM and single-molecule representation baselines.\n\nWe will add this control experiment and detailed discussion in the revised manuscript to further strengthen the theoretical claims.\n\n**Q7:** Why does the model use a blended input for DDI prediction, instead of encoding each full molecule separately and combining them later?\\\n**Response:** The model uses full molecular structures during inference, with stochastic blending applied only in pretraining to enhance representation learning (see response to Q4). Our choice to maintain fusion-before-encoding is based on: **1)** Many DDIs arise from localized interactions between functional substructures, not from global structural similarity [2,6]. By blending fragments early, the encoder can directly model cross-molecular spatial dependencies, which are crucial for detecting such interactions. **2)** In Appendix C.1 (Table 7), we further compare pre- and post-fusion strategies using molecular fingerprints, again showing the advantage of early fusion. Similarly, in Figure 3, early fusion outperforms a shared-encoder baseline that encodes drugs separately and fuses representations afterward.\n\nThese results demonstrate that predicting from fused inputs is not a compromise, but a deliberate and effective design choice that enhances structural interaction modeling. We will make this rationale more explicit in the revised paper.\n\n**W1:** The method requires high computational resources, which may hinder reproducibility.\\\n**Response:** The model can be run on machines with >=16GB GPU RAM. To support reproducibility and broader adoption, we provide pretrained checkpoints and code for easy integration and evaluation.\n\n[1] Zeng, Xiangxiang, et al. \"Accurate prediction of molecular properties and drug targets using a self-supervised image representation learning framework.\" Nature Machine Intelligence (2022)\\\n[2] ZHONG, Yi, et al. Learning motif-based graphs for drug–drug interaction prediction via local–global self-attention. Nature Machine Intelligence, 2024\\\n[3] HE, Kaiming, et al. Masked autoencoders are scalable vision learners. CVPR. 2022.\\\n[4] ZHAO, Fan, et al. Self-supervised feature adaption for infrared and visible image fusion. Information Fusion, 2021\\\n[5] HE, Kaiming, et al. Momentum contrast for unsupervised visual representation learning. CVPR. 2020.\\\n[6] Nyamabo, A. K., Yu, H., & Shi, J. Y. (2021). SSI–DDI: substructure–substructure interactions for drug–drug interaction prediction. Briefings in Bioinformatics, 22(6).\n\nThank you once again for your valuable feedback, which has helped us improve our work. We hope these clarifications effectively address your key concerns and provide sufficient justification for a higher evaluation."}]

Author rebuttal:

Meta-review: This paper proposes a self-supervised framework for drug-drug interaction (DDI) prediction. During pretraining, S2VM considers pairs of molecules as 2D images, and jointly encodes the pairwise information using Vision Transformer by reconstructing each molecule’s image from a blended image. The resulting pretrained encoder is fine-tuned for DDI prediction. Extensive experiments demonstrate the effectiveness of the proposed method. The reviewers initially suggested the following strengths and weaknesses:

## Strengths:
- The paper presents an original contribution to the field.
- Extensive experiments demonstrate the effectiveness of the suggested self-supervised objective trained on large-scale (~200M) unlabeled pairs.

## Weaknesses:
- The method only uses 2D images as input, while neglecting potentially informative features, such as biological context.
- The justification for the choice of input modality should be provided. S2VM does not use graph-based molecule representation, despite the fact that it is more direct and a common representation of molecular structure.
- The comparison against JEPA-inspired approach (I-JEPA [1]) is not sufficient.
- The supporting analysis to validate that the model truly learns mutual information and structural correlation between two drugs is insufficient.

The authors have adequately addressed most of the reviewers’ concerns, though some clarifications and additions remain necessary. In particular, the authors are required to provide the additional experiments presented during the rebuttal, especially the comparisons with MUFFIN [2]. Furthermore, the inference phase of the model should be clarified (Reviewer Kenp Q4) for better understanding. The authors need to include experiments conducted during the rebuttal and revise the content related to the inference process in the revised paper.

Since all reviewers provided positive ratings after the rebuttal, and considering the importance of the work in DDI as well as its powerful empirical performance, I recommend acceptance to NeurIPS.

[1] Assran, Mahmoud, et al. “Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture.” CVPR, 2023.

[2] Chen, Yujie, Muffin: multi-scale feature fusion for drug–drug interaction prediction. Bioinformatics, 37(17): 2651–2658, 2021.
Decision: Accept (poster)