GRAM-DTI: Adaptive Multimodal Representation Learning for Drug–Target Interaction Prediction
Authors: Feng Jiang
Amina Mollaysa
Hehuan Ma
Yuzhi Guo
Tommaso Mansi
Junzhou Huang
Mangal Prakash
Rui Liao
Venue: ICLR 2026 Poster (2026)
Keywords: Drug-target interaction prediction
Multimodal representation learning
Adaptive modality dropout
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)

Abstract: Drug target interaction (DTI) prediction is a cornerstone of computational drug discovery, enabling rational design, repurposing, and mechanistic insights. While deep learning has advanced DTI modeling, existing approaches primarily rely on SMILES–protein pairs and fail to exploit the rich multimodal information available for small molecules and proteins. Inspired by recent successes in multimodal molecular property prediction, we introduce GRAM-DTI, a pre-training framework that integrates multimodal small molecule and protein inputs into a unified representation. GRAM-DTI extends volume-based contrastive learning to four modalities, capturing higher-order semantic alignment beyond conventional pairwise approaches. To handle modality informativeness, we propose adaptive modality dropout, dynamically regulating each modality’s contribution during pretraining. Additionally, IC50 activity measurements, when available, are incorporated as weak supervision to ground representations in biologically meaningful interaction strengths. Experiments on four publicly available datasets demonstrate that GRAM-DTI consistently outperforms state-of-the-art baselines. Our results highlight the benefits of higher-order multimodal alignment, adaptive modality utilization, and auxiliary supervision for robust and generalizable DTI prediction. Our code is available at https://github.com/uta-smile/GRAM-DTI.

Published as a conference paper at ICLR 2026
GRAM-DTI: ADAPTIVE MULTIMODAL REPRESENTA- TION LEARNING FOR DRUG-TARGET INTERACTION PREDICTION
Feng Jiang' Amina Mollaysa Hehuann Ma', Yuzhi Guo', Tommaso Mansi",
Junzhou Huang'. Mangal Prakash21, Rui Liao² ?
Universityy of Texas at Arlington, "Johnson & Johnson Innovative Medicine
ABSTRACT
Drug target interaction (DTI) prediction is a cornerstone of computational drug discovery, enabling rational design, repurposing, and mechanistic insights. While deep learning has advanced DTI modeling, existing approaches primarily rely on SMILES protein pairs and fail to exploit the rich multimodal information available for small molecules and proteins, We introduce GRAM-DTI, pre- training framework that integrates multimodal molecular and protein inputs into unified representations. GRAM-DTI extends volume-based contrastive learning to four modalities, capturing higher-order semantic alignment beyond conventional pairwise approaches. To handle modality informativeness, we propose adaptive modality dropout, dynamically regulating each modality's contribution during pre- trriirriiin..... Additionally, IC50 activity measurements, when available, incor- porated as weak supervision to ground representations in biologically meaningful interaction strengths. Experiments four publicly available datasets demonstrate that GRAM-DTI consistently outperforms state-of-the-arr baselines. Our results highlight the benefits of higher-order multimodal alignment, adaptive modality utilization, and auxiliary supervision for robust and generalizable DTI prediction. Our code is available at wttptts://ggittae com/ at smi le/ GRAM-DTI
INTRODUCTION
Drug target interaction (DTT) prediction is central challenge in computational drug discovery, derpinning applications in rational drug design, repurposing of approved drugs, and elucidation of mechanisms of action (Vefghi et al. 2025 Jiang al. [024b] Ma al. 2022). Traditional ex- perimental screening, though reliable, is prohibitively expensive and cannot feasibly cover the vast chemical and proteomic search space (Jiang et al. 2024a Ma al. 2024). Computational methods
therefore play an increasingly critical role in prioritizing candidate drug-protein pairs for exper- imental validation, accelerating discovery pipelines and reducing cost (Panahandeh & Mansouri]
2025
Liao et al. 2025).
DTI prediction methods have evolved from similarity-based and network-based heuristics to ma- chine learning and, more recently, deep learning approaches (Shi et al. 2024 Paanahandeh & Man- souri 2025). Early methods relied on molecular similarity or interaction propagation but struggled with generalization. Modern neural models, including graph neural networks and sequence-based architectures now dominate, learning directly from raw SMILES and amino acid sequences (Peng et al. 2024 Zhao et al., 2025 Liu et al. 2025 Xia et al. 2023). However, these approaches remain largely restricted to SMILES protein pairs, overlooking the richer multimodal information available for molecules and proteins that could yield more robust and generalizable interaction predictions.
While multimodal pre-training has been recently explored by a few works for DTI prediction (Lu et al. 2025 Ye et al.) 2021] CCen al. 2020). existing approaches suffer from three limitations. Firstly, they rely on pairwise contrastive learning anchored to a single modality. Such schemes can- capture higher-order interdependencies the number of modalities increases (Cicchetti al.
'These authors contributed equally as first authors. Conresoondence to: fxj88 4 43 (mavs. udu.edu
1 These authors contributed equally as senior (last) authors.

---

Published as a conference paper at ICLR 2026
Pretraining
Downstream task
Protein
Molecule
F
6,
HTA
Protein
shone
shore
Multimodal Alignment
Molecule
Fr
E
Motecular
Volume based Multimodal Alignment
Trainoble
5
modalities
Fixed
C
g
Gradient-Infemmet
Modality Dropout
fust
En
Encoder
F6 Projector
modalities
Gradion
5
Emboechlig
MLPs
Figure I: Overview of GRAM-DTI architecture. Left: pretraining phase with volume-based multi- modal alignment across four modalities (SMILES, text, HTA, protein sequences). The framework uses gradient-informed adaptive modality selection to dynamically regulate modality contributions during training. Right: downstream task prediction.
2024). Secondly, they assume all modalities are equally informative, ignoring that data sources often differ in quality, completeness, and relevance across samples and training stages. Static fu- sion can therefore lead to suboptimal representations when dominant but less informative modalities overshadow complementary signals. Finally, valuable supervision signals such IC50 activity mea- surements are publicly available for subset of pre-training despite their direct biological relevance for DTI prediction task.
pairs, yet they remain unutilized during
To address these gaps, we propose GRAM-DTI, a novel multimodal pre-training framework specif- ically tailored for downstream DTI prediction task (see Fig. [T]. To this end, curate high quality multimodal dataset consisting of diverse protein and small molecule modalities and adapt the re- cent volume-based contrastive learning strategies from other domains (Cicchetti al., 2024) Jiang et al. 2025) for geometric alignment of these modalities. Unlike traditional contrastive learning techniques, this offers theoretically principled and scalable approach for aligning multiple modal- ities. Beyond volume based contrastive learning, our framework is novel in its flcxibility to learn dynamically weight each modality based on its informativeness during pre-training while also supporting activity-base labels as auxiliary supervisory signals, when available. Our main contri- butions are as follows.
We introduce GRAM-DTI, pre-training framework for DTI that integrates multimodal small molecule protein and modalities into a unified representation with molume-basse con- trastive learning.
We introduce adaptive modality dropout, dynamically regulating modality contributions during pre-training to prevent dominant but less informative modalities from overwhelming complementary signals.
We leverage IC50 activity measurements as additional weak auxiliary supervision, ground- ing learned representations in biologically meaningful drug target interactions.
We demonstrate state-of-the-ar performance across four public datasets and multiple eval- uation settings relevant for real-world drug discovery applications.
RELATED WORKS
Multimodal Molecular Representation Learning Recent advancements in molecular represen- tation learning have shifted towards integrating multiple data modalities to enhance predictive per- formance. For instance, frameworks like TRIDENT (aaag et al. 2025) combine SMILES strings, hierarchical taxonomic annotations, and functional text of small molecules to capture richer molec- ular semantics. These approaches leverage contrastive learning pretraining align diverse data
2

---

Published as a conference paper at ICLR 2026
sources, which improves generalization across various molecular downstream tasks even in the ab- sence of fully paired datasets. Beyond TRIDENT, several molecular foundation models have been introduced, including MolFM (Luo et al. 2023 and MolCA (Liu et al. 2023), which integrate molecular graphs, textual deseriptions, and domain-specific annotations into unified representations. These works highlight the broader trend of leveraging multimodal pre-training to construct general- purpose molecular representations.
Drug-Target Interaction (DTI) Prediction DTT prediction has traditionally relied unimodal representations, such as SMILES strings for drugs and amino acid sequences for proteins. Early deep learning models such DeepDTA (Öztürk et al. 2018). MT-DTI (Shin al., 2019). and TransformerCPI (Chen et al. 2020) demonstrated the effectiveness of sequence-based architectures for interaction prediction. Beyond sequence- based methods, more recent work has explored graph neural networks and SE(3)-equivariant geometric deep learning models, such as GraphDTA (Nguyen al., 202T) and EquiBind (Stärk al., 2022). which leverage spatial and structural information of drugs and proteins to enhance binding affinity prediction. In parallel, knowledge graph-based meth- ods such as NeoDTI (Wan et al., 2019) and Hetionet-based repurposing frameworks (Himmelstein iet al. 2017) exploit biomedical networks to capture higher-order relations among drugs, targets, and diseases. More recently, multimodal approaches have been proposed to better capture the complex- ity of ttagett interactions. For example, MDTipps (Xia al. (023) integrates knowledge graphs, gene expression profiles, and structural information, while MGNDTI (Peng et al.) (2024) employs a multimodal graph neural network to improve robustness and generalization. Another emerging di- rection is pre-training with large-scale unlabeled data to mitigate the scarcity of labeled DTI pairs. For instance, DTTAM (Lu al. 2025 introduces separate pretraining for drug and target modalities before merging the learned representations for DTI prediction.
Modality Dropout Modality dropout techniques have been proposed enhance the robustness of multimodal models by preventing over-reliance on any single modality. For instance, the Learnable Irrelevant Modality Dropout (IMD) method (Alfasly et al. 2022) selectively drops irrelevant modal- ities during training, improving performance in multimodal action recognition tasks. Additionally, approaches like aggressive modality dropout have been shown to mitigate negative co-learning ef- fects and enhance model accuracy in multimodal settings (Magal et al., (025). Beyond dropout, adaptive fusion mechanisms have also been investigated. Cross-attention and gating strategies (Tsai let al. 2019; Peng et al. 2024; Mollaysa et al.) (2025) dynamically regulate modality contributions, while tensor fusion methods (Zadeh et al. (017)) capture higher-order interactions across modali- ties. These ideas inform the design of adaptive strategies in molecular contexts, where modality informativeness often varies across data sources and training stages.
Unlike existing works, our GRAM-DTI framework captures higher-order semantic relationships beyond simple pairwise alignment/fusion. Furthermore, to the hest of our knowledge, we are the first explore strategies for adaptive modality dropout in the context of DTI prediction.
3 METHODOLOGY
Building upon recent advances (Cicchetti et al. 2024]] Jiang et al., 2025) in volume-based modal- ity alignment for effective representation learning. we extend the foundational concept volume loss (Cicchetti al. 2024), originally formulated for audio-video-tex data, to the domain of protein- small molecule interactions. We aim to learn a unified embedding space that: 1) captures semantic relationships across modalities; 2) remains robust when modalities vary in informativeness; and 3) improves downstream DTI prediction task.
Formally, assume pretraining dataset D = [(fff ./,, sett sef 1 8IC50) | where 2x, uf, a/ and xe denote the SMILES sequence, textual description of molecule, hierarchical taxonomic annotation (HTA) (Jiang et al. 2025) of molecule, and protein sequence, respectively. The variable 8!C50 in- dicates the IC50 activity class !C5 if a measured IC50 value is available for the protein-molecule pair (a/, [z)) and O otherwise. As illustrated in Fig. 1 employ pre- pre-trained encoders E; (Mol- Former (Ross et al. (202)) for SMILES, MolT5 (Edwards et al., (2022) for text and HTA, and ESM-2
(Lin et al. 2023) for proteins) to obtain initial modality-specific embeddings.
3

---

Published as a conference paper at ICLR 2026
To keep pre- vreitrainng efficient and scalable, we freeze the backbone encoders and train lightweight neural projectors F that map each modality embedding into a shared representation space where they are semantically aligned. The resulting projected embeddings are denoted f", where m e {SMILES, text, HTA, protein).
3.1 GRAMIAN VOLUME-BASFRI MULTIMODAL. ALIGNMENT
In contrast to traditional multimodal representation learning approaches which have been known to fail in capturing the complex interdependencies among three or more modalities (Cicchetti et al.) 2024 Jiang et al. 2025), volume loss uses Gramian volume-based alignment of modalities ensuring semantic coherence across all modalities simultaneously.
Gramian Volume Given embeddings fi f. Ed that are learned from the four modali- ties ag x 2 sf respectively, first normalize them such that [| || 1. We can then construct the Gram matrix G E IR 4x4 where
=ky = ( (f), k.j E 18,t,
The 4-dimensional volume spanned by these embedded vectors is equal to the square root of the determinant of the Gramian matrix (Cicchetti et al., 2024); V(fi, fi ft f(¹ = de) From multimodal alignment perspective, smaller volume intuntively suggests stronger semantic alignment, as the embeddings occupy a more compact and cohesive subspace and vice-versa.
Given the Gramian volume, contrastive objective is cast as vol- ume ionimizztiin/mmxxmmzzttti As proposed in (Cicchetti et al.] (024), to construct negative pairs, of the four modalities. Therefore, for batch
Voleme-Based Contrastive Loss
choose anchor modality {s, l, h, of samples, the contrastive loss on their learned embeddings is defined as follows:
B
1
Σ B
log
1=1 2j=1 exp(-V fP)/T)
where, for example, the first modality f is chosen as the anchor a, negative pairs structed by permuting the anchor, and t is the temperature parameter. We also add the ensure symmetric alignment: L
reverse loss (w.rt. negative pairs construction)
ssp(-V(a,,j of (fff)//) Liml log En' (enp-V ff//""
The combined volume-based loss is
1 Lvol b ( + LiTa)
(3)
3.2 GRADIENT- INFORMED ADAPTIVE MODALITY SELECTION
While volume- based contrastive loss treats all modalities equally, different modalities may vary in quality and relevance, with contributions that change during training. Static fusion strategies risk either underutilizing weaker modalities or overfitting to dominant ones. We propose a gradient- informed modality dropout mechanism that dynamically adapts modality usage based on their in- stantaneous contribution to the loss function.
Gradient Contribution Analysis Assume Li denotes mini-batch loss at training step i. We mea- sure the importance of modality E {s,t, h, p) by the magnitude of the gradient with respect its embedding:
0L;
gft
(4)
0.mm
where fm IRd is the learned embedding of modality m gradient step Ê. To avoid noisy decisions, o Dー where
track the history of gradient contributions over the past K steps: 9 Σnー ak
Σ
(0, 1) an exponential decay factor which yields smooth, temporally discounted importance score for each modality.
4

---

Published as a conference paper at ICLR 2026
Adaptive Modality Dropping Strategy We employ principled adaptive strategy that siders both the magnitude and variance of gradient contributions. Let H 2 0;" and
of = Vm mx)² denote the mean and standard deviation of weighted gradients across modalities the current gradient step i. We will drop modality from the volume based contrastive loss calculation with a probability of Pdrop- which is a hyperparameter. The criteria to drop modality defined as follows:
arg max of'
dominance detected, e.g., 97" > Mi + N00
m
drop
arg minm Dt
otherwise,
(5)
none
with probability (1 Pámop)
where >, = 1.5 is the threshold multiplier. This means that we adaptively drop modalities based on two criteria: 1) Dominance prevention: modality's contribution is much larger than others, we drop it to avoid overfitting; 2) Low-contribution pruning: Otherwise, we drop the modality with the smallest gradient contribution to encourage use of more informative signals. This dynamic selection balances stability and diversity, ensuring all modalities remain engaged throughout training.
3.3 WEAK SUPERVISION THROUGH IC50 ACTIVITY MEASURE
the IC50 values for wide range of protein-small molecule pairs are available on public data sources such BindingDB (Gilson al. 2016), introduce additional classification task as an auxiliary objective during pre-training. However, IC50 labels are not available for all possible protein-small molecule pairs, this task provides only weak supervisory signal during pre-training when IC50 information is available. We train a classifier F!C50 to predict the IC50 class from the
learned embeddings of all four modalities: fluwed [ft; fr' R4d, Note that IC50 values are continuous, but given the inherent challenges of IC50 regression, including heterogeneous value distributions, wide dynamic ranges spanning several orders of magnitude, and noisy measurements (Qureshi et al. 2015 Bavi et al.) 2016 Ashraf et al.) 2023), we formulate the problem as a three-class classification task by employing discretizations IC50 values (see Appendix(A)
However, this discretization comes with class-imbalance described in Appendix issue, employ weighted cross-entropy loss:
A
To address this
Lics
Σ wy log p(u. )(ueed)
((uilf(ued)
(6)
S
where S denotes the set of samples with valid IC50 annotations, and class weights computed as: We C.N. where Nooal being the total number of samples, C the number of classes, and Na the number of samples in class c.
Now
Auxiliary Bimodal Contrastive Loss As the downstream task involves protein and molecule em-
beddings only, to emphasize aligmment between these two, we also explicitl incorporate traditional pairwise contrastive losses between SMILES and protein modalities:La: = (L + C where
Cg-p and Cp-+ follow the standard CLIP-style contrastive formulation (Radford al. 2021)
3.4 UNIFTED TRAINING OBJECTIVE
The complete training objective integrates all components with appropriate weighting:
Liotal
Lua + AgLai + Cic50
where A1, A2 Aa are hyperparameters. Note that Lyal and Lai are applied on all the training instances while Licso only applied for pairs of protein and molecule with valid IC50 annotations. For gradient-base based dropping of modality in volume contrastive loss, we use = dgL + X3 See Appendix C details model architecture and parameters.
5

---

Published as a conference paper at ICLR 2026
4 EXPERIMENTS
4.1 DATASET
For wre-training, we employ the multimodal molecular dataset from TRIDENT (Jiag et al. 2025 consisting of 47,269 triplets of SMILES, text descriptions, and HTA annotations. We extend this dataset by integrating protein binding information from BindingDB Gilson et al. 2016). creating quadruplets of (SMILES, Text, HTA, Protein) with IC50 measurements when available. To prevent data leakage, we removed overlapping (SMILES, protein) pairs from our downstream evaluation datasets. The final pretraining dataset contains 6,545 unique molecules and 4,418 proteins, forming 50,968 quadruplets, of which 16,035 include quantitative IC50 measurements for auxiliary super- vision. See Appendix B for detailed dataset construction and statistics. Ideally, we would remove any drug or target that appears in the downstream tasks from the pretraining corpus. However, given the number of downstream tasks we evaluate, this would leave too little data for effective pretrain- ing. Consequently,
only exclude overlaps at the (SMILES, protein) pair level. To verify that our method does not memorize entity-specific patterns, we perform an overlap analysis on the Activation dataset; results are provided in appendix E.3
We evaluated our approach on four benchmark datasets from the DTIAM framework (Lu et al. 2025]. These datasets cover two types of prediction tasks: drug-target interaction (DTI) prediction using the Yamanishi. .08 and Hetionet datasets, and mechanism of action (MoA) prediction using the Activation and Inhibition datasets. 1) Activation dataset obtained from the Therapeutic Target Database (TTD) (Zhou et al.] 2022). containing 1,426 drugs, 281 targets, and 1,913 known activation interactions. 2) Yamanishi 00 originally introduced by (Yamanishi et al., 2008) consists of four sub-datasets: G-Protein Coupled Receptors, Ion Channels, Nuclear Receptors, and Enzymes. We use the combined dataset constructed by (Ye et al. 2021), containing 791 drugs, 989 targets, and 5,127 known DTIs. 3) Hetionet dataset constructed by (Himmelstein al. 2017). which integrated biomedical data from 29 public resources, comprising 1,384 drugs, 3,763 targets, and 49,942 DTIs. 4) Inhibition dataset derived from TTD (Zhou et al. [2022) containing 14,049 drugs, 1,088 targets, and 21,055 known inhibition interactions. For detailed dataset statistics, see Appendix Table
Pre-training Our mourl contrastive learning framework employs two-stage training pipeline designed for computational efficiency and scalability. In the first stage, we extract em- beddings using domain-specific pre-trained encoders: Mol.Former-XI. (Ross et al.) 2022) for SMILES sequences, MolT5 (Edwards et al., 2022) for textual descriptions and HTA annotations, and ESM2 (Lin et al.] (023) for protein sequences. In the second stage, we train lightweight pro- jection networks that map these modality-speciffi embeddings into a unified representation space, where volume-based contrastive alignment is performed using distributed training across multiple GPUs. The complete training procedure, including our novel gradient- gradient-informed adaptive modality dropout strategy, is detailed in Algorithms [ 2 in the Appendix.
Notably, we deliberately exclude Cvel from the gradient computation for modality dropping. Instead, Lui and Lacso assess modality importance for two key reasons. First, the bimodal con- trastive loss and IC50 loss provide stable, interpretable signals about each modality's 's contribution without creating computational circularity. Second, IC50 values, though sparsely available, of- fer biologically meaningful supervision that directly reflects protein-molecule interaction strength, making the gradients from Lıc50 particularly valuable for identifying which modalities are most important for drug- -tagett activity prediction. Comprehensive training configuration details are pro- vided in Appendix C In Table we present the network architecture along with the hyperparameter values used in our experiments. In Tables 6 7 g
we also provide the sensitivity and in Figure 4
analysis with respect to the hyperparameters.
We construct negative samples using an anchor point (Eq. 2 in each negative sample only a single modality is altered while the remaining modalities remain aligned. We hypothesize this is the most challenging negative sample scenario, since the model must distinguish the positive case, where all modalities are aligned, from a negative case in which all but one modality are aligned. To assess alternative strategies,
also evaluate an aggressive multi-domain negative sampling scheme in which negative samples are formed by varying multiple modalities simultaneously. The results are presented in Appendix section C.6

---

Published as a conference paper at ICLR 2026
Table 1: Mean performance comparison between GRAM-DTI and state-of-the-arr baselines DTT
and MoA prediction tasks across multiple datasets and data splitting scenarios. GRAM-DTI demon- strates superior performance in most evaluation settings. i indicates reproduced results; other results are from baseline papers. Bold denotes best performance.
Seraale
CPL-GNN
MPNN-CNN
TrandommetP
KGE-NFM
DITAM
DITAM
GRAM-DII
Worn stat:
0.451
0.516
0,502
0.817
0-61-00-825
0.563
0.62-4000455
0.642:00021
AUPR
Dry cod ster
0.167
0.408
0.400
0.341
0.30=0.0150
0.440=10662
0.550
0.61:+000222
0.628-00U1222
Tugget cold stat
0,390
0.502
0.616
0.761
0.84_000000
0.80910.0312
0.219
0.39|10.0020
Worr
0.821
0:952
0.953
0.948
0.967-0.0000
0.977 +07740
0.868
0
0.914-0
AURDCC
Deug stad
0,629
0.797
0.767
0.779
03188-0:0255
0,82810.8285
0,879
0.907110:0075
0.9131800065
Tuget cold stat
0.800
0,856
0.570
0.525
090110000180
0,955100.155
0.652
0.79210:00 80
0:441
0.789
0.879-0.0995
0.650100002
0.845:000070
AUPX
0.210
0,433
0.301
0 0:514-0.080
0.529:008626
0,830
0.751=0.0045
0.756 1736--0088 COSA
Trgn cold start
0.453
0:470
0.681
0625-000210
0.536---00200
0.215
0.465-0.0600
Warest::
0.981100001
LO0008
AUROC
0,685
0.752-006555
0JEE5+HURESS
0.948
0.922-2000028
0.940 10018
cold
0,810
0.858
0.955
0917=000000
0.921
0.605
08198000005
Downstream task In the DTI and MoA prediction task, the objective is to determine whether a given drug-target pair interacts, which constitutes a binary classification problem. Note that ex- isting datasets only include those pairs that interacts (positive class). Following standard practice (Lu et al.) 2025), we generated negative samples using a 1:10 ratio with positive samples for all datasets. To evaluate the model's generalization performance, we employed three different data splitting strategies for train-test division: I) warm start: The data is split based on protein-molecule pairs, ensuring that no common pairs appear in both the training and test sets. 2) drug cold start: This split is performed at the molecule level, guaranteeing that no drug in the test set present in the training set. 3) targer cold start: Similar to the above, but split at the protein level, meaning no protein in the test set seen during training. These three settings allow us to assess how well the model performs when faced with unseen molecule-protein pairs, unseen molecules, or unseen proteins, respectively. For evaluation, we followed the cross- cross-validttion protocols established in the original DTIAM framework (Lu et al. 2025):: 10-fold cross- volsdaani for DTI prediction tasks (Ya- manishi 08 and Hetionet datasets) and 5-fold cross- validaanliitiinnn for MoA prediction tasks (Activation and Inhibition datasets). Note that generated negative samples at a 1:10 ratio relative to positive samples across all datasets, to ensure consistency with baseline methods and a fair comparison. Ad- ditional results with varying negative-sample ratios are provided in Appendix our model performance changes as the ratio is adjusted.
E.4 illustrating how
4.2 EXPERIMENTAL RESULTS
We evaluated GRAM-DTI against ofate-of-the-art models across multiple benchmark datasets to demonstrate its effectiveness. For DTT prediction tasks, Table [1 presents a comparison with five baselines: CPL-GNN (Tsubaki et al. 2019). MPNN-CNN (Gilmer et al., 2017). TransformerCPI (Chen et al. 2020). and KGE-NFM (Ye et al. 2021) and DTIAM (Lu et al. (2025), on the Yaman- 0s and Hetionet datasets. For MoA prediction tasks, we compared GRAM-DTI against two baselines: AI-DTI (Lee et al.] (023) and DTIAM (Lu et al. 2025) on the Activation and Inhibition datasets. The different baseline sets reflect the distinct methodological approaches and evaluation standards established for DTTI and MoA prediction in the computational drug discovery community
and follows prior works (Lu et al. 2025 Panahandeh & Mansouri, 2025).
GRAM-DTI demonstrates strong performance across benchmark datasets, with particularly notable gains in target cold start scenarios. For DTI tasks, our method achieves substantial improvements on Yamanishi. .08 in both warm start and target/drug cold start settings. On the larger Hetionet dataset, GRAM-DTI outperforms most baselines across multiple evaluation scenarios. For MoA prediction, GRAM-DTI consistently surpasses baselines on the Activation dataset, especially under target cold start conditions. On the Inhibition dataset, while GRAM-DTI does not outperform existing baselines in warm start and drug cold start settings, exhibits excellent performance in target cold start.
Overall, GRAM-DTI outperforms sfate-of-the-ar baselines in nearly all evaluation settings 10 out of 12 for DTI and out of 12 for MoA tasks. Its strongest gains emerge on smaller datasets (Yaman- ishi_08 and Activation), where pre-training provides the greatest benefit under limited supervision, thus validating its potential for real-world drug discovery applications with limited available labeled data. On larger datasets (Hetionet and Inhibition). GRAM-DTI remains on par with outperforms strong baselines, particularly in cold start conditions. These results highlight the robustness and gen- eralizability of our multimodal alignment framework, especially when extending novel proteins.
7

---

Published as a conference paper at ICLR 2026
Table 2: sero-shot retrieval performance comparison between GRAM-DTI and DTIAM baseline across four datasets. Results show Recall @K K metrics for bidirectional retrieval tasks: S-PP and P-S. GRAM-DTI demonstrates superior retrieval capability across most scenarios and datasets using only pretrained representations. Bold denotes best performance.
Direction
Metric
Yamanishi 08
Hetionet
Activation
Labibition
DITAM
GRAM-DDII
UTTAM
GRAM-IITT
DITAM
GRAM-DTI
DTIAM
GRAM-DTI
800388000004
01043=000022
08028=0.0000
060136-1U0011
S
R:010
00341400042
00.091
01434=000051
0.1340::0.002
0.0357:-00011
01990+00181
0.4649 00.007
0.2066-0.0000
0.3616-0.0000
0.0000
0. 0.3184-0.0229
0.5688 R.0172
0.10 0104
0.1994-0.0018
0064466000002
0.0142-0.0120
03071
0U8370=-10066
8:810
008499600009
0.2066
0.1315:0.0095
0.1049:000005
03H636000000
0.2444
0,00286:00000
0003
0.0N19::00065
0.3670+000166
0.5540-0.0148
013622000044
00.841-
0.2206-00004
0.60299
0.0568+-0 0049
0.2325
4.3 ZERO-SHOT RETRIEVAL TASK
In addition to predicting drug-target interactions, an important aspect of evaluating our model's effectiveness is its ability to accurately retrieve relevant molecules or proteins based given query. The retrieval task assesses the model's capacity to learn meaningful, high-quality representations that preserve semantic relationships across different modalities. This task is particularly relevant for applications such drug repurposing and target identification (Luo et al. 2016] Pushpakom et al. (2019). where retrieving similar compounds or proteins can guide experimental validation and discovery.
To evaluate the retrieval capability of GRAM-DTI,, we conduct a series of experiments across the same four datasets. For each dataset, we formulate two retrieval scenarios: (i) retrieving proteins given a drug query (S))), and (ii) retrieving drugs given a protein query (P-+S). Using the learned representations directly from our pre-traine pre-training framework without any additional training, we com- pute similarity scores between query and candidate items. The performance is measured using standard metrics, including Recall @K (R @R@1, R@ 10, R@ 100), which indicate the proportion of relevant items retrieved within the top-K results.
The results, summarized in Table [ demonstrate that method outperforms DTIAM (the best baseline from DTI and MoA experiments in Table [ across nearly all datasets and metrics. No- tably, the superior performance in @@ and R@ 10 indicates that our model effectively captures the semantic relationships necessary for accurate retrieval, highlighting the quality of the learned mul- timodal representations. These strong zero-sho shot retrieval results provide compelling evidence that our multimodal pretraining framework successfully learns meaningful drug-target representations that generalize well beyond the specific downstream prediction tasks. More detailed experimental results can be found in Appendix E
4.4 ABLATION STUDY
Note that our main pre-training objective consists of three components (see 7 To evaluate the contribution of each component, we conducted a comprehensive ablation study, comparing the performance of our model with each component systematically removed.
We conduct five ablation experiments to evaluate the contribution of each component. Exp I uses the full objective with modality dropout applied on volume loss calculation, i.c., L = Liotal. which is the same as our GRAM-DTI setup. Exp 2 pre-trains without volume loss, using C = AaLsi + AsCicso- Exp 3 pre-trains without traditional pairwise contrastive loss, employing L = X L + A AiLicso- Exp pre-trains without IC50 supervision, using L = Coul + Xy Finally, Exp 5 uses the full objective but without modality dropout. The ablation study results on Activation dataset is presented in Figure 2 while the same for Yamanishi 08 dataset is reported in Appendix Figure 5 Across all setups, the full GRAM-DTI model (Exp 1) with all components enabled generally outperforms other variants where one component is removed. Impact of Gramian Volume-BB Volume-Based Alignment. Gramian volume-based alignment provides sub- stantial benefits across most evaluation scenarios. Comparing it (Exp 1) with the variant excluding volume loss (Exp 2) reveals consistent improvements across the majority of metrics, particularly in challenging scenarios like target cold start where models must generalize to previously unseen pro- teins. The volume- based approach effectively captures higher-order relationships among the four modalities that cannot be achieved through pairwise alignments alone, leading to more robust mul- timodal representations.
8

---

Published as a conference paper at ICLR 2026
AJROC
AJMEC
Seraiiit
ACTA
16-
L14
Dx1 2x0 5 534 Dx5
Ex tm Da3 5e4 by3
Do3 Do Em
Eee Ex!
De tx32 0633
Dx De0
Do Da 503
by4 593
00 099 tpp 024 003
64 042 991 954 by3
009 66 K6
6994 003 09.4
001 20 022 py1 D33
05 054 012
234 053
D61 202 963 254 D53
000 66
00 0992 202 054 600
D61 204 003 D54 055
Figure 2: Ablation study results on the Activation dataset across five experimental configurations and three data splitting scenarios. The full GRAM-DTI model (Exp 1) outperforms variants with removed components in most cases, demonstrating the synergistic contribution each training ob- jective component.
Impact IC50 Auxiliary Supervision and Contrastive Loss. Incorporating IC50 auxiliary su- pervision consistently improves performance across most evaluation scenarios (with the exception of Activation target cold start) as seen by comparing Exp 1 with Exp 4 (without IC50 supervision). Same conclusion holds when comparing Exp with Exp 3, which suggests that the bimodal trastive loss also ensures robust drug- bongppooein alignment and complements volume-based alignment. Together, these components capture both molecular activity principles and critical drug-protein re- lationships for effective prediction.
Impact of Adaptive Modality Dropout. Removing adaptive modality dropout (Exp 5), we see in figure 2 the performance consistently deteriorates, often by large margin, compared to the no-dropout setting. By dynamically regulating modality contributions during training, the adaptive dropout prevents dominant modalities from overwhelming complementary signals while ensuring all modalities remain engaged. This prevents overfitting to specific modality combinations, ultimately leading to more generalizable representations. To further validate this design choice, we compared our probabilistic dropout strategy against "soft" weighting alternatives (e.g., weighted- gra- dients). Results (see Appendix E.6) demonstrate that our "hard" dropout strategy provides stronger regularization effect and superior downstream performance.
Impact of Molecular Encoder Strength. To assess the modularity of our framework, we evalu- ated GRAM-DTI using more advanced molecular encoders, specifically Uni-Mol2 (Ji et al. (024) and BioT5+ (Pei et al. 2024), in place of MolFormer. As detailed in Appendix E.7 we observe that stronger encoders yield further performance gains, confirming that GRAM-DTI effectively leverages improvements in upstream foundation models.
To visualize how GRAM-DTI learns unified representations, we examine embedding evolution across training epochs using t-SNE on 3,000 randomly sampled quadruplets (Figure3) 3). Initially, the four modalities form distinet, separate clusters. As training pro- gresses, volume-based alignment gradually transforms rigid modality boundaries into semantically integrated representations while preserving modality-specific structures. By epoch 40, embeddings show substantial cross-modal integration where instances cluster by semantic relationships rather than purely by modality type. This evolution pattern provides visual evidence that our approach successfully balances cross-modal alignment with modality-speciff information retention, support- ing the quantitative improvements observed in downstream tasks. Additional analyses with varying sample sizes provided in Appendix F
Multimodal Embedding Evolution

---

Published as a conference paper at ICLR 2026
Insteal
Dosch 18
Eaoch
Ereck 38
teech
10023
Poacs
Figure 3: Evolution of multimodal embeddings during GRAM-DTI pre-training visualized using t-SNE on 3,000 samples. Four modalities (SMILES, Text, HTA, Protein) progressively align from separate clusters to semantically integrated representations, demonstrating effective volume-based multimodal alignment.
Impact of missing modalities during pre-training Bringing in as many relevant modalities in the pretraining would help learn better representation for the corresponding downstream task. However, what if during training, not all the modalities are available? We investigated this question by consid- ering scenario where certain modality is not available during pretraining. The results are presented in Appendix section E.2 Moreover, we can extend pretraining to include samples with missing modal- ities, which would substantially increase the size of our training set. To assess whether all modalities are beneficial, our current pretraining phase includes only samples in which all four modalities present, a choice that significantly limits the dataset. As a proof of concept, we evaluated whether including samples with only subset of modalities improves downstream performance. The results, presented in Appendix E..5 indicate a promising direction: incorporating partial partial-modee modality samples can expand the pretraining corpus and may enhance model performance.
4.5 FALSE NEGATIVE CASE ANALYSIS
To understand better when the model fails to predict drug target activity, we first systematically identified the top 10 "hardest" false negatives in the Activation dataset-pairs where the model predicted a strong negative signal despite a positive ground truth label. These are listed in table 22 in the appendix section E.8 From this list, we performed detailed case study on Rank: Drug D03XIS (R-568) targeting T92076 (CASR). Our analysis suggests this prediction difficulty likely stems from the unique and complex biology of this pair, which is statistically rare in typical drug- target datasets: This case may be challenging because CASR is Class C GPCR, fundamentally different from the Class A GPCRs that dominate drug databases. Three key factors may contribute to the prediction difficulty: (1) CASR has a large extracellular Venus flytrap domain, contrasting with the compact transmembrane binding pockets typical of Class A GPCRs; (2) it functions as an obligate homodimer with complex inter-protomet allosteric signaling; (3) R-568 acts as positive allosteric modulator rather than a traditional orthosteric agonist. The prediction difficulty may reflect the biological rarity of Class C GPCR allosteric modulators in drug discovery.
CONCLUSION
We presented GRAM-DTI, a multimodal pretraining framework that extends volume-based con- trastive learning to four modalities with gradient-informed adaptive modality dropout and IC50 auxiliary supervision. Evaluation across four benchmark datasets shows GRAM-DTI consistently outperforms baselines, particularly in cold start scenarios. Ablation studies (Appendix section 4.4) confirm synergistic contributions of each component. These results highlight the potential of mul- timodal pretraining for drug discovery, where integrating diverse data sources leads to more robust prediction models. Currently, the need to construct complete quadruplets (SMILES, Text, HTA, Protein) and remove overlapping (protein, SMILES) pairs with the downstream task has limited the scale of our pre-training dataset, restricting the diversity of molecules and proteins. To fully unlock the potential of GRAM-DTI and improve generalization to unseen molecular and protein targets, ex- panding the pre-training corpus will be crucial. In addition, incorporating rrolaen-related modalities beyond sequence information could further enhance performance.
ACKNOWLEDGEMENTS
This work was partially supported by a Johnson & Johnson grant for LLM based toxicity prediction, US National Science Foundation IIS-2412195, CCF-2400785. the Cancer Prevention and Research Institute of Texas (CPRIT) award (RP230363) and the National Institutes of Health (NIH) R01 award (IRO1 AT190103-01.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Saghir Alfasly, Jian Lu, Chen Xu, and Yuru Zou. Learnable irrelevant modality dropout for multi- modal action recognition on mppality-specific annotated videos. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 20208-20217, 2022.
Faisal Bin Ashraf, Sanjida Akter, Sumona Hoque Mumu, Muhammad Usama Islam, and Jasim Uddin. Bio-activity prediction of drug candidate compounds targeting sars-cov-2 using machine learning approaches. Plos one, 18(9):e0288053, 2023.
Rohit Bavi, Raj Kumar, Light Choi, and Keun Woo Lee. Exploration of novel inhibitors for bru- ton's ' tyrosine kinase by 3d qsar modeling and molecular dynamics simulation. PloS one, 11(1): c0147190, 2016.
Lifan Chen, Xiaogin Tan, Dingyan Wang, Feisheng Zhong. Xiaohong Liu, Tianbiao Yang, Xiaomin Luo, Kaixian Chen, Hualiang Jiang, and Mingyue Zheng. Transformercpi: improving compound- protein interaction prediction by sequence-based deep learning with self-attention mechanism and label reversal experiments. Bioinformatics, 36( 36(16):44064444 4414, 2020.
Giordano Cicchetti, Eleonora Grassucci, Luigi Sigillo, and Danilo Comminiello. Gramian multi- modal representation learning and aligmment. arXiv preprint arXiv:2412.11955 2024.
Carl Edwards, Tuan Lai, Kevin Ros, Garrett Honke, Kyunghyun Cho, and Heng Ji. Translation between molecules and natural language. arXiv preprint arXiv:2204.1 11817, 2022.
Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In International conference on machine learning, pp. 1263-1272. Pmlr, 2017.
Michael K Gilson, Tiqing Liu, Michael Baitaluk, George Nicola, Linda Hwang, and Jenny Chong. Bindingdb in 2015: public database for medicinal chemistry, computational chemistry and sys- tems pharmacology. Nucleic acids research, (4(D1))D104--D105. 2016.
Daniel S Himmelstein, Antoine Lizee, Christine Hessler, Leo Brueggeman, Sabrina Chen, Dex- ter Hadley, Ari Green, Pouya Khankhanian, and Sergio Baranzini. Systematic integration of biomedical knowledge prioritizes drugs for repurposing. eLife, 6:e26726, 2017.
Xiaohong Zhen Wang, Zhifeng Gao, Hang Zheng. Linfeng Zhang, Guolin Ke, et al. Uni-mol2: Exploring molecular pretraining model at scale. arXiv preprint arXiv:2406. 14969, 2024.
Feng Jiang, Yuzhi Guo, Hehuan Ma, Saiyang Na, Weizhi An, Bing Song. Yi Han, Jean Gao, Tao Wang, and Junzhou Huang. Alphaepi: Enhancing cell epitope prediction with alphafold 3. In Proceedings of the 15th ACM International Conference Bioinformatics, Computational Biol- ogy and Health Informatics, pp. 1-8, 2024a.
Feng Jiang, Yuzhi Guo, Hehuan Ma, Saiyang Na, Wenliang Zhong, Yi Han, Tao Wang, and Junzhou Huang. Gte: graph learning framework for prediction of t-cell receptors and epitopes binding specificity. Briefings in Bioinformatics, 25(4):bbae343, 2024b.
Feng Jiang, Mangal Prakash, Hehuan Ma, Jianyuan Deng, Yuzhi Guo, Amina Mollaysa, Tommaso Mansi, Rui Liao, and Junzhou Huang. Trident: Tri-modal molecular representation learning with taxonomic annotations and local correspondence. arXiv preprint S2kiv:2506.21022 2025.
Won- Yung Lee, Choong- Ycol Lee, and Chang-Eop Kim. Predicting activatory and inhibitory drug- target interactions based on structural compound representations and genetically perturbed tran- scriptomes. PLOS One, 18(4)e0222042, 2023.
Won-Yung
Qian Liao, Yu Zhang, Ying Chu, Yi Ding, Zhen Liu, Xianyi Zhao, Yizheng Wang, Jie Wan, Yi- jie Ding, Prayag Tiwari, et al. Application of artificial intelligence drug-target interactions prediction: review. upj biomedical innovations, 2(1):1, 2025.
Zeming Lin, Halil Akin, Roshan Rao. Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil Ori Kabeli, Yaniv Shmueli, et al. Evolutionary-scale prediction of atomic-level protein structure with language model. Science, 11196663)):12333111 1330, 2023.

---

Published as a conference paper at ICLR 2026
Sizhe Liu, Yuchen Liu, Haofeng Xu, Jun Xia, and Stan Li. Sp-dti: subpocket transformer for interaction prediction. Bioinformatics, 41(3):btaf011, 2025.
Zhiyuan Liu, Sihang Li, Yanchen Luo, Hao Fei, Yixin Cao, Kenji Kawaguchi, Xiang Wang, and modeling with cross-modal projector and uni- Tat-Seng Chua. Molca: Molecular 12211.12200 12798, 2023.
modal adapter. arXiv preprint arxiv:2310
Zhangli Lu, Guoqiang Song, Huimin Zhu, Chuqi Lei, Xinliang Sun, Kaili Wang, Libo Qin, Yafei Chen, Jing Tang, and Min Li. Dtiam: a unified framework for predicting drug-target interactions, binding affinities and drug mechanisms. Nature Communications, 16(1):2548, 2025.
Heng Luo, William Mattes, Donna Mendrick, and Huixiao Hong. Molecular docking for identi- fication of potential targets for drug repurposing. Current topics in medicinal chemistry, 16(30): 3636-3645, 2016.
Yizhen Luo, Kai Yang, Massimo Hong, Xing Yi Liu, and Zaiqing Nie. Molfm: A multimodal molecular foundation model. arXiv preprint arXiv:2307.094. 09.444444444 2023.
Hehuan Ma, Feng Jiang, Yu Rong, Yuzhi Guo, and Junzhou Huang. Robust self-training strategy for various molecular biology prediction tasks. In Proceedings of the 13th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics. pp. 1-5, 2022.
Hehuan Ma, Feng Jiang, Yu Rong, Yuzhi Guo, and Junzhou Huang. Toward robust self-training paradigm for molecular prediction tasks. Journal of Computational Biology, 31(3):213-228, 2024.
Nicholas Magal, Minh Tran, Riku Arakawa, and Suzanne Nie. Negative positive co-learning with aggressive modality dropout. arXiv preprint arXiv:2501 2,00006500.00065, 2025.
Amina Mollaysa, Artem Moskale, Pushpak Pati, Tommaso Mansi, Mangal Prakash, and Rui Liao. Biolangfusion: Multimodal fusion of dna, mrna, and protein language models. arXiv preprint (rXiv:25066089366 2025.
Tin Nguyen, Hien Le, Timothy P Quinn, Thin Nguyen, Trung Le, and Svetha Venkatesh. Graphdta: prediction of drug-targe binding affinity using graph convolutional networks. Bioinformatics, 37 (8):1140-1147, 2021.
Hakime Özüürk, Arzucan Özgür, and Elif Ozkirimli. Deepdta: deep drug-targe binding affinity prediction. Bioinformatics, 34(17):1821-i8)) 2018.
Fatemeh Panahandeh and Najme Mansouri. A comprehensive review of neural network-based ap-
proaches for drug-target interaction prediction. Molecular Diversity, pp. 1-48, 2025.
Qizhi Pei, Lijun Wu, Kaiyuan Gao, Xiaozhuan Liang, Yin Fang, Jinhua Zhu, Shufang Xie, Tao Qin, and Rui Yan. Biot5+: Towards generalized biological understanding with iupac integration and multi-task tuning. arXiv preprint arXiv:2402. 17810, 2024.
Lihong Peng. Xin Liu, Min Chen, Wen Liao, Jiale Mao, and Liqian Zhou. Mgndti; a drug-tare target in- teraction prediction framework based multimodal representation learning and the gating mech- 2024.
anism. Journal of Chemical Information and Modeling, 64 64(16):6684-66)
Sudeep Pushpakom, Francesco Iorio, Patrick A Eyers, K Jane Escott, Shirley Hopper, Andrew Wells, Andrew Doig, Tim Guilliams, Joanna Latimer, Christine McNamee, et al. Drug repurposing: progress, challenges and recommendations. Nature reviews Drug discovery, 18(1):41-58, 2019.
Abid Qureshi, Himani Tandon, and Manoj Kumar. Avp-ic50pred: multiple machine learning techniques-based prediction of peptide antiviral activity in terms of half maximal inhibitory con-
centration (ic50). Peptide Science, 104(6))753-763 2015.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. PmLR, 2021.
12

---

Published as a conference paper at ICLR 2026
Jerret Ross, Brian Belgodere, Vijil Chenthamarakshan, Inkit Padhi, Youssef Mroueh, and Payel Das. Large-scale chemical language representations capture molecular structure and properties. Nature Machine Intelligence, 4(12):
2022.
Wen Shi, Hong Yang, Linhai Xie, Xiao-Xia Yin, and Yanchun Zhang. A review of machine learning- based methods for predicting drug-target interactions. Health Information Science and Systems, 12(1):30, 2024.
Bonggun Shin, Sanghyun Park, Kyungsook Kang, and Joyce Ho. Self-attention based molecule representation for predicting drug-target interaction. Proceedings of Machine Learning Research, 106:955-970, 2019.
Hannes Stärk, Octavian-Euged Ganea, Lagnajit Pattanaik, Regina Barzilay, and Tommi Jaakkola. Equibind: Geometric deep learning for drug binding structure prediction. In International Con- ference on Machine Learning, pp. 20503-20521. PMLR, 2022.
Yao-Hung Hubert Tsai, Shaojie Bai, Makoto Yamada, Louis-Philippe Philippe Morency, and Ruslan Salakhutdinov. Multimodal transformer for unaligned multimodal language sequences. In Pro- ceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 6558- 6569, 2019.
Masashi Tsubaki, Kentaro Tomii, and Jun Sese. Compound-protern interaction prediction with end- to-end learning of neural networks for graphs and sequences. Bioinformatics, 35(2):309-3) 318, 2019.
Ali Vefghi, Zahed Rahmati, and Mohammad Akbari. Drug-targe interaction/affinity prediction: Deep learning models and advances review. Computers in Biology and Medicine, 196: 10438, 2025.
Feng Wan, Lin Hong, An Xiao, Tong Jiang, and Jianyang Zeng. Neodti: neural integration of neighbor information from a heterogencous network for discovering new drug-target interactions. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 1237-1245, 2019.
Xiaoqiong Xia, Chaoyu Zhu, Fan Zhong, and Lei Liu. Mdtips: a multimodal-data- drug- target interaction prediction system fusing knowledge, gene expression profile, and structural data. Bioinformatics, 39(7):btad411 2023.
Yoshihiro Yamanishi, Michihiro Araki, Alex Gutteridge, Wataru Honda, and Minoru Kanehisa. Pre- diction of target interaction networks from the integration of chemical and genomic spaces. Bioinformatics, 24(13) 24(13):1232-i240) 2008.
Qing Ye, Chang- Yu Hsich, Ziyi Yang, Yu Kang, Jiming Chen, Dongsheng Cao, Shibo He, and Tingjun Hou. A unified drug-target interaction prediction framework based on knowledge graph and recommendation system. Nature communications, 12(1):6775, 2021.
Amir Zadeh, Minghai Chen, Soujanya Poria, Erik Cambria, and Louis-Philiphy Morency. Tensor fusion network for multimodal sentiment analysis. In Proceedings of the 2017 Conference on Empirical Methods Natural Language Processing, pp. 1103-1114. 2017.
Siqin Zhang, Kuo Yang, Zhenhong Liu, Xinxing Lai, Zhen Yang, Jianyang Zeng, and Shao Li. Drugai: multi-view deep learning model for predicting drugetarget activating/nhhibitng mech- anisms. Briefings in bioinformatics, 24(1). 2023.
Yanpeng Zhao, Yuting Xing, Yixin Zhang, Yifei Wang, Mengxuan Wan, Duoyun Yi, Chengkun Wu, Shangze Li, Huiyan Xu, Hongyang Zhang, et al. Evidential deep bearning-based drug-target interaction prediction. Nature Communications, 16(1):6915, 2025.
Ying Zhou, Yintao Zhang, Xichen Lian, Fengcheng Li, Chaoxin Wang, Feng Zhu, Yunqing Qiu, and Yuzong Chen. Therapeutic target database update 2022: facilitating drug discovery with enriched comparative data of targeted agents. Nucleic acids research, D0(DI): 2022.
13

---

Published as a conference paper at ICLR 2026
A IC50 VALUES DISCRETIZATIONS
Given the inherent challenges of IC50 regression-incuueeee heterogeneous value distributions, wide dynamic ranges spanning several orders of magnitude, and noisy measurements-wwe for- mulate the problem as a three-class classification task. The IC50 values are discretized based on pharmaceutical relevance thresholds:
0 if IC50 < 10μM (effective)
IC50 class
1
if 10pM I IC50 1000μMM (moderate)
2 if IC5O 1000pMM (ineffective)
This discretization strategy aligns with established drug discovery practices (Qureshi et al. 2015 Bavi al. 2016, Ashraf al. 2023) where compounds with ICS0 10μM are considered highly active, those between 10 1000pM show moderate activity, and those > 1000μMM are typically considered inactive.
B DATASET
Pretraining Data Our pretraining dataset builds upon the high-quality multimodal molecular dataset from TRIDENT (Jiang et al. 2025). which provides comprehensive molecular represen- tations through the integration of SMILES strings, natural language descriptions, and Hicrarchical Taxonomic Annotations (HTA). The original TRIDENT dataset contains 47,269 carefully curated (SMILES, Text, HTA) triplets sourced from PubChem, where each molecule is annotated across 32 diverse taxonomic classification systems.
To enable protein-molecule interaction modeling, extended this dataset by incorporating binding affinity information from BBn a comprehensive database of measured binding affinities for protein-molecule interactions. We mapped molecules from the TRIDENT dataset to BindingDB entries using molecular identifiers, creating 5-tuples of the form (SMILES, Text, HTA, Protein, IC50). This integration combines the rich semantic and structural information from TRIDENT with quantitative binding affinity measurements, providing a unified multimodal representation that cap- tures both molecular properties and protein-molecne molecule interactions. Following standard practices in molecular property prediction, we implemented careful data filtering to prevent information leak- age between pretraining and downstream evaluation. Specifically, we removed all SMILES --rootennn binding pairs that appear in downstream task datasets to ensure fair evaluation and prevent over- fitting to specific polecular-proteen combinations seen during pretraining.
After filtering, 6,545 unique molecules have associated protein binding information. Considering that each molecule can interact with multiple proteins, this results total of 50,968 quadruplets (Protein, SMILES, Text, HTA), covering 4,418 unique proteins. Among these quadruplets, 16,035 entries include quantitative IC50 measurements, providing high-quality binding affinity annotations for modeling.
Downstream Task Datasets We evaluated our approach on four benchmark datasets (see Table 3 from the DTIAM framework (Lu al. 2025). covering drug-target interaction (DTI) prediction and mechanism of action (MoA) prediction tasks. 1) Activation dataset obtained from the Therapeutic Target Database (TTD) (Zhou al. 2022), containing 1,426 drugs, 281 targets, and 1,913 known activation interactions. 2) Yamanishi.0f originally introduced by (Yamanishi et al. 2008 and consists of four sub-datasets: G-Protein Coupled Receptors (GPCR), lon Channels (IC), Nuclear Receptors (NR), and Enzymes (E). We use the combined dataset constructed by (Ye et al.) (021), containing 791 drugs, 989 targets, and 5,127 known DTIs. 3) Hetionet dataset constructed by Hiimeeltten et al. 2017), which integrated biomedical data from 29 public resources, comprising 1,384 drugs, 5,763 targets, and 49,942 DTIS. 4) Inhibition dataset also derived from TTD (Zhou al. 2022). containing 14,049 drugs, 1,088 targets, and 21,055 known inhibition interactions.
The MoA refers to how a drug works on its target to produce the desired effects, which involve two major roles: activation and inhibition mechanisms. Distinguishing the activation and inhibition MoA between drugs and targets is critical and challenging in the drug discovery and development process, as well as their clinical applications Zhang al. 2023
14

---

Published as a conference paper at ICLR 2026
Table 3: Statisties of downstream task datasets for binary classification. Known Interactions repre-
sents the number of positive drug-target binding pairs, while Total Samples includes both positive samples and 10 times negative samples generated following standard practice.
Dataset
Task Type
Drugs
Targets
Known Interactions
Total Samples
Yamanishi 08
DTI
791
989
5,127
56,397
Hetioner
DTI
1,384
5,763
49,942
549,362
Activation
MoA
1,426
281
1,913
21,043
Inhibition
MoA
14,049
1,088
21,055
231,605
C PRE-TRAINING SETUP AND ARCHITECTURAL DETAILS
C.I PRE-TRAINING INFRASTRUCTURE
Our four-modal contrastive learning framework employs a two-stage training pipeline. First, we extract embeddings from domain-specific pre-trained models (MoLFormer-XL. (Ross et al. 2022) for SMILES, MoIT5 et al. 2022) for text/HTA, ESM2 (Lin et al. (003) for proteins). Second, train projection networks and the GRAM4Modal loss using distributed training across multiple GPUs. The complete training procedure is detailed in Algorithm which incorporates our gradient-based modality dropping strategy (Algorithm) 2
Notably, we deliberately exclude Loo from the gradient computation for modality dropping to avoid circular dependency, where the volume loss computation would depend gradients derived from that same computation. Instead, we use L Ap + >3 to assess modality importance for two key reasons: 1) Avoiding circular dependency: The bimodal contrastive loss and IC50 loss provide stable, interpretable signals about each modality's 's contribution without creating computational cir- cularity; 2) Leveraging weak supervision: IC50 values, though sparsely available, offer biologically meaningful supervision that directly reflects protein-molecule interaction strength. The gradients from Licso thus provide valuable information about which modalities are most important for pre- dicting drug-tate ttrget activity, making them suitable signals for adaptive modality selection. Table provides comprehensive training configuration details.
Algorithm Four-Modal Contrastive Learning with Gradient-bast Modality Dropping
Require: Pre-computed embeddings {-] 2 a x?}
Require: Drop probability Párop. temperature
Ensure: Projected features {f ft, fa, fr}
1: fm + Fm (E, (z" for m fs,t, h.
2: pm - 1f"
for all modalities
3: + Gradien(Basedrrop( (( ([m), L. Parop)
4: if d.should_ = False then
5: V7 + GRAM4Modal {fāi fan fih))
6: V, < GRAM4Modal (ff, (f f {"
7: else
8:
ma < d.anchor modality {m1, m23 remaining.modee
9:
{ma)
10: V, +- GRAM3Modal ( (f {Sm /m
11: V, < GRAM3Modal (fm {m f pm }
12: end if
13: Sj t -V,TT -V
14: Lval +- ( + Ltm)
15: return Liotal >1 Arhai + XgLicso
15

---

Published as a conference paper at ICLR 2026
C.2 MODEL ARCHITECTURE
The projection networks F map pre-computed embeddings to a unified 512-dimensional space. Each projection consists of three linear layers with GELU activations, layer normalization, and dropout (rate=0.1). The IC50 classification head F/ concatenates all four modality features flused = [ : s"; [e] and predicts binding affinity classes through a two-layer MLP with dropout (rate=0.3). The pre- pre-trained encoder specifications detailed in Table 5 All encoders Em are frozen during training to leverage their pre-trained representations while only fine-tuning the pro-
jection networks F% for computational efficiency.
Table Training Configuration Parameters
Parameter
Configuration
Hardware
Multi-GPU NVIDIA (CUDA)
Training framework
PyTorch DDP, NCCL
Batch size
1280 per GPU
Learning rate
10
(Adam)
Epochs
40
Temperature т
0.07
Drop probability Purop
0.8
Gradient history length K
Decay factor
0.9
Threshold multiplier A
1.5
Loss weights A₁ X2, A
1.0,
1.0, 1.0
Label smoothing
0.1
Algorithm Gradient-based Adaptive Modality Dropping
Require: Features ] mm..t... current loss Li. drop probability Párop
Require: Gradient history length K, decay factor o, threshold Xa 1.5
Ensure: Drop decision (should_drop. Mddrop- anchor_modality]
1: if' rrandom) > Pdrop not training then
2: return {False, none, protein}
3: end if
4: for m (s,t, h. p) do 0c, 5: 9f" 5..h
2 6: Update gradient history for modality
7: end for
8: for m
fs,t, t, h. p] do ΣK a" Ik-tat
9: 9:th
end for
11: x 1 0.th , + V Em (0 22))2
for m (8,1, h, p) do
13:
if gf > 14 og then
(1)
14:
break
15: end if
16: end for
17: if drop not found then
(0)
(f)
18: mrop arg minm F
19: end if
20: Manchor raanaannnnhaae waw.loom_hoiie(ss_11 h, \ (mmpp))
21: return True, m (
drop' M
16

---

Published as a conference paper at ICLR 2026
Table 5: Pre-trained Encoder Specifications
Modality
Model Em
Output Dim
SMILES (x*)
MoLFForer-XLL XL-both- 10pet
768
Text (x²)
MolT3-base
768
HTA (x")
MolTS-base (shared)
768
Protein (TP)
SSM2.133.650M_URE
1280
C.3 COMPUTATIONAL EFFICIENCY
Our method is highly efficient because we freeze the large encoder backbones (ESM2, MolFormer, MoITS) and only train the lightweight projection layers. This significantly reduces computational and memory overhead.
Hardware: All experiments were conducted on a single A6000 GPU.
Peak Memory: The peak GPU memory usage during pretraining is only 0.12 GB.
Batch Size: We use large batch size of 1280.
Pretraining Speed:Each pretraining epoch takes approximately 3 seconds to complete.
This demonstrates that our method is not memory-intensive and computationally very efficient.
C.4 HYPERPARAMETER TUNING AND SENSITIVITY ANALYSIS
We tuned the hyperparameters for the pretrained model and eventually set the final optimal values as: A, 1.5, Å = A2 As 1, K
presented in table 4 To further analysis the model performance sensitivity with respect to each parameters, we investigated the impact of the each parameters value the downstream task. In table 6 7 and X
we show the result the Activation dataset when we change the parameters value from the optimal values that are used in our final model. These results demonstrate that model's performance is stable within a reasonable range of these hyperparameters, with our chosen settings providing a robust and effective performance.
Table 6: Downstream-task performance on the Activation dataset: sensitivity to changes in hyper- parameter values from the optimal setting, evaluated under the Warm-start setup. The row bold indicates the optimal hyperparameter values used during pretraining and the corresponding down- stream performance.
A1
X
A
A gradient length K
AUPRC
AUROC
1
1
1.5
5
0.6424
0,9142
0.5
1
1.5
5
0.6449
0.9125
1
0.5
1
1.5
0.6237
0.9175
1
0.5
1.5
5
0.6326
0,9102
1
2
5
0.6340
0,9014
1
1.5
10
0.6155
0.9130
Table Downstream-taat performance on the Activation dataset: sensitivity to changes in hyper- parameter values from the optimal setting, evaluated under the Drug cold start setup.
Aa
A
gradient std multiplier
gradient history length
AUPRC
AUROC
1
1
1.5
5
0.6278
0.9125
0.5
1.5
0.6404
0,9129
I
0.5
1
1.5
5
0.6299
0,9107
1
0.5
1.5
5
0,6152
0,9114
1
1
5
0.6183
0,9009
1.5
0.6156
0.9103
17

---

Published as a conference paper at ICLR 2026
Table 8: Downstream-task performance the Activation dataset: sensitivity to changes in hyper- parameter values from the optimal setting, evaluated under the Target cold start setup.
A1
A2
gradient std multiplicr
gradient history length
AUPRC
AUROC
1
1
1.5
5
0,4497
0,8335
0.5
1.5
0,4394
0.8224
1
0.5
1
1.5
5
0.4274
0,8190
1
0.5
1.5
5
0,4571
0,8273
1
1
2
5
0,4286
0.8270
1
1.5
10
0.4343
0.8295
AUPRC
AUROC
AUPRC
AUROC
140
0.88
0.55
0.20
AUFRI
AUROC
0.00
0.82
2.40-
75
1.0
1.5
20
15
Gradient Std Muttipller
Gradlent Ste Multipller
Gradient History Length
Gradient Hiletor Length
3. Drup Oil Spit
Targut 5UR
INam Splt
Figure 4: Effect of gradient optimization parameters model performance. AUPRC and AUROC scores are shown for varying gradient standard deviation multiplier (left) and gradient history length (right) across three data split scenarios: drug cold split, protein (drug) cold split, and warm split. Error bars indicate standard deviation.
To further demonstrate the impact of the gradient_std gradiiet.sdd.mulipiiee and pondient.histone Length param- eters on model performance, we conducted a sensitivity analysis. We fixed all other parameters and varied either gradient std steeisstmmmlllll multiplier or gradient.history. length observe how performance changed. The sensitivity graphs are shown in Figure
For the "Activation" dataset:
gradient std multiplier:
Increasing this parameter until certain range had a stable effect on performance. Beyond this point, AUPRC for warm and drug cold starts began to drop, while protein cold start AUPRC remained stable. Simultaneously, AUROC for drug and warm starts decreased, while protein cold start AUROC increased.
gradient history length: Model performance was relatively stable with respect to its value increase across all evaluation setups.
C.5 VOLUME COMPUTATION DETAILS
The GRAM4Modal and GRAM3Modal functions compute volumes using Gram matrix determi- nants. For anchor features S and target features f", Js), the 4x4 Gram matrix G has entries Gkj (f* f The volume is computed as V =Vdoct(")) then converted to similarity via negative volume scaling: -V/r.
Algorithm implements our gradient-informed adaptive modality selection strategy, which main- tains consistency between forward C and reverse 2 contrastive computations by using a single drop decision per forward pass.
C.6 NEGATIVE SAMPLING STRATEGIES
We construct negative samples by fixing all but one modality, producing hard negatives in which only single modality is mismatched while the remaining modalities aligned. This single-
18

---

Published as a conference paper at ICLR 2026
modality perturbation yields more challenging learning signal, as the model must distinguish the fully aligned positive case from near-aligned negatives. To assess alternative strategies, we also evaluate aggressive multi-domain negative-sampling scheme in which multiple modalities perturbed simultaneously. Specifically, for each sample in batch B, we generate negative samples by permuting all modalities. Results on the Activation dataset comparing cross-negative sampling the current strategy under the volume-loss pretraining setting are reported in Table and Table I0
Table 9: Model performance comparison on activation dataset in terms of AUROC when using different negative sample strategies: Cross-Negative vs Current-Negative (Mean=Std))
Split Type
Cross-Negative
Current-Negative
Warm start
0.9142:10.0071 0,9164=0.0093 0.9125=#00068
0.9142:10.0078
Drug cold start
Target cold start
0.8388=0.0272
0.8335=000258
Table 10: Model performance comparison on activation dataset in terms of AUPR when using dif- (Mean+-Std)
ferent negative sample strategies: Cross-Negative vs
Split Type
Cross-Negative
Current-Negative
Warm start
0.6326=:.00322 0.6239::000245 0.4618++000313
0.6424=0.0221
Drug cold start
0.6278::0.0222
Target cold start
0.4497+:0.074
C.7 SENSITIVITY TO BATCH SIZE AND IN-BATCH NEGATIVES
Our method uses the standard in-batch negative formulation: for batch of size N, each sample uses the other banks. To test sensitivity, we varied the per-GPU batch size from 32 up to $12. The main paper results used batch size of 1280. As expected, performance generally improves with larger batch sizes, since more in-batch negatives benefit the contrastive and volume losses. The corresponding results are shown in table and
samples as negatives. We do not use stabilization techniques such as memory
Table 11: AUROC Performance vs. Batch Size (Activation Dataset)
Batch Size
Warm Start
Drug Cold Start
Target Cold Start
32
0.901
0,906
0.819
128
0.905
0.906
0.821
512
0.916
0.918
0.839
1280
0.914
0.913
0.834
Note that the 512 batch size results are very close to the 1280 results, suggesting performance may begin saturate beyond batch size 512.
C.8 DOWNSTREAM TASK ARCHITECTURE
For drug-target interaction (DTI) prediction evaluation, we employ a lightweight classification archi- tecture that leverages the pre-trained embeddings from our four-modal framework. The downstream architecture is detailed in Algorithm and uses only the drug (SMILES) and protein modalities relevant for binding prediction.
C.9 EVALUATION METRICS
We employ five standard binary classification metrics to comprehensively assess DTI prediction performance, Given the confusion matrix with true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN), the metrics defined as follows:
19

---

Published as a conference paper at ICLR 2026
Table 12: AUPRC Performance
Batch Size (Activation Dataset)
Batch Size
Warm Start
Drug Cold Start
Target Cold Start
32
0.615
0.607
0,432
128
0.625
0.619
0.438
512
0.642
0,629
0,458
1280
0.642
0.628
0.450
Algorithm 3 Target Interaction Prediction
Require: Pre-trained embeddings ff E 12112
Require: Drug-protein pair (x;,, binding label Mij E 1}
Ensure: Binding prediction Diy
1: fe FROZEN F (x;
))) {Use pre-trained SMILES embedding)
2: f/ FROZEN( (Ep (xx) {Use pre-trained protein embedding]
3: f pooncat [ f3] E I 102 (Concatenate embeddings)
4: h1 < (22 (fcmnal))
5: h1 Dropout, 3. (h1)
6: h2 k ReLU (ineartiiarss (h1))
7: hy Dropout,..(hz)
8: logits + Linear2se-12 (122)
9: Dij arg (ax(ssffmaa((looiis))
10: return Øi
Area Under ROC Curve (AUROC) AUROC measures the model's ability to discriminate be- tween positive and negative classes across all classification thresholds:
AUROC / TPR(FPR-((t) (t))
(9)
where TPR
TP+FN and FPR
TP
FP
FP+TN
Area Under Precision-Recall Curve (AUPRC) AUPRC is particularly informative for imbal- anced datasets and measures performance across different precision- recall trade-offs:
AUPRC L. Precision Recal (t)) dt
(10)
where Precision
TP FP+FP
and Recall TP TP+EN
Sensitivity (Recall) Sensitivity measures the proportion of actual positive cases correctly identi-
fied:
TP
Sensitivity = TP +
(11)
FN
F1-Score
F1-score provides the harmonic mean of precision and recall, balancing both measures:
2 Precision x Recall
T.
(12)
Precision + Recall 2 T-TP+FP+F
Accuracy
Accuracy measures the overall proportion of correct predictions:
TP + TN
Accuracy TP TN FN
(13)
D ABLATION STUDY
To complement the ablation study presented in Section 4.5 on the Activation dataset, we provide additional comprehensive ablation experiments the Yamanishi 08 dataset in Figure 5 This ad-
20

---

Published as a conference paper at ICLR 2026
ditional evaluation allows assess the generalizability of our component contributions across different datasets and task characteristics.
D.1 EXPERIMENTAL SETUP
The ablation study on Yamanishi follows the same experimental configuration as described in Section 4.5, evaluating five distinct setups:
Exp 1: Full GRAM-DTI model with all components and adaptive modality dropout
Exp Training without Gramian volume-based loss (L AgLan + A3
Exp 3:
Training without bimodal contrastive loss (L Av + A3Licso)
Exp 4: Training without IC50 auxiliary supervision (L = Lv + A2
Exp Training with full objective but without adaptive modality dropout
D.2 RESULTS ANALYSIS
The results on Yamanishi 08, shown in Figure demonstrate consistent patterns with those observed on the Activation dataset, confirming the robustness of our design choices across different datasets.
Consistent Superior Performance of Full Model: Across all three data splitting scenarios (warm start, drug cold start, target cold start) and five evaluation metrics (AUROC, AUPRC, Sensitivity, F1, Accuracy), full GRAM-DTI model (Exp 1) generally achieves the highest performance, demonstrating the synergistic benefit of all proposed components.
1. The volume-based multimodal alignment provides substantial benefits over traditional pair- wise approaches
2. Adaptive modality dropout prevents overfitting and improves generalization
3. IC50 auxiliary supervision enhances biological relevance of learned representations
4. The synergistic combination of all components yields optimal performance
These consistent findings across different datasets and evaluation scenarios validate the generaliz- ability of our GRAM-DTI framework design principles.
E ADDITIONAL EXPERIMENTAL DETAILS
Table 13: Performance metrics with standard deviations for GRAM-DTI across all evaluation datasets and data splitting scenarios. Results are reported mean + standard deviation across cross- validation folds.
Dataset
Split Type
AUROC1 1
AUPRCt
Sensitivity
F1 t
Accuracy 1
warm start
0.9771:00.0422
0.9036:10.0079
0.7954:10.0152
0.8353+0.0096
0.9715:20.0015
Yamanishi
drug cold start
0.8079
0..002+
0.2020-H0.0575
0.3090+0.0693
0,9193++00013
target cold start
0.9553=0.0155
0.8494:0.0312
0.7189:10.0433
0.7840:10.0885
0.9643=20.042
warm start drug cold start
0.9808=000011
0.8586110.0082
0.7580-10.008.
0.7891
0.9632-10.010
Hetionet
0.8550=20.0385
0.5291
0.2981 1:0.0645
0.4227-10.061
0.9273-10.0131
target cold start
0.9210=20.0079
0.6258-0.0239
0.4569-10.0448
0.5502+10.00
0.9325
warm start
0.9142::0.0078
0.6424+0.0221
0.5455-10.0240
0.5950-10 0075
0.9364-10.0026
Activation
drug cold start
0.9125=000068
0.6278=00.0222
0.5139
0.5879+0.0088
0.9347+0.0030
target cold start
0.8335=10.0258
0.4497.0.0374
0.245111.0050
0,3447+0.0620
0.9168-10.014
warm start drug cold start
0.9491110.0018
0.784910.000
0.6588 1009
0.7202+1.0066 0061
Inhibition
0.9298=20.0018
0.7555=00.0054
0.5949110.0176
0.6801 0081
0.9492-0.0011
target cold start
0.8234:10.018
0.4541
0.2584:10.0277
0.368710.0872
0.9220-0.0087
21

---

Published as a conference paper at ICLR 2026
AJRCC
AMPCC
Senaiitt
019
6/4
0.0
be Ex
Exp
Dve1 Em te
0o44
Del De1 te
0a44 byg
L 2e by
0o4 Exp3
Dm 5x2 Dx
Ex
091 062 66 994 kgg
09. 60 66 004 6gp
0904 294 by 604 6995
60 034 051 144 09
612
6/1
D51 PO4
254 Eg5
O51 282 68 004 b45
D51 D61 06 054 032
00. 581 Dy2 004 6355
60 032 053 D44
Figure 5: Ablation study results on the Yamanishi dataset across five experimental configurations and three data splitting scenarios. The full GRAM-DTI model (Exp 1) consistently outperforms variants with removed components across most metrics and scenarios, demonstrating the robust contribution of each training objective component. Results complement those shown in Figure (Activation dataset) and confirm the generalizability our design choices across different DTI prediction benchmarks.
E.1 STANDARD DEVIATION RESULTS FOR MAIN PERFORMANCE COMPARISON
provides comprehensive performance statistics for GRAM-DTI, including standard devi- ations across all evaluation metries, datasets, and data splitting scenarios. These detailed statistics demonstrate the stability and reliability of our approach across validdatio folds.
Table 13
E.2 PERFORMANCE ANALYSIS WHEN CERTAIN MODALITIES ARE MISSING AT PRETRAINING TIME
Bringing in as many relevant modalities in the pretraining would help learn better representation for the corresponding downstream task. However, what if during training, not all the modalities are available? We investigated this question by considering scenario where certain modality is not available during pretraining. For drrget interaction tasks, the drug and protein modalities are crucial. If either is unavailable during pretraining, the model cannot learn joint embeddings for in- teracting pairs. for one the missing modality, what one can do is use only the embedding obtained from original encoders (ESM2/molformer) without further fine tuning it with the contrastive learn- ing, this scenario will effectively falls back to regime similar to DTIAM, where drug and protein embeddings are learned separately. This highlights key strength of GRAM-DTI: pretraining with both modalities present, it learns aligned embeddings that improve downstream performance. For auxiliary modalities such functional descriptors and HTA:
If both are missing during pretraining, this is equivalent to the ablation study (Exp2), where the volume-loss component removed (as those two modality anticipate in the training through volume loss). In this case, we observe slight decrease in performance, indicating that these modalities provide useful signals for alignment.
If only one is missing, as shown Table and 15 the model still benefits from the avail- able modality, with moderate drop in performance. This demonstrates that GRAM-DTI can gracefully handle partial modality availability, but full modal pretraining yields the strongest embeddings.
22

---

Published as a conference paper at ICLR 2026
Table 14: Model performance the Activation dataset in terms of AU ROC for different modality configurations: 3-mod-no-text: all modality except functional descriptors are available during pre- training. 3-mod-no-HTA: all modality except HTA are available during pretraining 4-mod: all four modalities are available during pretraining (current setup).
Split Type
3-mod-no-kel no-text 3-mod -o-HTA
4-mod
Warm start
0.907
0.901
0.914
Drug cold start
0.907
0.903
0.913
Target cold start
0.828
0.821
0.834
Table 15: Model performance on the Activation dataset in terms of AU PRC for different modality configurations: 3-mod-no-text: all modality except funnctiona descriptors are available during pre- training, 3-mod-no-HTA: all modality except HTA are available during pretraining. 4-mod: all four modalities are available during pretraining (current setup).
Split Type
3-mod-no-tex, no-text 3-mod no- oo-HTA
4-mod
Warm start
0.609
0.606
0.642
Drug cold start
0.615
0.611
0.628
Target cold start
0.437
0,440
0.450
E.3 OVERLAP ENTITIES ANALYSIS BETWEEN PRETRAINING AND DOWNSTREAM TASK
To verify that our method does not memorize entity-specific patterns, we conducted an overlap analysis on the Activation dataset between pretraining and downstream task dataset (Other datasets are shown in Table[[7). The results revealed 236 overlapping proteins and 314 overlapping SMILES. We removed pairs containing these overlapping entities from the pretraining data, resulting in 6,065 exact (SMILES, protein) pairs removed (1 (11.9% of pretraining data). The results are shown in Despite removing nearly 12% of the pretraining data, the performance drops are modest
Table 16
Table 16: Performance before and after cleaning the Activation dataset.
Split Type
Metric
Before Cleaning
After Cleaning
A
AUROC
0.914
0.901
-0.013
Warm Start
AUPRC
0.642
0.613
-0.029
AUROC
0.913
0.905
-0.008
Drug Cold Start
AUPRC
0.628
0.624
-0.004
RUC
0.834
0.795
-0.039
Target Cold Start
AUPRC
0.450
0.389
-0.061
across all splits (0.01-0.03). This demonstrates that our model's strong performance is not primarily driven by memorizing exact pairs, and further validates our cold-start claims:
Entity overlap contributes to performance but is not the dominant factor
The 4-modal learning framework captures transferable molecular representations rather than memorizing specific entity combinations
Drug cold-start generalization is particularly robust (A AUROC -0.008), showing mini- mal sensitivity to entity overlap
E.4 MODEL PERFORMANCE SENSITIVITY TO THE NEGATIVE-SAMPAA RATIO IN THE DOWNSTREAM TASK
All baselines generate negative samples at a 1:10 ratio relative to positive samples across datasets; to ensure a fair comparison, we adopt the same setup. To evaluate sensitivity to this choice, we also report results using alternative negative-sample ratios. In table 18] we show model performance as well as best baseline (DTIAM) on the Activation dataset when negative samples are generated at various ratio with respect to positive samples.
23

---

Published as a conference paper at ICLR 2026
Table 17: Entity-level overlap analysis between pretraining and downstream datasets. "Pairs Sharing Any Entity" denotes the number of pretraining pairs containing at least one protein or SMILES that appears in the downstream dataset. Note that exact interacting (SMILES, Protein) pairs present in the downstream tasks have already been strictly removed from the pretraining set to prevent data
leakage.
Dataset
Total Pairs
Proteins
SMILES
Pairs Sharing Any Entity
Percentage
activation
50,968
236
314
6,065
11.90%
Hetionet
50,968
1,936
853
42,242
82.88%
inhibition
50,968
860
1,382
36,372
71.36%
yamanishi_Or
50,968
344
20,223
39.68%
Table 18: Performance Comparison of GRAM-DTI and Baseline on Activation Dataset under Dif-
ferent Ratios
Ratio
Split Type
Method
AUROC
AUPRC
Precision
Recall
GRAM-DTI
0,862
0.850
0.813
0.835
Warm Start
DTIAM
0.855
0.841
0.764
0.822
1:1
GRAM-DTI
0.853
0.843
0.822
0.818
Drug Cold Start
DTIAM
0.831
0.829
0.794
0.882
GRAM-DTI
0.790
0.784
0.788
0.691
Target Cold Start
DTTAM
0.740
0.737
0.705
0.656
GRAM-DTI
0.893
0.688
0.751
0.634
Warm Start
DTIAM
0.888
0.674
0.704
0.609
1:5
GRAM-DTI
0.888
0.674
0.742
0.616
Drug Cold Start
DTIAM
0.870
0.654
0.739
0.712
GRAM-DTI
0.818
0,559
0.680
0,412
Target Cold Start
DTIAM
0.792
0.509
0.566
0.365
GRAM-DTI
0.914
0.642
0.697
0.502
Warm Start
DTIAM
0.903
0.623
0.670
0,540
1:10
GRAM-DTI
0.913
0.628
0.692
0.491
Drug Cold Start
DTIAM
0.907
0.611
0.725
0.583
GRAM-DTI
0.834
0.450
0.638
0.260
Target Cold Start
DTTAM
0.792
0.391
0.533
0.293
E.5 HANDLING PARTIAL-MODALITY DATA DURING PRETRAINING
we can extend pretraining to include samples with missing modalities, which would substantially increase the size of our training set. To assess whether all modalities are beneficial, our current pretraining phase includes only samples which all four modalities are present, a choice that sig- nificantly limits the dataset. As proof of concept, evaluated whether including samples with only a subset modalities improves downstream performance. From the pretraining dataset we created:
Fully Observed (80%): 80% of the original data, kept unchanged.
Partially Observed (20%): the remaining 20% where we randomly dropped one modality.
We compared training on only the 80% fully-observed subset vs. training on the full 100% dataset (80% full 20% partial) using masked-volume loss for the partial samples. This simulates shows how pretraining can be expanded when some modalities are missing at random.
As the table 19 shows, incorporating the 20% partial data via masked-volume training improves performance across all metrics and splits, with a notable improvement on the Target Cold Start (AUROC 0.828 vs. 0.791).
24

---

Published as a conference paper at ICLR 2026
Table 19: Performance with Partial-Modality ("Masked-Volumee) Training
Split Type
Metric
Fully Observed Only (80% data)
Full + Partial (100% data)
Warm Start
AUROC
0.905
0,912
AUPRC
0.627
0.634
Drug Cold Start
AUROC
0,903
0,907
AUPRC
0.613
0.615
Target Cold Start
AUROC
0.791
0.828
AUPRC
0.422
0,437
E.6 ANALYSIS OF DROPOUT VS. WEIGHTING STRATEGIES
To validate our Pradient-Informer Modality Dropout strategy, we compared it against two alterna- tive "soft" balancing mechanisms on the Activation dataset:
Weighted-Modality Gradients: Instead of dropping a modality, we scale its gradient by the inverse of its norm with probability Pärop-
Standard Weighted Loss: We assign learnable weights to each modality's loss term to
balance contributions without dropout.
As shown in Table 20 our probabilistic dropout strategy achieves the best performance. We hypoth- esize that probabilisticaly removing modalities forces the model to find alternative distinct paths for reasoning in the joint embedding space, acting as a stronger regularizer than soft weighting.
Comparison of Modality Balancing Strategies on the Activation Dataset. Our hard dropout Table 20:
strategy outperforms soft weighting approaches.
Strategy
Split Type
AUROC
AUPRC
Warm Start
0.914
0.642
Gradient Informeedd Dropout (Ours)
Drug Cold Start
0.913
0.628
Target Cold Start
0.834
0,450
Warm Start
0.909
0.618
Weighted Gradients
Drug Cold Start
0.910
0.624
Protein Cold Start
0.828
0.445
Warm Start
0.901
0.621
Standard Weighted Loss
Drug Cold Start
0.892
0.619
Target Cold Start
0.814
0.440
E.7 EXPERIMENTS WITH STRONGER MOLECULAR ENCODERS
Our default GRAM-DTI implementation uses MolForner for computational efficiency. To demon- strate the framework's 's extensibility, we replaced MolFormer with two larger, more advanced en- coders: Uni-Mol2 (84M parameters) and BioT5+. As shown Table|2T] utilizing stronger encoders consistently improves performance across all splits, particularly in the challenging target Cold start scenario. It shows that stronger encoders yield consistently better results.
E.8 FALSE NEGATIVE CASE ANALYSIS
To understand where the model fails, systematically identified the top "hardest" false negatives in the Activation dataset- pairs where the model predicted a strong negative signal despite a positive ground truth label. These are listed in table 22 below:
E.9 MOA TASK ADDITIONAL BASELINES
To extend the set of baselines beyond those used in the DTIAM study for the MoA task, we included two additional methods: DeepI)TA (Oztürk et al. 2018) and GraphDTA (Nguyen al. 2021
25

---

Published as a conference paper at ICLR 2026
Table 21: Sensitivity analysis using advanced molecular encoders the Activation dataset
Encoder
Split Type
AUROC
AUPRC
Warm Start
0.9142
0.6424
MolFormer (Original)
Drug Cold Start
0.9125
0.6278
Target Cold Start
0.8335
0,4497
Warm Start
0.9280
0.6768
Uni-Mol2 (84M)
Drug Cold Start
0.9270
0.6658
Target Cold Start
0.8642
0.4848
Warn Start
0.9273
0.6828
BioT5+
Drug Cold Start
0.9254
0.6840
Target Cold Start
0.8577
0,4805
Table 22: Top- 10 False Negative Pairs (Drug ID & Target ID)
Rank
Drug
Target
DONYIR
T36075
2
D08FKH
T12475
DOG2VT
T59604
DOJB3H
T88505
5
DOLSWA
T28893
6
DOK8NR
T72458
7
D03LQC
T52522
8
D03XIS
T92076
9
D07QAK
T28893
10
DOAJ2T
T88505
Table 23: Performance comparison between GRAM-DTT and state-of-the-art baselines (DeepDTA, GraphDTA. AI-DTI, DTIAM) on MoA prediction tasks across multiple datasets and data splitting scenario. GRAM-DTI demonstrates superior performance in most evaluation settings. indicates reproduced results; other results are from baseline papers. Bold denotes best performance.
Data
Metric
Scenario
DecpDTA
GruphDTA
AI-DTI
DTIAM:
GRAM-DTI
Warm Start
0.246:00.0322 0.255:10.209
0.082-0.0240
0.583
0.623:10.0245
0.022
Activation
AUPR
Drug Cold Start
0.091
0.550
0.611
0.628
Cold Start
0.109++0.013
0.124=0.0175
0.219
0.391+00.0200
0.450 0.0374
Warm Start
0.759=0.0200
0.784=0.0185
0.888
0.903+0.0088
0.914
AUROC
Drug Cold Start
0.765=+0.009
0.796=0.0062
0.879
0.907
0.913 0.006
Cold Start
0.573+0.0241
0.088
0.652
0.792+0.0240
0.834 0.0258
Warm Start
0.542=0.0195
0.585=0.0280
0.840
0.845=10.007
0.785=
Inhibition
AUPR
Drug Cold Start
0.531=0.0170
0.592=0.0195
0.830
0.731
0.756=0.0034
Cold Start
0.265+000210
0.284=0.0312
0.215
0.445=+0.0200
0.464=0.0559
Warm Start
0.854+0.0105
0.872-0.0098
0.952
0.0025
0.949-0.0018
AUROC
Drug Cold Start
0.849:t0.0185
0.876=0.0115
0.948
0.921
0.940=0.0018
Cold Start
0.635::0.0220
0.649=0.0117
0.605
0.819:100205
Although DTIAM remains the strongest baseline overall, our model GRAM-DTT achieves superior performance in most evaluation settings. The results are shown in the table 23
E. SIGNIFICANCE TEST
We ran one-sided Welch t-tests to assess whether GRAM-DTI outperforms DTIAM (the strongest baseline). Tests were computed from summary statistics (means and standard deviations) using n=10 (10 folds) for Yamanishi 08 and Hetionet dataset and n=5 (5 folds) for Activation and Inhibition dataset with the one-sided hypothesis H1: GRAM-DTI DTIAM. Table 24 and Table 25 report
26

---

Published as a conference paper at ICLR 2026
the corresponding p values for the MoA and DTI tasks, respectively, and Tables 26 and show the zero-shot retrieval results. Cells highlighted in light blue indicate the better method in each applied 48 2 0.00104,
row. Note that we performed a total of 48 tests. To control for multiple comparisons
Bonferroni correction and used an adjusted significance threshold of: yadiussed 0.05
rather than the conventional p 0.05. p-values highlighted in light blue indicate p 0.00104.
Table 24: Significance test on the MoA task: means (+ std) and one-sided p-values for H1: GRAM DTIAM (independent Welch test). One-sided p- p-values with p < 0.00104 are highlighted light
blue.
Dataset
Metric
Scenario
DTIAM (μ4 σ)
GRAM (14 + 0)
one-sided p value
Warm start
0.623 + 0.0245
0.642 = 0.0221
0.1171
Activation
AUPR
Drug cold start
0.611 + 0.0252
0.628 = 0.0222
0.1455
Target cold start
0.391 + 0.0320
0.450 = 0.0374
0.0143
Warm start
0.903 + 0.0088
0.914 = 0.0078
0.0352
AUROC
Drug cold start
0.907 = 0.0076
0.913 = 0.0068
0.1126
Target cold start
0.792 + 0.0240
0.834 + 0.0258
0.0144
Warm start
0.845 + 0.0070
0.785 = 0.0061
1.0000
Inhibition
AUPR
Drug cold start
0.731 + 0.0045
0.756 = 0.0034
< 0.0001
Target cold start
0.445 + 0.0620
0.464 = 0.0559
0.3123
Warm start
0.954 + 0.0025
0.949 + 0.0018
0.9961
AUROC
Drug cold start
0.921 + 0.0028
0.940 = 0.0018
< 0.0001
Target cold start
0.819 + 0.0205
0.823 = 0.0028
0.3435
Table 25: Significance test on the DTA task: means (# std) and one-sided p-values for Hy: GRAM DTIAM (independent Welch test). One-sided p- p-values with < 0.00104 are highlighted light blue.
Dataset
Metric
Scenario
DTIAM (μI + σ)
GRAM (μ + σ)
one-sided value
Yamanishi 08
Warm start
0.901 + 0.0085
0.904 + 0.0079
0.2122
AUPR
Drug cold start
0.439 + 0.0580
0.440 + 0.0662
0,4859
Target cold start
0.844 + 0.0350
0.849 + 0.0312
0.3700
Warm start
0.967 + 0.0050
0.977 + 0.0042
< 0.0001
AUROC
Drug cold start
0.818 0.0255
0.828 + 0.0285
0.2096
Target cold start
0.941 + 0.0180
0.955 + 0.0155
0.0395
Warm start
0.879 + 0.0095
0.859 + 0.0082
0.9984
Hetionet
AUPR
Drug cold start
0.514 + 0.0680
0.529 + 0.0626
0.3070
Target cold start
0.625 + 0.0210
0.626 + 0.0239
0.4610
Warm start
0.957 = 0.0015
0.981 0.0011
0.0001
AUROC
Drug cold start
0.752 + 0.0355
0.855 + 0.0385
< 0.0001
Target cold start
0.917 0.0090
0.921 0.0079
0.1525
E.11 ZERO-SHOT RETRIEVAL TASK METHODOLOGY
This section provides detailed methodology for the zero-shot retrieval experiments presented in Section 4.3 of the main text.
E.11.
TASK FORMULATION
The zero-shot retrieval task evaluates GRAM-DTI's ability to identify relevant drug-target pairs us- ing only the learned multimodal representations, without any specific fine-tuning. We formulate two complementary retrieval scenarios:
Drug-to-Protedt Retrieval (S P): Given a query drug (SMILES representation), retrieve the most relevant target proteins from candidate set.
27

---

Published as a conference paper at ICLR 2026
Table 26: Significance test Zero-shot retrieval task (Yamanishi_0n) and Hetionet): one-sided Welch t-test (H11 GRAM > DTIAM), values < 0.00104 are highlighted.
Yamanishi_08
Hetionet
Metric
DTIAM
GRAM
DTIAM
GRAM
R@
0.0038 +: 0.0004
0.0465 + 0 0.0027
< 0.00001
0.0043 + 0.0002
0.0331 +: 0.0038
0.0001
S->P
R 10
0.0341
0.0042
0.1691 + 0.0084
0.0000
0.0434 0.0051
0.1340 0.0025
0.0001
R 100
0.1960 0.0181
0.4449 + 0.0075
0.0001
0.2066 0.0109
0.3616
0.0063
0.0001
R@
0.00-10 + 0.0002
0.0742 + 0. 0.0120
< 0.0001
0.0104
0.0028
0.0236 + 0.0010
1,00000
P-5
R 10
0.0849 0.0089
0.2465 + 0.0256
< 0.0001
0.1319
0.0095
0.1049 0.0055
1,0000
R 100
0.3670
0.5540 0.0148
0.0001
0.3632 0.0474
0.3841 0.0082
0.1900
Table 27: Significance test on Zero-shot retrieval task (Activation and Inhibition): One-sided Welch
t-test (H1: GRAM DTIAM), p < 0.00104 values are highlighted.
Activation
Inhibition
Direction
Metric
DIIAM
GRAM
DIIAM
GRAM
P
R@1
0.0028 0.0002
0.0136 + 0.0011
< 0.0001
0.0004 0.0000
0.0055 0.0003
0.0001
S-P
R 10 R@
0.0266 1: 0.0037
0.1020 + 0.0067
< <.00001
0.0097 + 0.0006
0.0337 + 0.0011
0.0001
0.3184 +: 0.0229
0.5688 + 0.0172
< <0.0001
0.1036
0.0104
0.1994 0.0018
0.0000
R@
0.0071 0.0008
0.0370 0.0069
0.0000
0.0000
0.0000
0.0221 0.0061
0.0001
P--S
R 10
0.0163 + 0.0050
0.24554 + 0.0142
< 0.0001
0.0028
0.0001
0.0819
0.0065
0.0001
R 100
0.2206 0.0266
0.6029 + 0.0231
< 0.0001
0.0588 0.0049
0.2325 0.0094 0.0001
Protein-to-Drug Retrieval (P~S): Given query protein (sequence representation), trieve the most relevant drugs from candidate set.
E.1 EXPERIMENTAL SETUP
For each dataset, we construct retrieval queries and candidate pools as follows:
Query and Candidate Construction:
For each known drug-target interaction (dz.p)) ) in the data set, we treat d, as query and all proteins in the dataset as candidates for S-P retrieval
Similarly, treat Pj query and all drugs candidates for P-+ retrieval
Ground truth relevance is determined by known interactions in the original datasets
Embedding Generation: We generate embeddings using the pre-trained GRAM-DTI framework:
SMILES sequences are encoded using MoLFormer-XX XL, producing 768-dimensional repre- sentations
Protein sequences are encoded using ESM-2, producing 1280 1180-dimensional representations
Both modalities are projected to a shared 512-dimensional space using trained projectors from the multimodal pre-training phase
All embeddings are L2-normalized for cosine similarity computation
Similarity Computation: We compute cosine similarity between query and candidate representa- tions using the projected embeddings:
sim g, c) fg fo [15 [ll
(14)
where fa and fo are the normalized projected embeddings for query q and candidate c, respectively.
Ranking and Evaluation:
For each query, we rank all candidates by their similarity scores in descending order
2. We evaluate retrieval performance using standard ranking metries:
Recall @ 1 (R @ 1): Proportion of queries where the top-ranked candidate is relevant
28

---

Published as a conference paper at ICLR 2026
Recall @ 10 @R@ 10): Proportion of queries where at least relevant item appears
in the top- results
Recall @ 100 (R@ 100): Proportion of queries where at least one relevant item appears in the top-100 results
E.11.3 RETRIEVAL. TASK I.LUSTRATION
Figure illustrates the zero-shot retrieval evaluation process. Given a query protein Pj, the model computes cosine similarities with all candidate drugs in the dataset and ranks them by similarity scores. Retrieval metries (R@ R@ 10, e@ 100) measure whether known positive drug-target in- teractions appear within the top-k ranked candidates.
sim (q,c)
R@1
R@10
Pj
dz-
R@100
Query
Candidates
Illustration of zero-shot retrieval evaluation. A query protein Pj is compared against all candidate drugs {di-ı,d; -... using cosine similarity of learned embeddings. Recall @k metrics evaluate whether any known positive interactions appear in the top-k retrieved candidates.
Figure 6:
E.1 IMPLEMENTATION DETAILS
Model Architecture: We utilize the same encoder architectures and projector networks in the main pre-training framework:
SMILES projector: 768 768 512 512 (with GELU, LayerNorm, Dropout)
Protein projector: 1280 768 + 512 512 (with GELU, LayerNorm, Dropout)
Batch Processing: Due to computational constraints, embeddings generated in batches of 16 sequences to manage memory usage while maintaining efficiency.
No Additional Training: Critically, no additional training or fine-tuning is performed for the trieval task. We use the representations learned during the multimodal pre-training phase directly, demonstrating the quality of the learned representations.
Evaluation Protocol:
Following standard practice in information retrieval, we compute metrics across all queries in each dataset and report average performance. The evaluation uses only positive interactions from the retrieval datasets, ensuring fair assessment of the model's ability to identify true drug-target relationships.
The strong performance of GRAM-DTI in this zero-shot setting (Table 2 in main text) demonstrates that our volume-based multimodal alignment successfully learns semantically meaningful represen- tations that capture drug-target relationships without task-specife supervision.
F COMPREHENSIVE MULTIMODAL EMBEDDING EVOLUTION ANALYSIS
This section provides a comprehensive analysis of how GRAM-DTI learns unified multimodal rep- resentations across different sample sizes and training epochs. We examine embedding evolution patterns to understand the dynamics of volume-bar multimodal alignment and validate the effec- tiveness of our adaptive modality dropout mechanism.
29

---

Published as a conference paper at ICLR 2026
lotte
Epech 30
Epech
Baoch 38
Tpack
Figure 7: Embedding evolution analysis with 500 randomly sampled guadruplets, showing clear progression from separate modality clusters to integrated semantic representations.
SCALL
Figure 8: Embedding evolution analysis with 3,000 samples, demonstrating consistent patterns with reduced noise and clearer semantic sub-structures.
Figure 9: Embedding evolution analysis with 5,000 samples (shown in main text), providing optimal balance detail and computational efficiency.
F.1 EXPERIMENTAL SETUP
We conducted embedding evolution analysis across multiple scales ensure robustness of our ob- servations:
Sample sizes: 500, 3,000, and 5,000 randomly selected quadruplets
Training epochs: Initial state (epoch 0), 10, 20, 30, and
Visualization method: t-SNE with perplexity=30, max itte==1000
Preprocessing: L2 normalization of projected embeddings, standardization per modality
For each epoch, extracted embeddings from the four modalities using their respective pre- trained encoders (MolFormer-XL for SMILES, MolT5 for Text/HTA, ESM-2 for Protein), applied the trained projection layers to map into the unified 512-dimensional space, and performed t-SNE visualization.
G LARGE LANGUAGE MODELS USAGE STATEMENT
We only used Large Language Models to correct grammars and polish the writing.
30


Reviews:
[{"review_id": "5ey14EAtMu", "reviewer": "Reviewer_Coqd", "summary": "The paper presents GRAM-DTI, a method for predicting drug-target interactions. The method uses pre-trained embeddings from foundation models to encode proteins, HTAs, molecules, and molecular functions. A Gram matrix-based contrastive loss is used to align modalities, and a gradient-based strategy for dropping modalities is used to promote equal contributions across modalities. Across various DTI datasets, GRAM-DTI outperforms relevant baselines in both AUROC and Recall@{1,10,100}.", "questions": "- How does performance degrade when certain modalities are not available at pretraining time?\n- How efficient is the method in wall-time compared to other methods for pretraining and training? The contrastive strategy also seems more memory-intensive, so details on what kind of hardware would also be nice.\n- How comparable are GRAM-DTI with the baselines in terms of multi-modality? I.e. is GRAM-DTI privileged in any way?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The method has overall strong performance across datasets. In particular, Recall@K is much improved over DTIAM across the board.\n- The overall method is quite simple, which is a benefit for fast adoption and ease of implementation.\n- Evaluation across warm and cold start conditions is useful and reveals the adaptability of the method.", "weaknesses": "- There are no sensitivity studies for the hyperparameters of the method (gradient history length, threshold multiplyer, loss weights).\n- The negative sampling used seems weak. Keeping all but one modality to mine negative samples does not test mismatches across multiple domains.\n- Failure modes are underexplored. Specific examples of drugs and targets that GRAM-DTI does not perform well on could be more insightful.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cbFLfkLfwH", "reviewer": "Reviewer_RpbF", "summary": "The paper proposes a four-modal pretraining framework for DTI that aligns SMILES, molecule text, hierarchical taxonomic annotations, and protein sequences into a shared space using a gramian volume–based contrastive loss. It adds a gradient-informed adaptive modality dropout to prevent any one modality from dominating, and incorporates IC50 labels when available as weak auxiliary supervision discretized into three activity classes. Encoders are frozen, lightweight projectors are trained. Downstream, the model is evaluated on Yamanishi’08, Hetionet, TTD-Activation, TTD-Inhibition with warm/drug-cold/target-cold splits. I,t also reports zero-shot retrieval and ablations showing contributions from each loss and the dropout mechanism", "questions": "- What fraction of drugs and proteins in each downstream fold appear in pretraining (even if the exact pair was removed)? How do results change when you exclude shared entities from evaluation?\n\n- How sensitive are AUROC/AUPR to the negative ratio and to similarity-aware decoys? Any analysis of false-negative contamination?\n\n- How does performance vary with batch size (negatives are in-batch), and do you use any tricks (e.g., memory banks) to stabilize volume estimation?\n\n- Did you attempt partial fine-tuning (e.g., last layer of ESM-2 or MolFormer) and if so, what was the compute/accuracy trade-off?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Using Gramian volume to enforce multi-way alignment is a clean way to go beyond pairwise CLIP-style losses. The objective and negative sampling scheme are clearly laid out\n- he gradient-informed dropout (drop dominant or least-contributing modality with probability p drop is simple, implementable, and empirically helpful in ablations.\n- IC50-based auxiliary labels provide biologically meaningful signal and choice of head is clear as it is expalined why classification was chosen over regression and how class imbalance is handled.\n- Results cover DTI and MoA datasets with warm/cold-start splits, plus a zero-shot retrieval task and qualitative t-SNE alignment progression\n- The five-way ablation (remove volume loss, remove bi-contrastive, remove IC50, remove dropout) makes it easy to see each piece’s contribution.", "weaknesses": "- You remove exact (SMILES, protein) pairs that overlap with downstream sets, but many entities may still be shared between pretraining and evaluation. That’s common and acceptable for self-supervised pretraining, yet quantified overlap and a performance breakdown with vs. without shared entities to de-risk confounds would be helpful, especially for the drug/target cold-start claims. Please report counts/percentages of shared drugs and proteins between pretrain and each downstream fold, and stratify results accordingly.\n\n- The 1:10 negative generation follows prior work, but many “negatives” in DTI are unknowns rather than true non-binders, inflating AUROC and masking calibration issues. Consider similarity-aware negatives (e.g., scaffold- or family-matched), decoy generation, or at least sensitivity curves vs negative ratio, plus precision@k on class-imbalanced settings.\n\n- You acknowledge scale is constrained by requiring full ⟨SMILES, Text, HTA, Protein⟩ tuples. Consider masked-volume training (optimize volume over the available modalities per sample) to exploit partial pairs and unlock larger corpora. Report how performance scales when adding partial-modality data", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WsCZwWUNSi", "reviewer": "Reviewer_U2Jr", "summary": "The study presents a new strategy for drug-target interaction prediction that integrates multimodality through a new pretraining strategy, that leverages volume-based contrastive learning and incorporates a regression target whenever a weak label is available. The method also incorporates a modality dropout strategy.", "questions": "1. What ESM2 model was used: 8M, 35M 150M, 650M, 3B, or 15B?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The volume-based contrastive learning loss for aligning multiple modalities seems like an elegant solution for the problem of multi-modal alignment in DTI tasks and more broadly in drug discovery tasks.\n2. The modality dropout seems like a scalable solution for the exploding number of potential modalities and encoders that are able for molecular tasks.\n3. The ablation study is comprehensive.", "weaknesses": "1. The improvement of the method over SOTA is unclear, the margins of improvement are always really low when compared to DTIAM < 0.02 in most cases. This is not a meaningful improvement. Furthermore, without error bars or confidence intervals is impossible to determine if this small improvements are even statistically significant or just an artefact of different random seeds.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OV75RlneYP", "reviewer": "Reviewer_5mrg", "summary": "This work introduces GRAM-DTI, a multimodal pretraining framework for Drug-Target Interaction. Firstly, GRAM-DTI combines multimodal modalities from molecule and protein into a unified representation using a novel volume-based contrastive learning to capture higher-order interdependencies when the modality number increases. Secondly, this paper proposes an adaptive modality dropout to dynamically filter out less informative modalities to prevent suboptimal representations. Thirdly, this work uses IC50 values as additional weak auxiliary supervision in the pretraining stage to make full use of the publicly available IC50 activity measurements. The GRAM-DTI achieves good performance on four datasets and several real-world application settings.", "questions": "1. Is the encoder for molecule $E_m$​ trainable or not? In the pretraining stage of Figure 1, it did not show whether the $E_m$​ ​is trainable or not.\n2. Why does this model use these relatively old encoders instead of modern ones? What's the reason for choosing these? Is there any ablation analysis to prove the selection?\n3. What are the four F in the pretraining stage in Figure 1? What model structure is used for these modules? \n4. For the datasets Inhibition and Activation, why only use AI-DTI and DTIAM baselines, and not stay consistent with the other two datasets?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The motivations are interesting and attractive. Modality dropout could be a novel way to reduce the noise during the multimodal fusion, and the usage of new volume-based contrastive learning could capture more modality information when including more modalities.\n2. The evaluation results show the performance improvement on four datasets, indicating the potential gain from the multimodal fusion. The ablation studies have been evaluated on most of the submodules, which is relatively comprehensive.", "weaknesses": "1. The choices of encoders are relatively out-of-date; for instance, there are stronger encoders for molecular modeling such as UniMol-v2, MLM-FG (2024), TranFoxMol (2023), MuMo (2025), ChemBERTa, HiGNN, 3D-MolT5, Mol-Llama, etc. A stronger encoder may bring better performance. The rationale of these selections is not specified. \n2. For the modality dropout, the top and least contributed modalities have been dropped to pursue a balance in multimodal learning and prevent overfitting. However, how to make sure that the important information has been preserved and more noise has been filtered out? For example, if a modality creates a larger gradient than others, it may contain more important information in this specific case. In this situation, forcing the model to learn more from irrelevant modalities can even bring in more noise. Also, the larger gradient may tell the model more information about this input data. If you want to prevent overfitting, adding a weight factor may be a better approach rather than removing the whole modality.\n3. The removal of modality losses (compared with weighted modality loss) in the modality drop approach may not be a meticulous design. It may cause the gradient to become unstable and harm the pretraining efficiency and even the performance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)