# Drug-Target Affinity Prediction Based on Cross-Modal Fusion of Text and Graph
## 1 Introduction
Despite the significant progress achieved by DGraphDTA and subsequent models like SAM-DTA [11], particularly with SAM-DTA reducing the MSE to 0.229 on the same dataset, these models still predominantly relied on single-modality modeling. They failed to effectively integrate the advantages of textual and graphical modalities, limiting their potential in DTA prediction. Figure 1: A overview of representative methods for DTA prediction. In summary, previous deep learning-based methods for drug-target affinity prediction have relied either on textual sequences of drugs and proteins (such as the Simplified Molecular Input Line Entry System (SMILES [12]) for drugs and amino acid sequences for proteins) or on simple graph structures (where nodes represent atoms and amino acid residues, and edges represent chemical bonds and spatial proximities). However, these methods are limited in their ability to capture the intricate molecular interactions and complex structural dependencies that influence drug-target binding. Therefore, integrating various data types through cross-modal fusion methods to capture multidimensional information and provide a more comprehensive perspective will offer new possibilities for drug-target affinity prediction research. To address the above issues, this study proposes a deep learning model, CM-DTA, based on the cross-modal fusion of text and graphs for drug-target affinity prediction. The model consists of five key components: the drug and target representation module, the text modality processing module, the graph modality processing module, the feature fusion module, and the prediction module. First, in the drug and target representation module, the input drug and target data are preprocessed to generate four types of input data: the drug's SMILES expressions, molecular graphs (generated using the open-source software RDKit (2024.09.6) [13]), amino acid sequences of proteins (as the specific components of targets in this study), and protein graphs (predicted using the Pconsc4 program [14]). By introducing these four types of input data, this study addresses the issue of traditional methods focusing on a single modality and neglecting the interactions between multiple modalities. Secondly, two feature extraction modules are constructed, one for text data and one for graph data. The text modality is processed using a module based on Gated Recurrent Units (GRU), while the graph modality is handled using a Graph Isomorphism Network (GIN) [15] with a multi-perceptive neighborhood self-attention aggregation strategy. This allows for a deeper extraction of features from both the text and graph modalities, solving the issue of traditional methods' inability to adequately capture graph modality feature information. The features extracted by these two modules better support the subsequent fusion of drug and target features. In the feature fusion stage, a cross-modal bidirectional adaptive guided fusion module is designed. This module integrates the drug's text feature vector with the graph feature vector, as well as the target's text feature vector with the graph feature vector, combining them pairwise to generate two fused feature vectors, each weighted and guided by different modalities. This strategy addresses the problem of insufficient information transfer in cross-modal feature fusion, allowing the model to more accurately capture key information between modalities. Finally, in the prediction module, an explicit prediction strategy based on multi-head collaborative attention is applied. This module performs interactions between the two fused feature vectors of the drug and the target, concatenates them, and inputs the result into a multi-layer, fully connected neural network to complete the affinity prediction.