# Drug-Target Affinity Prediction Based on Cross-Modal Fusion of Text and Graph
## 2 Methods
### Cross-Modal Bidirectional Adaptive Guided Fusion Strategy
#### 2.2.2 Feature Refinement  
Although the feature extraction for each modality has been initially refined using a dual-layer GRU network for text features and an improved GIN network with a multi-perceptive neighborhood self-attention aggregation strategy for graph features, there remain limitations in capturing fine-grained dependencies within each modality. Specifically, the GRU network primarily focuses on sequence information but struggles to model long-range dependencies between words, limiting its ability to fully capture the global semantic relationships within the textual modality. Similarly, while the GIN network improves structural information extraction for the graph modality through its enhanced aggregation strategy, it's still constrained by predefined neighborhood scopes, making it difficult to dynamically adjust dependencies between nodes at the feature level.  
To address these limitations, this study incorporates an SA unit for further feature refinement within each modality before proceeding to cross-modal fusion. The SA unit leverages a multi-head attention mechanism to adaptively capture dependencies within each modality at the feature level, dynamically adjusting weight allocation based on feature importance. This process allows the model to focus on key features within a single modality, yielding more precise and semantically enriched representations. The introduction of the SA unit complements the GRU and the improved GIN by enhancing the modeling of intra-modal relationships, thereby providing more effective inputs for subsequent cross-modal fusion.  
During the feature fusion stage, the GA unit facilitates bidirectional feature guidance between the textual and graphical modalities. Using a multi-head attention mechanism, the GA unit computes interaction weights between the two modalities, enabling each modality to update its feature representation under the guidance of the other. Specifically, the GA unit allows the textual features in the drug branch to guide the focus areas of graphical features, and vice versa, ensuring that each modality can adaptively capture the most relevant aspects of the other. For instance, the textual features of drugs guide the weighting of molecular structure features generated by the graph modality, highlighting graph regions pertinent to the text. Similarly, graphical features emphasize critical words in the text modality, aiding the model in comprehending cross-modal information more effectively.  
Through a series of ablation experiments, it has been demonstrated that, compared to traditional weighted averaging fusion and the single guided-attention model shown in Figure 5, the dual guided-attention fusion module not only establishes bidirectional connections between different modalities, enabling more flexible and adaptive feature capture, but also allows for the design of cascading structures based on task requirements, achieving the layer-by-layer refinement of features. Through multi-level deep aggregation, this method fully explores the fine-grained interactions between modalities, significantly improving the model's ability to represent and predict the complex relationships between drugs and proteins.  
Figure 4: Cross-modal bidirectional adaptive guided fusion strategy.