CheapNet: Cross-attention on Hierarchical representations for Efficient protein-ligand binding Affinity Prediction

Authors: Hyukjun Lim, Sun Kim, Sangseon Lee
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
Keywords: Protein-Ligand Binding Affinity, Hierarchical Representation Learning, Cross-Attention Mechanism, Drug Discovery

Abstract:
Accurately predicting protein-ligand binding affinity is a critical challenge in drug discovery, crucial for understanding drug efficacy. While existing models typically rely on atom-level interactions, they often fail to capture the complex, higher-order interactions, resulting in noise and computational inefficiency. Transitioning to modeling these interactions at the cluster level is challenging because it is difficult to determine which atoms form meaningful clusters that drive the protein-ligand interactions. To address this, we propose CheapNet, a novel interaction-based model that integrates atom-level representations with hierarchical cluster-level interactions through a cross-attention mechanism. By employing differentiable pooling of atom-level embeddings, CheapNet efficiently captures essential higher-order molecular representations crucial for accurate binding predictions. Extensive evaluations demonstrate that CheapNet not only achieves state-of-the-art performance across multiple binding affinity prediction tasks but also maintains prediction accuracy with reasonable computational efficiency. The code of CheapNet is available at https://github.com/hyukjunlim/CheapNet.

Decision: accept (poster)
Reviewer scores: 5, 8, 5, 6, 6

Reviews and discussion:
<thread>
creation date:
1730437230172

thread_type:
official_review

<thread_object>
creator: reviewer_npnb

<content>
summary:
Summary:  The paper introduces CheapNet, a novel model designed for protein-ligand binding affinity prediction. Focusing on efficiency, CheapNet employs a cross-attention mechanism on hierarchical representations to address limitations of traditional atom-level methods, which often capture noise by treating all atom interactions equally. By integrating differentiable pooling, CheapNet selectively forms clusters of atoms that are relevant to binding interactions, reducing computational complexity and improving accuracy. Experimental results demonstrate CheapNet’s competitive, state-of-the-art performance across multiple datasets.

strengths:
1.The paper is clearly written, the authors focus on a critical issue in drug discovery for which they proposed a solution and demonstrated by well designed experiments, that their proposed solution works well.
2.The main idea of using cluster-based, attention-guided binding predictions is well-motivated and aligned with current needs in efficient, scalable drug discovery models.
3.CheapNet achieves state-of-the-art performance across multiple datasets, showcasing its ability to balance accuracy and computational efficiency effectively.

weaknesses:
1.The application of clustering and cross-attention is not novel for this field, as clustering is used in models like GemNet, Equiformer, and LEFTNet. Although CheapNet integrates these methods, it does not introduce substantial methodological innovations.

2.The paper lacks a discussion of relevant clustering methods and does not provide sufficient analysis of different clustering approaches. This omission makes it difficult to assess the comparative advantages of CheapNet’s differentiable pooling mechanism.

3.The comparison lacks depth with state-of-the-art methods, including GemNet, Equiformer, and LEFTNet, all of which employ unique strategies for interaction prediction that CheapNet could be benchmarked against more thoroughly.

4.Equation 2’s category count (number of clusters) is unclear. It would be beneficial to specify whether it is a predefined constant or dynamically determined based on molecular size or complexity. This information is crucial for assessing how different molecular structures might impact CheapNet’s clustering performance.

5.The authors should review and discuss the relevance of clustering methods used by existing models like LEFTNet, which employs a layered approach to handle structural hierarchies. Additionally, the work should compare or at least mention methods from “Generalist Equivariant Transformer Towards 3D Molecular Interaction Learning” to position CheapNet’s approach among recent advancements.

questions:
See in weakness.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730529771749

thread_type:
official_review

<thread_object>
creator: reviewer_vcdv

<content>
summary:
The manuscript proposes a cross-attention method based on atom clustering for protein-ligand affinity prediction. This approach employs a soft-assignment method to separately cluster atoms in the protein and ligand, followed by a cluster-level attention mechanism to facilitate information exchange between the two. The model demonstrates significantly improved performance over baseline methods, and ablation studies indicate that integrating both clustering and cross-attention mechanisms into existing methods enhances prediction accuracy.

strengths:
The method is innovative. By clustering atoms separately within the protein and ligand, it effectively improves affinity prediction performance.

weaknesses:
1. The method’s details are not clearly explained. For example, how are the numbers of clusters for protein and ligand selected? Additionally, the process for initializing the representations of the protein and ligand is unclear.

2. While the method is interesting, the reasoning behind the clustering and cross-attention mechanisms' positive impact on model performance is not fully explained. Further analysis would provide valuable insights. For instance, although a soft clustering method is applied, this component is not explicitly supervised in the loss function. Why does such a soft clustering approach improve the model's performance?

questions:
1. How the numbers of clusters for protein and ligand selected?

2. Is there any experimental evidence explaining why the clustering method enhances model performance?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730570348625

thread_type:
official_review

<thread_object>
creator: reviewer_2uzu

<content>
summary:
The authors developed a novel interaction-based model (called CheapNet) that combines atom-level representations with hierarchical cluster-level interactions using a cross-attention mechanism to predict binding affinity tasks. The authors showed that CheapNet can effectively capture key higher-order molecular representations necessary for accurate binding predictions. They also performed extensive evaluations to show that CheapNet can deliver state-of-the-art performance in various binding affinity prediction tasks while maintaining efficiency in computation.

strengths:
-Used both local and global information to predict the binding affinity. The idea is relatively novel.
-Compared the performance of the proposed approach to that of many baseline approaches.
-Demonstrated the model interpretability.

weaknesses:
-The model performance representation can be further improved, such as using p-values to evaluate whether the proposed approach is significantly better than the baselines? the authors claimed "significantly outperforming all baselines", but there are no metrics to support the conclusion.
-It is not clear whether all the comparisons in the results tables are fair comparisons. For example, all these baselines are based on the same data evaluation strategy as the proposed approach (or the same set of training, validation and test sets) ? If the baseline results are from the original papers, how can we make sure the performance evaluations are fair?

questions:
I am curious whether the author can apply the trained models to predict real-world cases involving known disease-causing proteins. Can the models identify compounds from a large library, such as ZINC250K, that are likely to bind to these proteins? Since the authors have emphasized the model's strong interpretability, I would appreciate seeing how the model’s functions are used to interpret the prediction results in this case.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730646611484

thread_type:
official_review

<thread_object>
creator: reviewer_vfue

<content>
summary:
Predicting protein-ligand binding affinity is essential for drug discovery. Due to the complexity of protein-ligand interactions, traditional prediction models, which mainly rely on the atom-level interactions, are often computational intensive and unable to capture the complex and higher-order interactions. This paper proposes a deep learning-based model, CheapNet, for protein-ligand binding affinity prediction. CheapNet uses a cross-attention mechanism on hierarchical representations to capture intricate molecular interactions while maintaining computational efficiency.

strengths:
1. CheapNet integrates both atom-level and cluster-level representations of protein-ligand complexes, this can significantly enhance the model's ability to predict protein-ligand binding affinity. The idea is novel and meaningful。

2. CheapNet employs the DiffPool method to cluster atoms in both the protein and ligand, reducing complexity while retaining the critical protein-ligand interaction patterns. The use of the cross-attention mechanism between protein and ligand clusters highlights the most relevant inter-molecular interactions, filtering out less impactful interactions and reducing the computational costs.

3. The authors utilized the PDBbind and  CSAR NRC-HiQ datasets to benchmark CheapNet against different types of protein-ligand binding affinity prediction models, They evaluated the model performance on ligand binding affinity and ligand efficacy prediction using different performance metrics. Subsequently, ablation studies were conducted to evaluated the model's effectiveness, focusing on adaptability of cluster-attention, hierarchical representations and attention mechanism, and cluster size. The experiments are comprehensive, and demonstrate CheapNet's superior performance.

weaknesses:
1. CheapNet relies on high-quality three-dimensional structural data. However, many proteins lack experimentally crystallized structures, which limits CheapNet's ability to make predictions for proteins without available three-dimensional structural data.

2. In the section 'Permutation Invariance of Clusters for Cross Attention', the authors demonstrate that CheapNet’s cross-attention mechanism ensures permutation invariance for protein and ligand cluster-level representations. However, in protein-ligand interactions, three types of symmetries—translation, rotation, and permutation—should be considered. In my opinion, discussing whether and how the model achieves rotation and permutation invariance in local coordinates, as well as translation, rotation, and permutation equivariance in global coordinates, is essential. Only focusing on discussing the permutation invariance is insufficient.

questions:
1. Discuss how to deal with the proteins which do not have the experimentally crystallized structures. For instance, combine some AI protein prediction models, or use alternative representations for the proteins without three-dimensional structures

2. Extend the discussion on whether and how CheapNet handle the symmetries of protein-ligand complexes. If CheapNet is not able to address other types of symmetries,  then discuss howthis might impact the model's performance or generalizability, and the further improvement.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730689792365

thread_type:
official_review

<thread_object>
creator: reviewer_4khp

<content>
summary:
This paper proposes a new solution to the protein-ligand binding problem, namely modeling molecules and proteins at a higher level than the atom (cluster level). This motivation comes from the fact that modeling only at the atomic level can easily lead to computational burden and reduced accuracy. Experiments on the ligand affinity prediction and ligand efficacy prediction tasks demonstrate the effectiveness of proposed method.

strengths:
1. The motivation is reasonable, and it is possible to enhance the generalization of the model by trying to model at levels other than atoms.

2. The proposed method is simple and easy to understand.

3. Code is provided and is executable.

4. The experimental results are satisfactory in terms of both accuracy and computational efficiency.

weaknesses:
1. The writing logic of the article is not smooth, making it less readable. Two examples: (1) In the first paragraph in Introduction, a better presentation would be to first introduce the task, then talk about the wet lab approach and limitations, and finally analyze the challenges of deep learning models in solving this problem. Then, the purpose of the sentence describing DTI is also unclear and can be deleted. (2) Why does line 047 begin with "however"? Didn't you just talk about the limitations of atom-level modeling?

2. The motivation is reasonable, that is, the entire functional group may interact with a certain protein region. However, the pooling method used does not seem to guarantee this. Can the author consider, at least, adding additional loss to ensure that clusters represent the functional group?

3. The significance of hierarchical representation is usually to allow the model to adaptively learn and select features from different information channels. I also agree that some interaction cases come from the entire functional group rather than the atom, but this is not absolute. Therefore, I prefer dual awareness at the atom-level and cluster-level. Although cluster-level representations are derived from atom encoders, this only complies with the strong assumption that individual atoms do not participate in interactions. The framework I suggest is to use atom selectors (such as attention selection or gating algorithms) to filter important atom representations to merge with cluster representations.

4. Let's analyze the title. This paper's greatest contribution seems to be to propose a specialized adaptive attention mechanism for hierarchical representation learning. However, the proposed cross-attention algorithm seems to be only for the cluster level. In addition, if this is the case, what is the difference between the proposed method and directly adopting the cross attention module in [1]?

5. This is a point I would like to discuss with the author. In this paper, the author allows complex input, but only uses this information in atom embedding computation. Perhaps there is a more explicit way to use this information. Since we already know the rough range of where the interaction occurs, why not use it to guide the coefficient of cross attention. For example, assuming that cl:1 and cp:2 are clusters of ligand and protein at the interface, the correlation coefficient between cl:1 and cp:2 should be much higher than the others.


[1] Learning Harmonic Molecular Representations on Riemannian Manifold. ICLR, 2023.

questions:
1. How to determine the number of clusters for ligand and protein?

2. In fact, the proposed method still needs to learn effective atom-level representation to obtain pooling results and cluster representation. What is the advantage in computational efficiency?

3. The method in this paper does not seem to be limited to processing protein and ligand interactions, but can also handle protein-protein related tasks (please correct me if I am wrong). If the authors can perform additional experiments such as protein-protein interaction, protein-protein docking or protein-protein interface prediction, it will further prove the scope of the proposed method.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732162142818

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Summary of the revised manuscript

comment:
We sincerely thank the reviewers for their thoughtful feedback and constructive suggestions, which have significantly improved the quality and clarity of our manuscript. Below, we summarize the key revisions made in response to the reviewers' comments:  

1. **Clustering and Cross-Attention Mechanisms**  
   - Clarified the explanation of Table 5 in Section 4.3.2 to better illustrate the impact of hierarchical representations and cross-attention mechanisms on CheapNet’s performance.  
   - Analyzed methodological differences between CheapNet and existing cluster-level approaches, such as LEFTNet and GET, in Section 2.2, and compared their performance on LBA and LEP tasks (Tables 2 and 3).  

2. **Handling Symmetries in Protein-Ligand Interactions**  
   - Clarified in Section 3.4 that CheapNet’s permutation invariance refers specifically to cluster order, ensuring consistent outputs regardless of cluster ordering.  
   - Expanded the discussion to explain how CheapNet handles translation and rotation invariance at the atom embedding stage via GIGN, and discussed the potential benefits of integrating SE(3)-equivariant encoders (e.g., EGNN) for handling 3D symmetries.  

3. **Lack of 3D Structural Data and Noise Robustness**  
   - Discussed the challenge of limited availability of high-quality 3D structural data and the potential use of predicted structures, such as those from AlphaFold3 (see Appendix A.18 for details).  
   - Evaluated CheapNet’s robustness to coordinate noise through experiments, showing its ability to maintain strong performance even with noisy predicted structures (see Tables A13 and A14 in Appendix A.18).  

4. **Application to Virtual Screening**  
   - Discussed CheapNet’s ability to predict protein-ligand interactions through a virtual screening task using the DUD-E dataset (see Appendix A.15 for details).  
   - Conducted a case study on Tyrosine Protein Kinase SRC (SRC) to demonstrate CheapNet’s interpretability and accuracy in identifying critical interaction regions (Figure A2 in Appendix A.15).  

5. **Extending 3D Information and Exploring Dual-Awareness**  
   - Discussed the potential to incorporate 3D information into later stages, such as the cross-attention mechanism, by aggregating atom-level edges into cluster-level weights to guide attention scores (see Algorithm A2 in Appendix A.19).  
   - Explored the idea of a dual-awareness framework combining atom- and cluster-level representations with atom-level selectors like TopKPooling and ASAPooling. Preliminary analysis (Table A.15 in Appendix A.19) highlights this as a promising direction for future work.  

6. **Statistical Analysis**  
   - Conducted Z-tests to assess the statistical significance of CheapNet’s performance improvements compared to baseline models. Results are summarized in Tables A3, A4, and A5 in Appendices A.6 and A.7, confirming that CheapNet consistently outperforms baseline models across multiple metrics.  

7. **Clarifications and Revisions**  
   - Refined the **Introduction** to improve the writing logic, clearly presenting the motivation, problem statement, and contributions of the study.  
   - Conducted a thorough review of the manuscript to enhance clarity, conciseness, and overall readability, ensuring that all sections align cohesively with the study’s objectives and contributions.  

---

Please find the new experiments and key revisions highlighted in **blue** in the revised manuscript.  

We hope that the added experiments, along with our detailed point-to-point responses, have addressed the reviewers’ concerns. Should there be any additional questions or points requiring further clarification, we would be more than happy to address them.  

Thank you again for your valuable time and thoughtful feedback in reviewing our work.  

**Best regards,**  
_The authors_
</content>
</thread_object>

</thread>
<thread>
creation date:
1732630324284

thread_type:
public_comment

<thread_object>
creator: ~yang_zhang28

<content>
title:
Questions Regarding Results and Data Preprocessing

comment:
Dear Author(s),

I am deeply interested in the field of binding affinity prediction and have read your paper with great enthusiasm. Currently, I am attempting to understand and study your paper, but I have a few questions regarding the results and data preprocessing. I would greatly appreciate your insights on the following:

1. Regarding GCN Results: In Table 4 (Ablation Study) of your paper, I noticed that the RMSE results of GCN on PDBbind v2013, v2016, and the v2019 holdout set are reported as 1.419, 1.280, and 1.463, respectively. These results are significantly better than those mentioned in the GIGN [1] paper (1.749, 1.513, 1.763) and even surpass those of EGNN. To my understanding, for the same model, given consistent data and experimental configurations, the results are expected to be comparable. Could you please provide more information on whether any additional processing was applied when using GCN?

2. Regarding Ablation Study Results: In the rebuttal of "Response to Reviewer vcDV (Part 2/2)", you presented ablation study results indicating that CheapNet without cluster and cross-attention achieved RMSEs of 1.345, 1.189, and 1.360, respectively, which outperform recent SOTA method like GIGN (1.380, 1.190, 1.393). Since CheapNet without cluster and cross-attention seems relatively straightforward, could you please share any additional details on whether any additional modules or data features were introduced? 

3. Regarding Data Preprocessing: In the rebuttal, you provided details about the test dataset and mentioned that "This database was usually segmented into three overlapping subsets, namely the general set, the refined set, and the core 2016 set." There is an overlap between the general set (training dataset) and the core-set (test dataset).  Could you kindly elaborate on the data preprocessing process? 

Thank you for your time, and I apologize for any inconvenience caused by my questions.

Sincerely,

[1] Yang, Z., Zhong, W., Lv, Q., Dong, T., & Yu-Chian Chen, C. (2023). Geometric interaction graph neural network for predicting protein–ligand binding affinities from 3d structures (gign). The journal of physical chemistry letters, 14(8), 2020-2033.
</content>
</thread_object>

</thread>
<thread>
creation date:
1733224998264

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Summary of Discussion Period

comment:
We sincerely thank the reviewers for their constructive feedback, which has greatly improved our work. CheapNet introduces a novel cluster-attention mechanism that uses soft clustering of protein-ligand complexes, combined with cross-attention, to identify biologically meaningful interactions for binding affinity prediction. 

During the discussion period, the reviewers **recognized the novelty, motivation, and strong performance of CheapNet** and provided insightful feedback on its methodology, experimental evidence, and potential applications.

---
Below, we summarize how the primary concerns raised were addressed:
1. **Impact of Cluster-Attention Mechanism (Reviewer vcDV)**
   - **Concern**: Clarify why the clustering approach improves performance and provide experimental evidence.
   - **Response**: We clarified the reasoning behind clustering and cross-attention mechanisms through ablation studies (Table 5). **The Soft clustering dynamically identifies biologically meaningful clusters based on atom embeddings, while cross-attention refines key protein-ligand interactions**. To further investigate, we experimented with auxiliary losses (Appendix A.11), such as link prediction and entropy regularization. However, these losses tended to group atoms based on geometric proximity rather than embedding similarity, and therefore did not contribute to performance improvement.
2. **Discussion with Recent Cluster-Level Approaches (Reviewer NPNb)**
   - **Concern**: Compare how CheapNet aligns with recent cluster-level approaches, such as LEFTNet and GET, both methodologically and empirically.
   - **Response**: CheapNet’s novelty lies in its **soft clustering of atoms** combined with **cross-attention**, which dynamically captures meaningful interactions without being limited by geometric constraints. This flexibility allows CheapNet to group atoms based on embeddings, distinguishing it from methods relying on predefined geometric or domain-specific knowledge. As shown in the LBA 30% results (Table 2), CheapNet achieves **better performance**, demonstrating its effectiveness in the protein-binding affinity prediction. 
3. **Broader Applicability Across Interaction-Related Tasks (Reviewer 4kHp, 2uzU)**
   - **Concern**: Explore CheapNet’s applicability to more diverse tasks (e.g., virtual screening) or tasks beyond protein-ligand binding (e.g., protein-protein affinity).
   - **Response**: CheapNet demonstrated **versatility in virtual screening (DUD-E, Appendix A.15) and protein-protein affinity prediction (PPA Benchmark v2, Appendix A.20)**. In both tasks, CheapNet outperformed baselines, achieving higher AUROC and EF 0.5% in virtual screening and excelling in challenging cases like the PPA Flexible category. These results highlight CheapNet’s adaptability across interaction-related tasks.
4. **Fairness in Performance Comparisons (Reviewer 2uzU)**
   - **Concern**: Ensure that all baseline comparisons use consistent training, validation, and test splits.
   - **Response**: We carefully reviewed the baseline methods to confirm adherence to standard data splits and evaluation protocols. In **"Response to Performance Comparison Fairness (Part 1/4~4/4)"**, we provided detailed explanations of the splits used for each model, referencing their respective papers and source code. This information is summarized in Appendix A.5 of the revised manuscript, ensuring transparency and consistency across all comparisons. We are confident that these efforts demonstrate the fairness and reliability of the reported results.
5. **Symmetries in CheapNet (Reviewer VFuE)**
   - **Concern**: Address how CheapNet handles translation, rotation, and permutation symmetries.  
   - **Response**: We **clarified that CheapNet’s ability to handle translation, rotation, and permutation invariance** relies on the properties of its atom-level encoder. The cluster-attention mechanism itself operates on graph representations and does not enforce additional symmetries, ensuring modularity and flexibility in adapting to various GNN encoders. To enhance symmetry-awareness, we explored integrating (S)E(3)-equivariant encoders (e.g., EGNN), as outlined in Section 3.4 and Appendix A.3.


**Future Directions**

Building on the reviewers’ insightful comments, future work could explore **hybrid strategies** that integrate atom- and cluster-level embeddings to enhance CheapNet’s flexibility and performance. Additionally,  leveraging predicted 3D structures (e.g., AlphaFold3) could further expand CheapNet’s applicability in real-world scenarios where experimental 3D structures are unavailable.

---

We hope this summary demonstrates that the reviewers’ concerns have been thoroughly addressed, and that the manuscript is now more robust, clear, and complete. We are deeply grateful to the reviewers for their invaluable insights, which greatly improved the quality and impact of this work.
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper introduces a novel cross-attention mechanism for molecular data based on soft-clustering.

Amonth the strenghts of the method reviewers emphasized clear writing, simplicity of the method and consistent improvements across various benchmarks.

One of the key weaknesses of the paper is its limited novelty. Prior works such as GemNet and LEFTNet also utilize clustering in cross-atention. Authors propose a novel learnable clustering, which delivers consistent but modest improvements over the closest method (as shows during the rebuttal phase).

Three reviewers voted for acceptance and two voted to reject the paper.

During the rebuttal, reviewers raised concerns about the reliance on high-quality 3D structural data, the model's handling of rotational and translational symmetries, and comparison to other clustering-based approaches. Most importantly, the authors have conducted ablation studies demonstrating the performance advantages of CheapNet's soft clustering and cross-attention. Given the broad application of such soft clustering to biological and chemical data, it clears the bar for acceptance.

To summarize, the work is well executed though has limited novelty. Given the soundness of the work and the generality of the self-attention mechanism, the paper clears the bar for acceptance. It is my pleasure to recommend accepting the work.