reviews: 
Comments: The paper introduces GNNAS-Dock, a novel GNN-based algorithm selection system designed to enhance molecular docking in blind docking scenarios. Recognizing that no single docking algorithm consistently outperforms others across all scenarios, a statement generally supported by the no free lunch theorem. The authors propose leveraging GNNs to predict the performance of various docking algorithms based on the structural data of ligands and proteins. They aim to predict the performance (measured in RMSD) of each candidate docking algorithm to identify the most accurate method for specific scenarios. This effectively allows to choose the most computationally efficient docking algorithm for each docking case, reducing computational time while maintaining high accuracy. Approach is validated on the PDBBind 2020 refined set, comprising approximately 5300 protein-ligand complexes. The candidate algorithms include a mix of traditional and deep learning-based methods: DiffDock, DSDP, TankBind, GNINA, SMINA, Qvina-W, and their combinations with p2rank, totaling nine algorithms. Proposed algorithm selection model achieves a mean RMSD of approximately 1.74A, significantly improving upon the best performing standalone docking algorithm, DiffDock, which has a mean RMSD of 2.95A. When considering computational efficiency, the model demonstrates a success rate of 79.7% in achieving an RMSD below the 2A threshold, with an average processing time of about 29 seconds per instance.
Strengths: This work takes the first step in addressing a prominent challenge in computational drug discovery. The implementation of a GNN using standard ligand and protein features is effective and keeps the approach straightforward.
Moreover, the combination of an accuracy model with an efficiency model is beautifully done and in line with solving the performance-accuracy trade-off often faced in DD.
Weaknesses: 1. There's no need to emphasize the use of GNN or these standard features, as they are already well-established in this field. The development details of GNN-Lig and GNN-Prot might be more suitable for an appendix, allowing the main manuscript to focus on more critical analyses.
2. An ablation study could significantly enhance the work. For instance, exploring the impact of various features on docking algorithm selection would provide valuable insights into the interpretability and relevance of specific features across different approaches. Additionally, a deeper examination of the training datasets for machine learning models, as well as the testing systems used for physics-based methods, would strengthen the study's foundation.
3. Currently, the approach appears more like a baseline than a fully developed solution. This is understandable given the limited prior work in this area, as mentioned in the introduction and related work sections. However, enhancing GNN-Lig and GNN-Prot by incorporating richer features, such as learned sequence or structural representations (e.g., ESM or GearNet embeddings as node features), could throw further insights in how current approaches to represent ligands and proteins work for this problem.
4. Perhaps the most significant concern is the reliance on PDBBind for testing. While PDBBind is a valuable addition, it does not fully capture the complexities of docking scenarios. Many docking methods are trained on PDBBind, which includes systems traditionally well-suited for docking, thus limiting the ability to assess the model’s prospective potential. A carefully curated dataset with temporal splits would provide a more realistic evaluation of the model's performance across diverse scenarios.https://arxiv.org/abs/2308.09639 (as an example)
Recommendation: 3
Reviewer confidence: 4
Soundness/correctness: 3
Clarity: 3