UniSite: The First Cross-Structure Dataset and Learning Framework for End-to-End Ligand Binding Site Detection
Authors: Jigang Fan
QuanLin Wu
Shengjie Luo
Liwei Wang
Venue: NeurIPS 2025 spotlight ()
Keywords: Ligand Binding Site Detection
Ligand Binding Site Prediction
Drug Discovery
Structure Based Drug Discovery
Primary area: machine_learning_for_sciences

Abstract: The detection of ligand binding sites for proteins is a fundamental step in Structure-Based Drug Design. Despite notable advances in recent years, existing methods, datasets, and evaluation metrics are confronted with several key challenges: (1) current datasets and methods are centered on individual protein–ligand complexes and neglect that diverse binding sites may exist across multiple complexes of the same protein, introducing significant statistical bias; (2) ligand binding site detection is typically modeled as a discontinuous workflow, employing binary segmentation and subsequent clustering algorithms; (3) traditional evaluation metrics do not adequately reflect the actual performance of different binding site prediction methods. To address these issues, we first introduce UniSite-DS, the first UniProt (Unique Protein)-centric ligand binding site dataset, which contains 4.81 times more multi-site data and 2.08 times more overall data compared to the previously most widely used datasets. We then propose UniSite, the first end-to-end ligand binding site detection framework supervised by set prediction loss with bijective matching. In addition, we introduce Average Precision based on Intersection over Union (IoU) as a more accurate evaluation metric for ligand binding site prediction. Extensive experiments on UniSite-DS and several representative benchmark datasets demonstrate that IoU-based Average Precision provides a more accurate reflection of prediction quality, and that UniSite outperforms current state-of-the-art methods in ligand binding site detection. The dataset and codes will be made publicly available at https://github.com/quanlin-wu/unisite.

UniSite: The First Cross-Structure Dataset and Learning Framework for End-to-End Ligand Binding Site Detection
Jigang Fan
Quanlin Wu?.* Shengjie Luo²
Liwei Wang 1.2.3,1
Center for Data Science Peking University
State Key Laboratory of General Artificial Intelligence, Peking University
CCenter for Machine Learning Research, Peking University
luoss pku. edu. en, {quanlin, wanglw) 0pku. edu. cn
Abstract
The detection of ligand binding sites for proteins is fundamental step in Structure-
Based Drug Design. Despite notable advances in recent years, existing methods, datasets, and evaluation metries are confronted with several key challenges: (1) current datasets and methods are centered on individual protein-ligand complexes and neglect that diverse binding sites may exist across multiple complexes the same protein, introducing significant statistical bias; (2) ligand binding site detection typically modeled as discontinuous workflow, employing binary seg- mentation and subsequent clustering algorithms; (3) traditional evaluation metrics do not adequately reflect the actual performance of different binding site prediction methods. To address these issues, first introduce UniSite-DS, the first UniProt (Unique Protein)-centrr ligand binding site dataset, which contains 4.81 times more multi-site data and 2.08 times more overall data compared the previously most widely used datasets. We then propose UniSite, the first end-to-end ligand binding site detection framework supervised by prediction loss with bijective matching. In addition, we introduce Average Precision based on Intersection over Union (IoU) more accurate evaluation metric for ligand binding site predic- tion. Extensive experiments on UniSite-DS and several representative benchmark datasets demonstrate that IoU-based Average Precision provides a more accurate reflection of prediction quality, and that UniSite outperforms current state-of-the. art methods ligand binding site detection. The dataset and codes will be made publicly available at https: htpp:////thhu com/ quann wwelllinnnwwuune unisite
Introduction
The detection ligand binding sites on target proteins one of the most critical steps in modern drug discovery strategies I1 2 3. Structure-based drug design approaches begin with the three- dimensional structure of the target protein, from which deep, druggable cavities are identified. These regions, referred to binding sites binding pockets, composed of sets of protein residues. Once the protein's 's sites are recognized, virtual screening a molecular library can be performed using potein-ligane docking and podtiinlligand affinity prediction 4. 5]. Alternatively, can be conducted based on the local structure of the binding sites to identify potential candidate compounds. As fundamental step, the accurate identification of protein binding sites can significantly facilitate and influence subsequent steps in drug discovery.
methods such de novo molecular design [ 7
'Equal contribution. Conresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Over the past several decades, some endeavours have been made to detect protein ligand binding sites. These methods have evolved from traditional techniques based on geometry [8], template searching [2], and energy probes [0] to machine learning methods based on surface features [[]] and further to deep learning methods utilizing convolutional neural networks (CNNs) 12 and graph neural networks (GNNs) [13 14 15]. Concurrently, a series of wrotein-ligand datasets have also
been established progressively, including scPDB [16] and PDBbind [17] datasets for protein-ligand ligand complex structures, as well as benchmark datasets such as HOL04K [88] and COACH420 [[9] for evaluating binding site detection methods.
Although the above efforts have significantly advanced the field of ligand binding site detection, current methods, datasets, and evaluation metries are confronted with substantial challenges:
Issue 1. All previous methods and datasets are PDB (Protein Data Bank file)-centric, specifically focusing on individual protein-ligand structures, which introduces considerable statistical bias. Due to experimental constraints, only a limited number of binding sites in the protein are typically observed in one single protein-ligand structure where ligands are bound. However, one protein can be associated with numerous distinct protein-ligand structures, which exhibit high structural similarity in their protein components yet considerable variation in their binding site regions 20 22 (Figure T. But existing datasets and methods only regard these structures as individual data entries, focusing on limited binding sites in single PDB structure. Training and evaluating on PDB-centric datasets introduces significant statistical bias, as the annotation paradigm of individual PDB structures overlooks many other ground truth binding sites.
Issue 2. Existing methods employ discontinuous workflows for binding site detection. Most approaches 88
I1 [3 [4] first perform semantic segmentation generate binary masks of potential binding residues/atoms,, then cluster them into discrete binding sites. Alternative implementations only predict binding site centers (15), and the associated residues need to be extracted using external methods. These fragmented pipelines highly rely on the post-processing methods (e.g. clustering algorithms), inherently limit end-to-end optimization and struggle with overlapping binding sites.
Issue 3. Traditional evaluation metrics inadequately reflect the actual performance of binding site detection. The most widely used evaluation metries are DCC and DCA (.]) DCC represents the distance hetween the predicted binding site center and the ground truth binding site center. DCA denotes the shortest distance between the predicted binding site center and any heavy atom of the ligand. These metrics suffer from two fundamental limitations (Figure 4) (1) they completely disregard the structural properties such as shape, size, and residue composition of binding sites, which are crucial for downstream tasks (Appendix[A) (A), and (2) the absence of proper matching criteria between predictions and ground truth may lead to double-counting of predictions.
To address the issues mentioned above, this paper makes the following contributions:
1) We introduce UniSite-DS, a manually curated, UniProt (Unique Protein)-centric dataset of protein ligand binding sites. Leveraging the unique identifiers assigned protein sequences in UniProt 23 we systematically integrated all ligand binding sites associated with given unique protein across multiple PDB structures. To the best of our knowledge, it the first UniProt-centric dataset. Notably, Un includes 4.81 times more multi-site proteins than existing datasets [16] 177, and the overall size of the dataset 2.08 times larger. The Uniprot-centric dataset corrects the statistical bias of previous PDB-centric datasets, thereby resolving Issue I and significantly broadening the available data.
2) We propose UniSite- 1D and UniSite-3D, two end-to-end methods for protein ligand binding site detection. Both models utilize a transformer encoder-decoder architecture, supervised by a set prediction loss with bijective matching. UniSite ID/3D directly predict N potentially overlapping binding sites without requiring post-processing clustering steps, thus completely resolves Issue 2. The UniSite- ID variant operates exclusively on 1D protein sequence inputs, providing structure- free binding site detection capability. For enhanced performance, the UniSite-3D variant incorporates 3D structural information while maintaining the same end-to-end prediction framework.
3) To overcome the limitations inherent in traditional evaluation methods outlined in Issue 3, we introduce an Average Precision (AP) metric based on Intersection over Union (IoU) for fair and comprehensive binding site assessment. Extensive experiments have demonstrated that the loU-d AP maintains strong concordance with method rankings under traditional metrics while overcoming their key limitations, providing a more accurate reflection of prediction quality.
2

---

4) Extensive experiments on UniSite-DS and classical datasets have demonstrated that our methods outperform the current state-of- oftte-of- methods in protein ligand binding site detection. These results indicate that the end-to-end detection framework, which operates without the need for specialized feature engineering, is already capable of exhibiting strong performance for binding site detection.
PDBbind2020
UniSite-DS UniProl-centric Dalasel
Structuee Painwise 88 Painwlae
PDB-centric Dataset
Binding
1YHM Binding
AHD Binding Site
Number of Mu--t--Stee Entries
Number of Unique Proteins
11510
Number of Unique Proteins
2487
5540
3000
762
1250
351
265
Posbind
HOLOAK
COACH420
UniSite-DS
Posbind
HOLOsK
COACH420
UniSRta-S 05
3.0
per Unique Protein
Figure 1: Comparison between UniSite-DS and previous datasets. (Top left) In PDBbind2020, only one ligand binding site and one structure are recorded for UniProt ID Q8WS26. (Top right) In contrast, UniSite-DS integrates distinct binding sites across all available structures (highly similar, mean TMMScore=0.99)), identifying 17 unique ligand binding sites derived from 13 representative PDB entries. (Bottom left and center) Comparison of UniSite-DS with other widely used datasets in terms of multi-site entries and the number of unique proteins. For HOL04K and COACH420, the most widely used mlig subsets were selected, where each entry corresponds in UniSite-DDS DS, each entry corresponds to a UniProt ID.. (Bottom right) Distribution of the number of unique proteins UniSite-DD with respect to the number of distinct binding sites they contain.
PDB structure, while
2 UniSite-DS: The First Uniprot-centric Dataset
A key challenge in detecting protein binding sites is how identify all potential binding sites 20 21 Most proteins contain an inherently conserved binding site, commonly referred to as the active site. The active site is shared among members of the same protein family, which means that molecules targeting this site will simultaneously target all other proteins within the family, which is highly likely lead off-target effects and side effects (24). Identifying other binding sites within the protein that can be targeted is crucial strategy. These sites are often located in regions topologically distant from the active site and modulate the protein's function through allosteric effects
25 26 27
As illustrated in Figure [ one single protein can correspond to a large number different ligand- bound structures. While the overall protein structure tends to be highly conserved, the ligand binding site regions vary considerably across these structures. The motivation behind constructing UniSite-DS lies in the recognition that identifying all potential binding sites of protein requires comprehensive examination of all its ligand-bound structures- an important consideration that has been overlooked by previous methods and datasets.
To construct UniSite-DS, we performed the following search and processing steps: (1) We utilized AHoJ [28] to systematically search for all protein-ligand interactions in the PDB database [29]: (2) To ensure dataset quality, we excluded entries with resolution greater than 2.5 or those determined by non-crystallographic methods; (3) Following P2Rank's filtering criteria [II) we removed solvent molecules and ligands composed of fewer than five atoms, resulting in total

---

of 143,197 protein-ligand interaction entries; (4) For each interaction, binding site residues were identified within 4.5A radius of the ligand: (5) We discarded entries with three or fewer binding site residues eliminate "floating" ligands; (6) Leveraging UniProt's unique protein sequence identifiers 23 we mapped binding site residues from all protein-ligand interactions of each UniProt entry to their corresponding sequences via SIFTS annotations [30, integrating all ligand binding sites across different PDB structures; (7) To eliminate data redundancy among ligand binding sites, we applied Mox-Mmximum Suppression (NMS) with an Intersection over Minimum (IoM) threshold of 0.7 and an Intersection over Union (loU) threshold of 0.5, excluding highly overlapping sites. This process resulted 13,464 distinct UniProt IDs, of which 4,846 contained multiple ligand-binding sites; (8) Based on the criteria from Proteina 3. we set the sequence length threshold to 800; (9) We mamually inspected all UniProt IDs with more than ten ligand binding sites, as well as those where single pootein-ligande complex structure contributed three more binding sites. As result, we identified 11,510 10 valid UniProt IDs, including 3,670 with multiple ligand binding sites. The distribution of ligand binding sites is shown in Figure T More details about the UniSite-DS curation workflow and manual inspection process are provided in Appendix B
As the first UniProt-cennn centric dataset, UniSite-Di DS encompasses 4.81 times more multi-site entries than previous datasets, and covers 2.96 times more UniProt entries than the widely used PDBbind dataset [17], as well as 2.08 times more than sc-PDB [16] (Figure [T). UniSite-DSS eliminates the statistical biases inherent in earlier datasets and significantly expands the available data on multi-site ligand binding sites. Notably, case studies conducted using UniSite-IDD DS (Appendix E) highlighted the limitations of current binding site prediction methods in handling multi-site proteins. This observation motivated us to develop novel end-to-end method for protein-ligand binding site detection.
3 The Proposed Methodology
In this paper, we formulate protein ligand bind- ing site detection as a set prediction task: given a protein P with an amino acid sequence S of length L, the goal of binding site detection is to identify a set of binding sites {m² Nge Noe 1==1
Discontinuous Workflow 1) Predict Binary Mask (Others)
2) Clustering
where each binding site is represented by bi- nary mask m. E 13 Here, m 1 indi- cates that the j-th residue is part of the i-th site, while m = 0 means it is not. Currently, most learning-basse
Direct Set Prediction
binding site detection methods adopt a discontinuous workflow: first predict- ing a score for each amino acid residue or heavy atom, and then clustering them into distinct bind- ing sites. To streamline this process, we propose UniSite, the first UniProt-centric and direct set prediction approach that adheres to the end-to- end paradigm (Figure 2). Two components are essential for direct set prediction in this context: (1) set prediction loss based on bijective match- ing between predicted and ground truth binding sites; and (2) an architecture capable of predicting a set of sites in a single forward pass. The architecture of UniSite is shown in detail in Figure
(Ours)
Figure 2: Comparison of detection approaches. (Top) Conventional learning- -baeed binding site detection methods typically employ a discontin- uous workflow: first predicting binary masks for residues/atoms, then clustering these masks into distinct binding sites. (Bottom) In contrast, our method directly outputs a set of N potentially over- lapping binding sites in a single step.
3.1 Set prediction loss for binding site detection
UniSite infers a fixed-size set of N predictions 2 = ((,,, mz) mm; E (0, in single forward pass, where m; represents the predicted binding site, and p. denotes the probability of binding and @ (non-binding) category. Since the ground truth set |2 = No and the prediction set |2| = N typically have unequal sizes, we assume N 2 Not and pad the ground truth set with 0 (non- binding) tokens. The padded ground truth set defined as 2 {(c," .mff)) cf' E {1,0), mf"
{0, 1
Ji= where of' = indicates a true binding site and cf 0 corresponds to the padding.

---

Niite queries
protein sequence MSRSAFTALL
Nitte scores
linear
residue index 1, 3,
input embedder
transformer encoder
transformer decoder
MLP
classification loss
ESM features
key,
embeddings
binary
sequence encoder
protein structure
mask predictions
pair-wise dot-product
residue-level features
graph neural network
structural encoder (optional)
decoder
segmentation module
The architecture of UniSite. Our models employ encoder extract the residue-level features. Then a decoder module used to generate embeddings of the N predicted binding sites. Finally, the segmentation module outputs N potentially overlapping binding site predictions. The encoder comprises dual pathways: sequence encoder and an optional structural encoder, allowing UniSite to operate with either sequence-only input combined sequence- strucccrrt information.
Figure 3:
To train a set prediction model, we require bijective matching o between the predicted set and the padded ground truth set gt
ped This matching obtained by minimizing matching cost Cmatch:
arg min Lmanch (2," (0(1))
where a is a permutation of N elements and Lmath quantifies the pairwise matching cost between ground truth site =!" and the prediction with index (i). Following prior work [32] [33,, we employ the Hungarian algorithm to compute the optimal matching and use the matching cost defined as:
Ematch (2 (0(1)) - #0> (og (cf") +1 [o( wmw.smmmm fmms ma(s))
(2)
where Lmass Xice + Adicehdice is combination of BCE loss and dice loss 34 Po(t) and ma(() denote the predicted probability and the binding site for the (ii)-th prediction, respectively. This matching cost considers both the class prediction and the similarity of the predicted and ground truth binding sites. Given the optimal matching ē compose a cross-entropy classification loss and the binary mask loss Lmas for each predicted site train model parameters:
N
CmaskRecaa20 = Auts Σ
- log Pa(i) (ct") + 1 1096 *0) Crmce ma(s))
(3)
3.2 UniSite architecture
As illustrated in Figure UniSite comprises three main components: (1) an encoder module that extracts residue-level representations F Lxxdsw of the protein; (2) a decoder module consisting of multiple Transformer decoder layers generate embeddings of the N predicted binding sites, and (3) segmentation module which combines the residue- -eveeeeeeeeeelleeee representations and the decoder embeddings produce the final predictions {(p.. m;))mm) m, E {0. 1) J= This architecture maintains conciseness while demonstrating strong compatibility with existing protein representation methods. In our implementation, we construct two variants: UniSite- 1D using only sequence encoding, and UniSite-3D incorporating both sequence and structural encoders.
Sequence encoder. The amino acid sequence encodes the primary information of a protein and serves as the fundamental input for the UniProt-centric binding site detection. First, an input embedding module receives three inputs: (1) learnable embeddings for the 21 amino acid types (20 standard amino acids plus an "unknown" category); (2) sinusoidal positional embedding [35) for residue indices; (3) pre-trained ESM-2 (36] protein embeddings. These three components are

---

concatenated along the feature dimension, and subsequently processed by 3-layer multilayer perceptron (MLP) to generate the initial per-residue features. Then a stack of Transformer encoder layers process the combined features to capture residue-residue interactions and global sequence patterns.
Structural encoder. Protein structure serves as the most critical input for structure- based tasks, including ligand binding site detection. Recent advances have proposed various structural feature extraction approaches, including hand-crafted algorithms [T] CNN-based methods [2 37 and graph neural network approaches [13] 15 To demonstrate the generality and effectiveness of our approach, we utilize GearNet-Edge [38], a standard E(3) innvarant GNN model, without introducing any custom architecture or specialized feature engineering.
Given protein P, the protein structure is represented a residue-level relational graph g (V, R)), where V and & represent the set of nodes and edges respectively, and R. is the set of edge types. Each node in the protein graph represents the alpha carbon of residue, while sequential edges, radius edges and K-nearest neighbor edges are considered in the graph. Based on the defined protein graph, the node features are updated through the relational graph convolution layers (3)] as follows:
( u
ReLU BN
W L 4 ) FER JEN-()
Σ
11 h (1-1)
un (1)
where h represents the feature of node i at the 1-th layer, .V-(1) fj i)), E E denotes the neighborhood of node with the edge type r, and W,, the convolutional kernel matrix shared within the edge type r. Specifically, BN represents a batch normalization layer and ReLU denotes the ReLU activation function.
Unlike conventional binding site detection methods that rely on structural input, our framework allows the structural encoder be optionally included. When incorporated, the structural features are concatenated with sequence features and projected via a linear layer to match the decoder's ' channels.
Transformer decoder. The decoder comprises multiple Transformer decoder layers [35] that simultancously process N embeddings of dimension dmodel- Q E: R xdduadd through multi-head self-attention and cross-attention mechanisms. Following established practices in [32 40), these input embeddings are learnable positional embeddings which refer to site queries. The attention mechanisms enable the decoder to perform global reasoning over all potential binding sites while incorporating contextual information from the residue-level protein features output by the encoder.
Segmentation module. We process the N site queries through a linear classifier followed by the softmax activation to generate class probabilities {p: (pette, . lnşite pf. E [0, 1]} Here, the classifier predicts an additional @ (non-binding) category to indicate when a query does not correspond to any actual binding site. For mask prediction, the site queries Q IR "dudni are converted to N mask embeddings Emack E R Xdanm by MLP. Finally, the binary mask prediction for each query is computed via dot- producuuttiin between the i-th mask embedding and the residue-level protein features F E IR I.x dhods, followed by a sigmoid activation:
miDi = sigmoid (Emask [i, -F..1"")
(5)
4 Rethinking the Evaluation Metrics for Binding Site Detection
DCC (Distance between the predicted binding site center and the true binding site center) and DCA (Shortest distance between the predicted binding site center and any heavy atom of the ligand) are the two most widely-used metries for binding site detection. A binding site prediction is considered successful when its DCC or DCA value is below a predetermined threshold. Previous works [II 37 [4 quantify prediction performance via the Success Rate, defined as the ratio of successful predictions to the total number of ground truth sites:
Success Rate (DCC or DCA) [Predicted sites DCC or DCActhreshold))
(6)
[[round truth sites)
However, these metrics suffer from two critical limitations: (Limitation 1) They disregard the prediction scores or ranks, and predictions may be double-counted due to the absence of proper matching criteria (Figure 4 A and Table SII.. (Limitation 2) They only evaluate the center of

---

A
B
C
D
center
cemter gouss center
canter pneponte
canter specth
conter pespeceet
contor spont
gt site center pred site center
DCC,=4.000 00A, DCC,=4.40A D0A,-1.83A DCA,=1.69A
DCC=2.57A
loU=0.12
DCC=0,477 4-40 loU=0.00
DCA=1.75A
DCA=0.46A
Figure 4: DCC or DCA failure cases. (A) Repeated counting of the same predicted site since absence of matching. (B) Different ligands bound to the same site lead to deviations in DCC or DCA calculations. (C-D) Failed predictions classified as successful by DCC or DCA but below the IoU threshold.
binding sites and are ligand-dependent (DCC typically considers the ligand center as site center). Since different ligands can bound to one binding site, relying solely ligand-centered evaluation causes these metrics to completely miss key structural properties such as the shape, size, and residue composition of the binding site. It leads to evaluation failures in certain scenarios (Figure 4 B-D), disregard the crucial information required for downstream tasks (Appendix
The quantitative analysis of the DCC and DCA metrics is provided in Appendix G to further substantiate their evaluative flaws. The analysis reveals that approximately 20% of proteins are subject to double-counting during evaluation. Furthermore, the measured mean ground truth DCC (2.15 Å, 92.65% < 4 Å) and DCA (1.57 Å, 98.88% < 4 Å) exhibit significant deviation from the ideal value of 0, revealing a systematic bias inherent to these metrics. These results directly correspond to the previously discussed limitations and confirm that DCC and DCA metrics significantly distort model performance assessment.
Previous works have recognized these limitations. To address Limitation 1, a common approach is calculate the DCC DCA for either the top-n top-(n+22) predicted binding sites [11] 13 where n is the number of ground truth sites. For Limitation 2. DeepSurf [4] and Utgés et al. 42 propose computing the IoU between the predicted and ground truth binding site residues. Given two binding sites mA, mB E {0, where L is the protein sequence length, the IoU of two sites is defined as:
14)
sum(m48mB)
IoU(mA, mg) sum(mn |m))) where m, mg E (0, 13
However, these methods fail to address the core issues, as they still lack proper matching between predicted and ground truth sites, and the top-n or top-(n+2) metries introduce information leakage.
To overcome these limitations, we propose to calculate the Average Precision (AP) metric based on the residue-level ToU as fair metric for method evaluation. We calculate AP as follows: First, we sort all predictions by confidence scores. Then, we match each ground truth site the predicted site with the highest score and residue-level IoU above predetermined threshold, enforcing one-to-one assignment constraint. Finally, we compute AP as the area under the interpolated precision-recall curve following COCO evaluation protocols 4 which is widely used in object detection. The pseudo-code for AP calculation is provided in Appendix K The AP metric offers two significant advantages: (1) the residue-leve IoU enables accurate shape and size comparison between binding sites; (2) the one-to-one matching scheme inherently prevents double-counting of predictions.
Experiments
5.1 Settings
UniSite-DS used for training and validation. We employ MMSeq2 44 ensure that no test UniProt sequence has similarity above 0.9 to any sequence in the training set. For each UniProt sequence, we select the PDB structure with the highest sequence identity as representative structure. Additionally, we compare UniSite with baseline methods on widely-used binding site benchmark datasets, HỘL04K II] and COACH420 [T]. Following DeepSurf (41 and EquiPocket [13), we use the mlig subsets of HOL04K and COACH420 for evaluation. Since
Dataset.

---

our models are trained under a UniProt-centric schema, we only consider single-chain structures, denoting the test datasets as HOLO4K-se and COACH420 (all structures in COACH420 are originally single-chain). All test UniProt entries are strictly excluded from the training set. More details are provided in Appendix F
Implementation details. We set dmndel = 256 by default. The transformer encoder consists of 6 standard Transformer encoder layers with a feed-forward dimension 1024 and the dropout rate of 0.1. We employ Transformer decoder layers following the architecture of DETR 32 By default, We use 32 site queries, where each query is associated with a learnable positional encoding and a zero- iero-initialized query embedding. The multi-layer perceptron in the segmentation module consists of 2 hidden layers with 256 channels. For mask prediction, we use a combination of BCE loss and dice loss (34]
Lassk = Xace Adice
(8)
where Ance Adice 5.0. The classification loss weight Acts is set to 2.0, and we downweight the classification loss by factor of 10 when ci' = 0 to mitigate class imbalance. Following DETR (22. we apply segmentation modules which share the same weights after each decoder layer, and supervise their predictions by the set prediction loss. We optimize the model using the AdamW optimizer 45 with learning rate of 1.0 x 10-4 and a weight decay factor of 0.05. For the structural encoder, we implement the GearNet-Edge network following the origin paper (38 without specialized feature engineering. Notably, we train the GearNet- BearNee-EEdge network from scratch rather than loading pre-truc pre-trained weights. All models are trained on NVIDIA RTX 4090 GPUs.
Our method predicts the associated residues along with a confidence score for each binding site. For applications requiring binding site centers, we compute these as the centroids of the convex hull encompassing all atoms within each predicted binding site 14 46
Table 1: Results on UniSite-DS. We highlight the top two performing methods for each metric in bold. Fpocket-rescore denotes sites initially predicted by Fpocket and subsequently rescored by P2Rank. VN-EGNN only outputs centers of predicted sites. For each center, We include the
residues within 9Å radius, which has the best AP performance (Appendix H
Method
Type
Input
APo.3t
APo.st
Fpocket g
Geometry-base
structure
0.1836
0.1017
Fpocket-rescore" r
structure + Fpocket result structure
0.5075
0.2349
P2Rank |||
Machine-leariing
0.5056
0.2157
DeepPocket
12
CNN-based
structure + Fpocket result
0.4273
0.2334
GrASP 14
structure
0.4469
0.2848
VN-EGNN
15
GNN-based
structure
0.1621
0,0705
UniSite-ID UniSite-3D
sequence
0.5121
0.3033
Ours
structure
0.5603
0.3835
Evaluation metrics. For comprehensive evaluation, compare IoU-based AP and traditional DCC or DCA metrics in HOLO4K-sc and COACH420. Since a Uniprot-centric data entry can contain ligands from multiply PDB structures, coordinate-dependent metries like DCC and DCA become unsuitable due to potential inconsistencies in ligand spatial arrangements. Consequently, we merely use loU-based AP metric on Uni Following EquiPocket (13), we set the DCC or DCA threshold to 4A, and compute the DCC or DCA success rate of top-n predictions. We calculate AP using IoU thresholds of 0.3 and 0.5.
5.2 Results on UniSite-DS
The results on UniSite-DS are shown in Table The geometry-based method Fpocket [8] exhibits inferior performance since it merely considers the geometry and electronegativity. P2Rank [I] achieves better results by extracting the protein surface features with Random Forest. DeepPocket [2 utilizes 3D-CNN to rescore and refine Fpocket predictions, improving the preformance in APo.s- Notably, rppccee--escooee which combines Fpocket's 's initial predictions with P2Rank's re-ranking, surpasses both P2Rank and DeepPocket, highlighting the importance of proper scoring in binding site detection as captured by our AP metric. For graph models, GrASP [14 achieves further improvements in APo.s by employing graph attention networks. However, VN-EGNN [5] performs poorly under
8

---

AP metrics, because it only outputs predicted binding site centers, discarding structural properties (shape and size) or residue identification. For evaluation purposes, we include the residues within 9A radius of each predicted center, which has the best AP performance (Appendix H
Trained on the UniProt-centric dataset, UniSite- 1D outperforms all baseline methods without protein structure, demonstrating remarkable capability for structure-free binding site detection, particularly valuable for site-aware protein-ligamt docking (Appendix [A). UniSite-3D further improves the performance remarkably by incorporating structure information. The above observations not only validate the effectiveness of our methods, but also reveal the significant statistical biases inherent in previous PDB-centric datasets, which limit the performance of prior methods.
5.3 Results on HOLO4K-sc and COACH420
The results on HOLO4K-sc and COACH420 are shown in Table 2 Fpocket 8 P2Rank and DeePocket 12 exhibit consistent performance ranking across different datasets and metrics. The improvement achieved by Fpocket-rescore is also consistently evident. These indicate the concordance between our proposed loU-based AP and traditional DCC or DCA metrics. Besides, the AP metric demonstrates superior discriminative power. On HOLO4Kssc, while DeepPocket and GrASP [4] show almost identical performance in DCAxop-n (< 0.01), they diverge substantially (> 0.10) in APo.3. Similarly, on COACH420, performance differences among Fpocket-rescor, P2Rank, and GrASP are more pronounced under AP evaluation than with DCC metrics. As exception, VN-EGNN [5] performs well DCC or DCA while performing poorly under AP, as merely predicts the binding site centers, discarding structural properties and reisidue identification. Notably, is problematic since both the structural properties and the residue identification binding sites critical for downstream tasks (Appendix(A). Since UniSite-DD DS is UniProt-centric dataset while HOLO4K-sc and COACH420 are both PDB-centric, there is a training-test gap for UniSite-ID/3D. Even so, both UniSite- ID and UniSite-3D maintain strong performance the PDB-centric benchmarks across all evaluation metrics, demonstrating the effectiveness of our framework. More results are provided in Appendix
Table 2: Results on H0L04K -ce and COACH420. We highlight the top two performing methods for each metric in bold. Fpocket-rescore denotes sites initially predicted by Fpocket and subsequently rescored by P2Rank. b VN-EGNN only outputs centers of predicted sites. For each center, We include the residues within a 9Å radius, which has the best AP performance (Appendix H).
HOLO4K
COACH420
Method
APo.at
DCC
DCAio--n
APo.3t
DCCop--
DCAm 1
Fpocket g
0.2711
0.3076
0.4382
0.2106
0.2708
0.4107
Fpocket-rescore"
0.5899
0.5183
0.7654
0.5602
0.4405
0.7113
P2Rank 11
0.6011
0.5300
0.8188
0.6188
0.4643
0.7411
DeepPocket
12
0.5415
0.4925
0.7369
0.5184
0.3958
0.6756
GrASP 14
0.6668
0.5131
0.7416
0.7150
0.4851
0.7620
VN-EGNN
15
0.2606
0.5861
0.6999
0.2637
0.5446
0.7530
UniSite- ID (ours)
0.6867
0.5538
0.7692 0.7879
0.5921
0.4554
0.7351
UniSite-3D 3D (ours)
0.7091
0.5716
0.7196
0.4702
0.7381
Table 3: Effect of sequence similarity. The sec- ond column indicates the sequence identity be- tween training sets and test sets.
Table 4: Effect of site queries. This table shows results of UniSite-3D trained with varying number of site queries.
Method
Similarity
APost
APo.sT
# of queries
APo.st
APo.st
UniSite-1D Uni
0.5121
0.3033
16
0.5515
0.3795
<0.9
0.5603
0.3835
32
0.5603
0.3835
48
0.5562
0.3861
UniSite- ID UniSite-3D
0.5056
0.2945
<0.7
0.5579
0.3734
64
0.5615
0.3867
UniSite- ID Uni
0,4338
0.2243
<0.5
0.4677
0.2801

---

5.4 Ablation study
Sequence similarity. Both the structure and function of proteins diverge with the decreasing of sequence similarity. It is necessary to evaluate our protein ligand binding site detection method across varying similarity thresholds. We employ MMSeqs2 [44] to partition UnSite-DS with three similarity thresholds: 0.5, 0.7 and 0.9, ensuring that no test protein exceeds the corresponding similarity to any training protein. Compared to threshold 0.9, UnSite- ID/3D exhibit only a slight decrease under threshold 0.7 (Table 3 indicating that methods possess generalization ability. As proteins with sequence similarity below 0.5 typically belong to evolutionarily distant families and exhibit markedly different structural folds, we observe significant AP performance degradation for UniSite-11 1 and UniSite-3D under threshold 0.5.
Number of site queries. As shown in Table UniSite-3D exhibits stable performance across varying numbers of site queries. select 32 queries as default configuration, considering both the computation cost and the coverage of ground truth binding sites (99.5% proteins in UniSite-DS have sites less than 20).
Concluding Remarks and Future Perspectives
Key Contributions. In this paper, we introduce UniSite-DS, the first UniProt-centric dataset of protein ligand binding sites, which systematically integrates all ligand binding sites across multiple PDB structures for each unique protein. UniSite-DS corrects the statistical bias in previously available PDB-centric datasets and methods, while significantly broadening the available data. To amend the discontinuous workflows in existing binding site detection methods, we proposed UniSite- MiaSite-ID/3DD ID/DD, two end-to-en methods supervised by set prediction loss with bijective matching. In addition, introduce loU-based AP as a more accurate evaluation metric. Extensive experiments on UniSite-DS and several benchmark datasets demonstrate that frameworks achieve superior performance, and the loU-based AP metric can provide a more accurate reflection of binding site prediction quality.
Limitations and Future Work. The current version of UniSite-DS involves manual curation to remove unreasonable entries. A promising direction for future work is to develop automated methods for repairing and reintegrating excluded data to further enhance the dataset's coverage and quality, Additionally, our current model design aims to demonstrate the effectivenesss of the end-to-end ligand binding site learning framework, without incorporating specialized feature engineering. Future investigations could explore the inclusion of specialized feature engineering to further improve model performance and generalization ability.
Acknowledgements
We thank the reviewers for their constructive feedback, and Hannes Stärk, Chenyu Wang, Zuobai Zhang, Zaixi Zhang, and Bozitao Zhong for helpful discussions. Liwei Wang is supported by National Science and Technology Major Project (2022ZD01 14902) and National Science Foundation of China (NSFC92470123,, NSFC62276005).
References
[1] Ryosuke Nakashima, Keisuke Sakurai, Seiji Yamasaki, Kunihiko Nishino, and Akihito Yam- aguchi. Structures of the multidrug exporter acrb reveal a proximal multisite drug-binding pocket. Nature, 480(7378):565--66) 201
[2] Fabien Vincent, Arsenio Nueda, Jonathan Lee, Monica Schenone, Marco Prunotto, and Mark Mercola. Phenotypic drug discovery: recent successes, lessons learned and new directions. Nature Reviews Drug Discovery, 21(12):899 9944, 2022.
[3] HC Stephen Chan, Yi Li, Thamani Dahoun, Horst Vogel, and Shuguang Yuan. New binding sites, new opportunities for gper drug discovery. Trends biochemical sciences, 44(4):312-330, 2019.
10

---

[4] Stephen J Campbell, Nicola D Gold, Richard M Jackson, and David R Westhead. Ligand binding: functional site location, similarity and docking. Current opinion in structural biology. ((()):89-395, 2003.
[5] Shuya Li, Tingzhong Tian, Ziting Zhang, Ziheng Zou, Dan Zhao, and Jianyang Zeng. Pocketan- chor: Learning pructure-bassd pocket representations for protein-ligand interaction prediction. Cell Systems, 1498):692-705, 2023.
[6] Alexander S Powers, Helen H Yu, Patricia Suriana, Rohan V Koodli, Tianyu Lu, Joseph M Paggi, and Ron Dror. Geometric deep learning for structure-based ligand design. ACS Central Science, 912):2257-2266) 2023.
[7] Arne Schneuing, Charles Harris, Yuanqi Du, Kieran Didi, Arian Jamasb, Ilia Igashov, Weitao Du, Carla Gomes, Tom L. Blundell, Pietro Lio, et al. Structure-based drug design with equivariant diffusion models. Nature Computational Science, 4(12):899-909 2024.
[8] Vincent Le Guilloux, Peter Schmidtke, and Pierre Tuffery. Fpocket: an open source platform for ligand pocket detection. BMC bioinformatics, 10:1-11, 2009.
[9] Michal Brylinski and Jeffrey Skolnick. A threading-based method (findsite) for ligand-binding site prediction and functional annotation. Proceedings of the National Academy of sciences, 105(1):129-135 134, 2008.
[10] Chi-Ho Ngan, David Hall, Brandon Zerbe, Laurie Grove, Dima Kozakov, and Sandor Vajda. Ftsite: high accuracy detection of ligand binding sites on unbound protein structures. Bioinformatics, 28(2):286 28772888688722222 2012.
[11] Radoslav Krivák and David Hoksza. P2rank: machine learning based tool for rapid and accurate prediction of ligand binding sites from protein structure. Journal of cheminformatics, 10:1-12, 2018.
[12] Rishal Aggarwal, Akash Gupta, Vineeth Chelur, CV Jawahar, and Deva Priyakumar. Deep- pocket: ligand binding site detection and segmentation using 3d convolutional neural networks. Journal of Chemical Information and Modeling, 62(21):5099-5079, 2021.
[13] Zhewei Wei, Ye Yuan, Chongxuan Li, Wenbing Huang, et al. Equipocket: an e (3)-equivrriant geometric graph neural network for ligand binding site prediction. Forty-first International Conference Machine Learning, 2023.
[14] Zachary Smith, Michael Strobel, Bodhi P Vani, and Pratyush Tiwary. Graph attention site prediction (grasp): identifying druggable binding sites using graph neural networks with attention. Journal of chemical information and modeling, 64(7):2637-2644, 2024.
[15] Florian Sestak, Lisa Schneckenreiter, Johannes Brandstetter, Sepp Hochreiter, Andreas Mayr, and Günnter Klambauer. Vn-egnn: E (3)-qqivvriaat graph neural networks with virtual nodes enhance protein binding site identification. arXiv preprint vrXiv:2404.07199 2024.
[16] Jérémy Desaphy, Guillaume Bret, Didier Rognan, and Esther Kellenberger. sc-pdb: 3d- database of ligandable binding sites-10 years on. Nucleic acids research, 43(D1):D39-D440) 2015.
[17] Renxiao Wang, Xueliang Fang, Yipin Lu, and Shaomeng Wang. The pdbbind database: Collection of binding affinities for protein- ligand complexes with known three-dimensional structures. Journal of medicinal chemistry, 47(12):2977-2980, 2004.
[18) Peter Schmidtke, Catherine Souaille, Frédéric Estienne, Nicolas Baurin, and Romano Kroe- mer. Large-scale comparison of four binding site detection algorithms. Journal of chemical information and modeling, 50 30(12):2191-2200 2010.
[19] Jianyi Yang, Ambrish Roy. and Yang Zhang. ligand binding site recognition us- ing complementary binding-specifie substructure comparison and sequence profile alignment. Bioinformatics, 29(20):2588225 258,

---

[20] Feng He, Cheng-Gue Wu, Yang Gao, Sabrina N Rahman, Magda Zaoralová, Makaía M Papasergi-Scott, Ting-Jia Gu, Michael J Robertson, Alpay Seven, Lingjun Li, et al. Allosteric modulation and g-protein selectivity of the ca2+-sensing receptor. Nature, (26(8001):1141- 1 1148, 2024.
[21] Edda SF Matthees and Carsten Hoffmann. The ca2+-sensing receptor and the pocketome: com- paring nature's complexity with human intervention in receptor modulation. Signal Transduction and Targeted Therapy. 9(1):173, 2024.
[22] Johannes Morstein, Victoria Boweut, Micah Fernando, Yue Yang, Lawrence Zhu, Meredith Jenkins, John T Evans, Keelan Z Guiley, D Matthew Peacock, Sophie Krahnke, et al. Targeting ras-, rho-, and rab-family gtpases via a conserved cryptic pocket. Cell, 187(22):6379-6) 6392, 2024.
[23] UniProt Consortium. Uniprot: a worldwide hub of protein knowledge. Nucleic acids research, (7551))550--511 2019.
[24] Jifa Zhang, Yinglu Zhang, Jiaxing Wang, Yilin Xia, Jiaxian Zhang, and Lei Chen. Recent ad- vances in alzheimer's 's disease: Mechanisms, clinical trials and new drug development strategies. Signal transduction and targeted therapy, 9(1::111, 2024.
[25] Timothy P Hughes, Michael J Mauro, Jorge E Cortes, Hironobu Minami, Delphine Rea, Daniel DeAngelo, Massimo Breccia, Yeow-Tee Goh, Moshe Talpaz, Andreas Hochhaus, et al. Asciminib in chronic mycloid leukemia after abl kinase inhibitor failure. New England Journal of Medicine, 381 381():)3355-2333 2019.
[26] Lauren M Slosky, Yushi Bai, Krisztian Toth, Caroline Ray, Lauren K Rochelle, Alexandra Badea, Rahul Chandrasekhar, Vladimir M Pogorelov, Dennis M Abraham, Namratha Atluri, et al. 3-arrestin-biased allosteric modulator of ntsrl selectively attenuates addictive behaviors. Cell, (81(6))136441333 2020.
[27] Jigang Fan, Yaqin Liu, Ren Kong, Duan Ni, Zhengtian Yu, Shaoyong Lu, and Jian Zhang. Harnessing reversed allosteric communication: novel strategy for allosteric drug discovery. Journal of Medicinal Chemistry, 64(24))17728-17744 2021.
[28] Christos P Feidakis, Radoslav Krivak, David Hoksza, and Marian Novotny. Ahoj: rapid, tailored search and retrieval of apo and holo protein structures for user-defined ligands. Bioinformatics, 38(24):5452-5453 2022.
[29] Helen Berman, Kim Henrick, and Haruki Nakamura. Announcing the worldwide protein data bank. Nature structural & molecular biology, 10(12):980-985 2003.
[30] Sameer Velankar, José M Dana, Julius Jacobsen, Glen Van Ginkel, Paul J Gane, Jie Luo, Thomas Oldfield, Claire O'Donovan, Maria-Jesus Martin, and Gerard J Kleywegt. Sifts: structure integration with function, taxonomy and sequences resource. Nucleic acids research, 41(DI):D483-D433 2012.
[31] Tomas Geffner, Kieran Didi, Zuobai Zhang, Danny Reidenbach, Zhonglin Cao, Jason Yim, Mario Geiger, Christian Dallago, Emine Kucukbenli, Arash Vahdat, et al. Proteina: Scaling flow-based protein structure generative models. arXiv preprint arXiv:2503.00710, 2025.
[32] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In European conference on computer vision, pages 213-229. Springer, 2020.
[33] Russell Stewart, Mykhaylo Andriluka, and Andrew Y Ng. End-to-end people detection in crowded scenes. In Proceedings of the IEEE conference computer vision and pultern recognition, pages 2325 2233,, 2016.
[34] Fausto Milletari, Nassir Navab, and Seyed-Ahmad Ahmad Ahmadi. V-net: Fully convolutional neural networks for volumetric medical image segmentation. In 2016 fourth international conference on 3D vision (3DV), pages 565-571. Iece, 2016.
12

---

[35] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural information processing systems, 30, 2017.
[36] Alexander Rives, Joshua Meier, Tom Sercu, Siddharth Goyal. Zeming Lin, Jason Liu, Demi Guo, Myle Ott, C Lawrence Zitnick, Jerry Ma, et al. Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. Proceedings of the National Academy of Sciences, 1 ((115)):2216231111 2021.
[37] José Jiménez, Stefan Doerr, Gerard Bartinez-Roselly Alexander S Rose, and Gianni De Fabritiis. Deepsite: protein-binding site predictor using 3d-convolutional neural networks. Bioinformatics, 33(19):3036-304 2017.
[38) Zuobai Zhang, Minghao Xu, Arian Jamasb, Vijil Chenthamarakshan, Aurelie Lozano, Payel Das, and Jian Tang. Protein representation learning by geometric structure pretraining. In International Conference on Learning Representations, 2023.
[39] Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. Modeling relational data with graph convolutional networks. In The semantic web: 15th international conference, ESWC 2018, Heraklion, Crete, Greece, June 3-7, 2018, proceedings 15, pages 593-607. Springer, 2018.
[40] Bowen Cheng, Alexander G. Schwing, and Alexander Kirillov. Per-pixel classification not all you need for semantic segmentation. In Advances in Neural Information Processing Systems, 2021.
[41] Stelios K Mylonas, Apostolos Axenopoulos, and Petros Daras. Deepsurf: surface-based deep learning approach for the prediction ligand binding sites on proteins. Bioinformatics, 37812)1681-1169, 2021.
[42] Javier Utgés and Geoffrey Barton. Comparative evaluation of methods for the prediction of protein-ligand binding sites. Journal of Cheminformatics, 16(1):126, 2024.
[43] Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. arXiv preprint arXiv: 1405.0313, 2014.
[44] Martin Steinegger and Johannes Söding. Clustering huge protein sequence sets in linear time. Nature communications, 9(1):2542, 2018.
[45] Iyya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint 17X1v:1711.05101, 2017.
[46] R Tyrrell Rockafellar. Conjugate convex functions in optimal control and the calculus of variations. Journal of Mathematical Analysis and Applications, 32(1):174-222, 1970.
[47] Marcel L Verdonk, Jason C Cole, Michael J Hartshorn, Christopher W Murray, and Richard D Taylor. Improved protein ligand docking using gold. Proteins: Structure, Function, and Bioinformatics, 52(():609 -233699966666666 2003.
[48] Oleg Trott and Arthur Olson. Autodock vina: improving the speed and accuracy of docking with new scoring function, efficient optimization, and multithreading. Journal of computational chemistry, 31 312))555-4611 2010.
[49] Zhirui Liao, Ronghui You, Xiaodi Huang. Xiaojun Yao, Tao Huang. and Shanfeng Zhu. Deep- dock: enhancing ligand-protein interaction prediction by a combination of ligand and structure information. In 2019 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pages 31 3173317. IEFE, 2019.
[50] Gengmo Zhou, Zhifeng Gao, Qiankun Ding, Hang Zheng, Hongteng Xu, Zhewei Wei, Linfeng Zhang, and Guolin Ke. Uni-mol: A universal 3d molecular representation learning framework. In The Eleventh International Conference Learning Representations, 2023.
13

---

[51] Patrick Bryant, Atharva Kelkar, Andrea Guljas, Cecilia Clementi, and Frank Noé. Struc- ture prediction of protein-ligand complexes from sequence information with umol. Nature Communications, 15(1):4536, 2024.
[52] Martin Buttenschoen, Garrett M Morris, and Charlotte M Deane. Posebusters: Ai-based docking methods fail to generate physically valid poses or generalise to novel sequences. Chemical Science, (399):3130-311 2024.
[53] Josh Abramson, Jonas Adler, Jack Dunger, Richard Evans, Tim Green, Alexander Pritzel, Olaf Ronneberger, Lindsay Willmore, Andrew Ballard, Joshua Bambrick, et al. Accurate structure prediction of biomolecular interactions with alphafold 3. Nature, 630(8016):493-500,, 2024.
[54] lambic Therapeutics. Transforming computational drug discovery with neuralplexer2. https:/www..ummiicca/ppsstt
2024. Accessed: 2025-05-01.
[55] Manfred Hendlich, Friedrich Rippmann, and Gerhard Barnickel. Ligsite: automatic and efficient detection of potential small nolecule-binding sites in proteins. Journal of Molecular Graphics and Modelling, 15(6):359-363, 1997.
[56] Jie Liang, Clare Woodward, and Herbert Edelsbrunner. Anatomy of protein pockets and cavities: measurement of binding site geometry and implications for ligand design. Protein science, 799):1884-1897 1998.
[57] Daniele Toti, Le Viet Hung, Valentina Tortosa, Valentina Brandi, and Fabio Polticelli. Libra- wa: a web application for ligand binding site detection and protein function recognition. Bioinformatics, 34(5):878-8
2018.
[58] Radoslav Krivák and David Hoksza. Improving protein-ligande binding site prediction accuracy by classification of inner pocket points using local features. Journal of cheminformatiis, 7:1-13, 2015.
[59] Zhao Yang, Bing Su, Jiahao Chen, and Ji-Rong Wen. Interpretable enzyme function prediction via residue-level detection. arXiv preprint arXiv:2501.05644 2025.
[60] Franck Da Silva, Jeremy Desaphy, and Didier Rognan. Ichem: versatile toolkit for detecting, comparing, and predicting protein-ligand interactions. ChemMedChem, 13(6):507-510, 2018.
[61] Clemens Isert, Kenneth Atz, and Gisbert Schneider. Structure-basee drug design with geometric deep learning. Current Opinion in Structural Biology, 79:102548, 2023.
14

---

A Impact of Binding Site Accuracy on Downstream Tasks
The accuracy of ligand binding site detection is critical for downstream tasks. Taking molecular docking as an example (Figure [ST], comparison of different methods leads to the following conclusions: (1) The definition of binding site residues can strongly impact the docking performance, highlighting the importance of our proposed loU-based AP metric. (2) Docking methods that do not specify binding sites (blind docking) show significant performance improvements when provided with binding site information, emphasizing the importance of accurate binding site identification.
B
E
PoseBusters success rate (RMSD<2A)
90.2
Success rate (96)
68
59
59
54.9
42-
D
20
Gold 25A
Gold 10A
Gold BA
Gold 64
Unii 25A
Uni-Mal 10A
Uni-Mal S
Uni-Mal EA
Umai wro site
HeuniiPlxxer wo slte NouraiPlexer aite
Umai si site
AlphaFolda wio site
AlphaFold wi site
ERRR
Figure S1: The significant impact of binding site detection on molecular docking, (A) Gold 47 defines the binding site using a sphere.
(B) AutoDock Vina 48 defines the binding site using cube. (C) DeepDock (49) and (D) Uni-Mol (50) identify the binding site by applying a fixed radius
around the ligand. (E) Docking success rates on the PoseBusters dataset under different binding site
configurations. Docking success rate is defined as the proportion of predictions with an RMSD less
than 2 Data sourced from 51 53
B UniSite-DS Curation Workflow and Representative Manual Inspection Case Studies
As shown in Figure S2 the UniSite-DS workflow comprises three main components: (l) dataset curation, (II) quality control, and (III) manual inspection. Below, we describe two representative types of cases encountered during manual inspection that required special consideration:
1. Case 1. Supramolecular assemblies across multiple protein subunits. The Photosystem I-LHCI Supercomplex is a core component of the photosynthetic machinery, consisting of multiple subunits such as UniProt ID: P05310 (Photosystem I P700 chlorophyll apoprotein
AI, PDB: Adkz_A), Q41038 (Chlorophyll a-b binding protein, 7dkz_B), and Q32904 (Chlorophyll a-b binding protein 3, 7dkz_C), Although each of these proteins binds large number of Chlorophyll a molecules, they function as part of a tightly integrated supramolecular assembly. Therefore, they are not suitable be included as independent entries in the UniSite-DS database and have been excluded in the current version.
2. Case 2. Large composite cavities jointly formed by multiple ligands. Trypanothione reductase (UniProt ID: Q389T8) is a key enzyme in Trypanosoma brucei that specifically catalyzes the reduction of trypanothione, functionally analogous to glutathione reductase in mammals. The ligands from structures PDB: 5s9x_A, _A 5s9t_A, and 2wov __ together form a large binding cavity. These ligand binding sites could potentially be merged. However, since each ligand actually occupies only a portion of the cavity rather than the entire cavity, whether such a composite cavity formed by different ligands should considered a unified binding site often depends on sseeeffcccc definitions in the literature. As a result, entries that require further iterrture-based validation were excluded from the current version of the UniSite-D DS database.
15

---

Curation Process
IV) Manual Inspection Case Studies
Systematic retrivaal of all protein ligand
(Case Entries involving supramolecular
assomblies across multiple protein subunits
teraction from PCS databese
Ideniffication of binding site residues within each ligand
integration of ligand binding sites acroes
different structures using UniPrat identifi ers and SIF IS annotations
laM D.7 and loU i 0.5 to exclude highly overtapping bindlng sites
Redundancy removal using
Additional quality control and manual in- spection ensure high-quality dataset
II) Quality Control
Exclude structures with resolution =2.5 determined by non-orystalographic methods
(Case 2) Entries with large composite cavities jointly formed by multiple ligands, each occupying only of the cavity
Remove salvent motecules
Discard entries with s3 binding site residues to eliminate floating ligands
III) Manual Inspection
Manual inspection of entries with =10 ligand binding siles with sites contributed by a single protein-lgend complex
Entries involving supramolecular
blees acroes mulliple protein suburits
Entries with large composite cavilies jointly formed by mutippe Igands, each occupying only part of the cavity
Figure S2: Overview of the UniSite-DS workflow. Workflow of (I) dataset curation, (II) quality control, and (III) manual inspection for UniSite-DS, together with (IV) representative manual inspection case studies.
C Related Work
Over the past several decades, numerous methods have been developed for detecting protiin-ligand binding sites, accompanied by advances in techniques leveraging the geometric, physical and chemical features of proteins.
Early methods relied on traditional computational algorithmss. Since most binding sites show up as
cavities in protein 3D structures, geometry-based methods (Fpocket (8) LigSite (55) identify and rank these hollow cavities through hand-crafted features like alpha spheres [56). Template-based
methods (FINDSITE [9] and LIBRA [57]) predict ligand binding sites by comparing the query protein with templates from known protein structure database. These methods typically generate a large number of predicted sites while performing poorly in ranking them.
Subsequent approaches like PRANK (58] and P2Rank [ employ traditional machine learning methods, particularly Random Forest. Based on the predictions of Fpocket (8. PRANK assigns "ligandibility" scores, which denotes ligand binding potential, to candidate sites. P2Rank is widely used method which integrates the geometric features of the protein surface with Random Forest Algorithm.
In recent years, deep learning methods have emerged for protein-ligand binding site detection. CNN- based approaches [12 37 4I| treat protein structures as 3D images, applying 3D convolutional neural networks similar to those used in computer vision. Alternatively, GNN-based methods [15] 14] I3 utilize graph neural networks by constructing graphs incorporating both geometric and chemical features of proteins. Despite their improved performance, these methods typically adopt discontinuous workflows: they first perform semantic segmentation to generate binary masks of potential binding residues/atoms, then cluster these masks into discrete binding sites. This fragmented pipeline heavily depends on post-processing (e.g., clustering algorithms), inherently limiting end-to end-to-end optimization and struggling with overlapping binding sites. DETR-based architectures have also been adapted for other protein tasks, such as ProtDETR [59]. which performs enzyme function classification rather than binding site detection.
16

---

D Baseline Justification
We selected representative baseline methods that span the major methodological paradigms in ligand binding site detection. Fpocket [8] represents traditional coametry--aseeee computational algorithms. P2Rank [T] is the most widely used machine learning approach. Among deep learning approaches, DeepPocket [12,, GrASP [14, and VN-EGNN [13] were included as they represent different neural network architectures: DeepPocket employs 3D CNNs, while GrASP and VN-EGNN utilize GNNs.
For fair and consistent comparison, we extracted ligand binding site residues from each baseline's output files according to their respective standard formats:
For each predicted pocket, Fpocket outputs file named pocket-(index) _amm pdb, which contains the atomic coordinates of the predicted binding site in PDB format. These atoms were directly used to identify the corresponding binding residues.
Fpocket:
P2Rank generates a {name} residue_ide column specifies the chain IDs and residue IDs of residues constituting each predicted binding site. These identifiers were parsed to obtain the binding site residues.
P2Rank:
csv file for each input structure. The
DeepPocket: DeepPocket refines and re-scores the pockets predicted by Fpocket, while maintain- ing the same output format.
GrASP outputs (name) beame)_probs... páb file, where the bfactor column encodes the predicted binding probabilities of heavy atoms. Following the original publication, we filtered atoms based on their predicted scores and clustered them spatially into distinct binding sites.
GrASP:
VN-EGNN produces prediction. csv file containing four columns (x, y. rank) that specify the 3D coordinates and ranking of predicted pocket centers. As the method does not directly output residue-level predictions, binding site residues were obtained by including all residues within a defined radius, as described Appendix H
VN-EGNN:
E
Case Study of Classical Methods
As discussed Section 2 previous approaches are developed based on PDB-centric datasets, which introduce substantial statistical biases. We conduct case study of these methods, and the results (Figure 53) indicate that, for multi-site proteins, existing approaches are weak in distinguishing between different ligand binding sites. This limitation motivated us to investigate the problems in existing approaches and to develop a new methodology.
B
UniSite-3D
C
GrASP
A
Ground Truth Sites
D
P2Rank
E
Fpocket
Figure S3: Case study of classical methods. For UniProt ID Q7YYQ9, the two ground truth binding sites are represented by dark blue and orange meshes, respectively. All predicted binding sites are shown as surfaces. (A) The two ground truth binding sites are colored in dark blue and orange. (B) The two binding sites predicted by our UniSite-3D method are colored in cyan and purple. (C) The single binding site predicted by the GNN-based method GrASP [4] is colored in green. (D) The single binding site predicted by the classical machine learning method P2Rank []] colored in red. (E) The single binding site predicted by the geometry- based method Fpocket [ is colored in cyan.
17

---

F External Datasets
scPDB [T6] is famous dataset for protein-ligand binding site detection, commonly employed for
training and validation in recent studies [I5 14 13). scPDB provides both protein and ligand
structures, accompanied by the structures of binding site extracted via VolSite [60]. Notably, only one binding site and one corresponding ligand are annotated for each data entry. In this work, we use the 2017 release of scPDB, which contains 17, 17,594 structures and 5,550 unique proteins. (Source:
fr/scPDB/)
PDBBind 17 is widely used dataset study protein-ligand interaction, especially for pro- tein-ligand docking [51 61]. Similar to scPDB, PDBBind annotates one ligand structure and one binding site structure in each data entry. In this paper, we use the general set of v2020, the lat- est academic- free edition. which comprises 19,443 structures and 3,888 unique proteins. (Source: http///www.pdbbidd.oom wtw.:/pdbi......cc//oooooo////// cn/dwwnload/)
HOL04K and COACH420 are two benchmark datasets utilized for protein-ligant binding site detection. Follow VN-EGNN [[5], EquiPocket [13] and GrASP [14], we employ the mlig subsets of these two dataset, which contain explicitly specified relevant ligands. HOLO4K-mli comprises 3,204 structures and 1,259 unique proteins, while COACH420-mliy covers 284 structures and 265 unique proteins. (Source: https://gituub..om//////////// daatsss
G Quantitative Analysis of Evaluation Flaws in DCC and DCA Metrics
In Section 4 we discussed two critical limitations in the design of the DCC and DCA metrics. Here, we conduct quantitative analysis on the HOL04K-sc benchmark demonstrate the flawed evaluation introduced by these metrics.
Limitation 1. The absence of proper matching criteria may lead to double-counting of predictions (Figure 4 A). We quantified the proportion of proteins affected by double-counting during evaluation on the HOLO4K benchmark (Table ST.. The results reveal that DCC and DCA metrics suffer from widespread double counting artifacts, which significantly distort model performance assessment.
Table S1: Double counting (DC) rate of DCC or DCA metrics HOLO4K-sc. We highlight the top two performing methods for each metric in bold. Fpocket sites rescored by P2Rank. Residdes within 9 each VN-EGNN predicted center, which has the best AP performance (Appendix H
HOLO4K-sc
Method
DC of DCCrp-nn
DC of DCAmp--
Fpocket
8
18.80%
18.31%
Fpocket-rescore
13.23%
13.66%
P2Rank 11]
18.98%
18.31%
DeepPocket 12
13.53%
14.21%
GrASP [14]
16.29%
14.88%
VN-EGNN 15
12.80%
11.70%
UniSite- ID (ours)
8.88 %
8.94% %
UniSite-3D (ours)
9.86 %
10.04% %
Limitation 2. DCC or DCA only evaluate the center of binding sites and are ligand-dependent, which leads to evaluation failures in certain scenarios (Figure B-D). For HOLO4K-sc, calculated these metrics of the centroid of ground truth binding residues for each protein. The results indicate that the mean ground truth DCC is 2.15 Á (92.65% < Å), and the mean ground truth DCA 1.57 (98.88%, < 4 Å). However, principle, both DCC and DCA should ideally be O when evaluated using ground truth binding residues, indicating these metrics inherently contain systematic bias.
To mitigate some of DCC's inherent limitations, defined corrected metric, DCC-residue, which uses the center of ground truth binding residues rather than the ligand center for calculation. This modification resolves failure cases caused by ligand diversity in traditional DCC evaluation. As shown in Table S2 the corrected DCC-residue metric exhibits improved consistency with loU-based AP ranking the performance different methods.
18

---

Table S2: IoU-based AP, DCC-residue and DCC results on H0L04K-sc. We highlight the top two performing methods for each metric in bold. Fpocket sites rescored by P2Rank. Residues within 9A of each VN-EGNN predicted center, which has the best AP performance (Appendix H
H0LO4K
Method
AP f
DCC-residueeap
DCCup-n 1
Fpocket g
0.2711
0.2982
0.3076
Fpocket-rescore"
0.5899
0.5005
0,5183
P2Rank [1]
0.6011
0.4972
0.5300
DeepPocket 212
0,5415
0,4902
0.4925
GrASP [4
0.6668
0.5379
0,5131
VN-EGNN 15
0.2606
0.5997
0.5861
UniSite- ID (ours)
0.6867
0.6400
0.5538
UniSite-3D (ours)
0.7091
0.6264
0.5716
H The AP Evaluation of VN-EGNN
Since VN-EGNN [S]] outputs only the centers of predicted binding sites, it is non-trivial to identify
the binding residues. order to evaluate the IoU-based AP, we include the residues with a fixed radius for each predicted center (Figure S4 As shown in Table S3 the radius of 9Å has the best performance across all datasets.
A
B
C
4.5A
6A
9A
12A
Figure $4: Binding sites derived from the predicted center by VN-EGNN using different radii Binding site residues are visualized using surface representation: (A) residues within a 4.5A radius; (B) residues within 6Å radius; (C) residues within a 9Å radius; (D) residues within a 12Á radius.
Table S3: AP evaluation results of VN-EGNN using different radii
UniSite-DS
HOL04K-sc
COACH420
Radius
APo.st
APo.st
APo.st
APo.st
APo.st
APo.st
4.5A
0.0054
0.0004
0.0049
0.0001
0.0072
0.0007
6À
0.0894
0.0088
0.1290
0.0172
0.1814
0.0189
9λ
0.1621
0.0705
0.2606
0.1346
0.2637
0.1138
0.0087
0.0010
0.1411
0.0014
0.1444
0.0013
I The Recall Evaluation of Different Methods
The AP metric provides a comprehensive evaluation of binding site detection performance by considering both precision and recall. However, also important to evaluate how many true binding sites can be recovered under different loU thresholds, since the goal of binding site detection to identify novel and biologically meaningful sites/pockets. We computed the Recall of different methods across multiple IoU thresholds the HOLO4K-sc dataset, and the results are summarized in Table S4 UniSite-3D 3D achieves state-of-the oh-thc-rrt Recall across all IoU thresholds.
Notably, the high recall obtained by Fpocket is due to its tendency to output nearly all potential cavities in a protein, rather than accurately identifying biologically relevant pockets. fact, Fpocket tends to assign lower scores to true binding sites, making it challenging for biologists to distinguish meaningful pockets. This limitation has motivated many studies to develop rescoring strategies for Fpocket. While Recall reflects the potential of method to detect biologically significant pockets, it does not account for the confidence score of each prediction. This limitation is addressed by the metric, which is why we adopt it as fair and balanced criterion for evaluating different methods.
19

---

Table S4: The AP and Recall results on H0L04K-se. We highlight the top two performing methods for each metric in bold. Fpocket sites rescored by P2Rank. Residues within 9A of each VN-EGNN predicted center, which has the best AP performance (Appendix H).
H0LO4K
Method
APp.3
APo.5
Recallo.3
Recallo.s
Recallo.7
Recallo.p
Fpocket
g
0.2711
0.1488
0.8361
0.5922
0.2130
0.0253
Fpocket-rescores
0.5899
0.2847
0.8361
0.5922
0,2130
0.0253
P2Rank
0.6011
0.2625
0.7814
0.5337
0.1868
0.0089
DeepPocket 12
0.5415
0.2891
0.7514
0.5824
0.2584
0.022
GrASP 14
0.6668
0.4126
0.7186
0.5374
0,2537
0.0159
VN-EGNN 15
0.2606
0.1346
0.7289
0.4874
0.0566
0
UniSite-1D ID (ours)
0.6867
0.4595
0.8212
0.6199
0.3535
0.0824
UniSite-3D (ours)
0.7091
0.5446
0.8469
0.6901
0.4106
0.1039
J Full Results on HOLO4K-sc and COACH420
Table S5: Full results on HOLO4K-SC. We highlight the top two performing methods for each
metric in bold. Fpocket sites rescored by P2Rank. Residues within 9Å of each VN-EGNN predicted center, which has the best AP performance (Appendix HH).
HOL04K-sc
Method
APo.3
APo.5
DCCrop-n
DCCio--n22
DCAtop-n
DCA1op-n+2
Fpocket 0.2711
0.1488
0.3076
0.4181
0.4382
0.5941
Fpocket-
0.5899
0.2847
0.5183
0.5941
0.7654
0.8577
P2Rank [II]
0.6011
0.2625
0.5300
0.5623
0.8188
0.8652
DeepPocket 112
0.5415
0.2891
0.4925
0.5478
0.7369
0.7851
GrASP 14
0.6668
0.4126
0.5131
0.5267
0.7416
0.7612
VN-EGNN 15
0.2606
0.1346
0.5861
0.6339
0.6999
0.7500
UniSite-11 1D (ours)
0.6867
0.4595
0.5538
0.6400
0.7692
0.8305
UniSite-3B 3D (ours)
0.7091
0.5446
0.5716
0.6470
0.7879
0.8422
Table S6: Full results on COACH420. We highlight the top two performing methods for each metric in bold. Fpocket sites rescored by P2Rank. Residues within 9Å of each VN-EGNN predicted center, which has the best AP performance (Appendix H).
COACH420
Method
APo.3
APo.5
DCCrop-n
DCC 10
DCAop-n
DCA1o--n+2
Fpocket 3
0.2106
0.1219
0.2708
0.3750
0.4107
0.5714
Fpocket-rescoret
0,5602
0.2905
0.4405
0.5179
0.7113
0.8333
P2Rank |||
0.6188
0.2618
0.4643
0.5000
0.7411
0.8034
DeepPocket [2
0.5184
0.2512
0.3958
0.4821
0.6756
0.7560
GIASP 14
0.7150
0.4914
0.4851
0.4970
0.7620
0.7917
VN-EGNN
15
0.2637
0.1138
0.5446
0.6071
0.7530
0.7768
UniSite-ID (ours)
0,5921
0.2998
0,4554
0.5238
0.7351
0.8006
UniSite-3D (ours)
0.7196
0.3977
0,4702
0,5387
0.7381
0.8095
20

---

K
Pseudo-code for Average Precision Calculation
Algorithm Average Precision Calculation
Input:
prediction _ist: A list, where each element represents the predictions for one protein. Each element is of the form {(8;, 88 IR, m, E {0, 1)4) where m, represents the i-th predicted binding site as a binary mask of length L, and 8; denotes
i-th confidence score. ground_ truth truth_list: A list, where each element represents the ground truth (gt) binding sites
for one protein. Each element of the form {m, Im," {0. 1) the i-th ground truth binding site as a binary mask of length L.
Jimi- )
Nge
where m. represents
iou threshold: IoU threshold for considering prediction as True Positive.
Output: AP: Average Precision under the given IoU threshold.
# Step 1: Determine TP (True Positive) or FP (False Positive) for each prediction
1: for truth_list) do ground_trure
groumd_rruthhe.prrpprtttet in ZiPr
2: Set all ground truths in ground_ brwths__eer_prteeiii unused teadd_trttt___pp_e
3: Sort dence score
m;)] E R, m, E {0, 1) by decreasing confi-
4: for = to do
5: Find ground_truth__ that has the max residue-level IoU(m) m.) with predi ction_i
6:
if ToU(m;" m.) > sou_threshold and grooud_truth, is unused then Mark predi ctionciiiiiee. TP
7:
8:
Set pround_truthh.] as used
9:
else
10:
Mark prediction FP
end if
12: end for
end for
Step 2: Sort all predictions across all proteins by decreasing confidence scores
14: all_predictions + FLATTEN ALL(prediciion list)
15: all-ground_trrue truths +- FLATTEN ALL(ground_trrv Ser___rut______ list)
16: Sort all_ppedicciions by decreasing confidence scores
Step 3: Calculate the precision-recall curve
17: Set cum_ TP, cum_FP as 0, pwwppiiiinnniiiieeeeepeeepeepeee and recall list as empty for prediction_i in all all_predictions do
18:
19:
if prediction_i i is marked as TP then
20:
cum_ TP +- cum_TP+
21:
else
cum_FP + cum_FP +
23: end if
cum_ (cum_ cum_FP)
25:
recall_list.ape wit...iidappppdcc.... (eaoonddccmmPPppeee htt.me/P/LLaalllggrooottttttttt
end for
Step 4: Calculate average precision
27: Calculate as area under the precision-recall curve
28: return AP
21

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The claims reflect our contribution about dataset, methodology and evaluation metrics, which are illustrated Section 2, Section and Section respectively.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer:
[Yes]
Justification:
We discussed the limitations in Section 6.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
22

---

Justification:
This paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We described the details of our model architecture, evaluation metries, and experiment settings in Section 3-5.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
23

---

Answer: [Yes]
Justification: The dataset and codes will be made publicly available at httttss://githi com/ quanlin- wwtiinwwwuunssiiiiii unisite
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to to reproduce the results. See the NeurlPS code and data submission guidelines (https
hetpolliiiggiddeeCooooooo SoweeeeeeeooeSbubiiceoo ssicnpolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We described the details of our experiment settings and model implementation in Section 5.1.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Error bars are not reported because it would be computationally expensive.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: As described Section 5.1, all experiments are conducted using NVIDIA RTX 4090 GPUs (24G memory) for about 2 days.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that didn't 't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
We have conducted with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Our work will propagate the development structure based drug discovery.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
25

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system Iearns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: This paper poses such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All assets used in this paper have been properly credited to their original creators or owners. Unless otherwise specified, all methods implemented in this work adhere the MIT License. All datasets include source information URLs, and their usage follows the licensing terms provided by the respective sources.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
26

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The dataset and codes will be made publicly available at htttts::/ggthi com gruunninnww/uunnnn
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
27

---

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification:
The core method development in this research does involve LLMs.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non standaaddard components.
Please refer to our LLM policy for what should or should not be described.
28


Reviews:
[{"review_id": "JvmxvCLHHk", "reviewer": "Reviewer_W7Hr", "summary": "The authors presented a dataset of protein ligand binding sites, two methods for protein ligand binding site detection, and a metric for evaluating the binding site predictions. The paper highlights three main issues. First, that existing datasets are PDB (Protein Data Bank)-centric, specifically focusing on individual protein-ligand structure, which introduces a statistical bias in training and evaluation models. Second, current binding site prediction pipelines are fragmented into disjoint steps. Finally, conventional evaluation metrics used to capture the performance of binding site detection are inadequate; these metrics are often concerned only with the center of the binding sites, they are ligand-dependent, and the absence of proper matching criteria between predictions and ground truth can lead to double-counting of predictions.\nThe authors introduced UniSite-DS, a UniProt-centric dataset that consolidates binding information across all available structures of the same protein; two end-to-end predictive models, UniSite-1D (sequence-based) and UniSite-3D (structure-based); and a new evaluation metric, Average Precision (AP) based on Intersection over Union for comprehensive binding site assessment. The authors run extensive experiments using UniSite and widely used benchmarks for binding site prediction, showing the performance of Unisite-1D/3D in comparison with other models across the AP-IoU and other accepted metrics.", "questions": "- In which way can past datasets induce statistical bias? Do the experiments in the manuscript showcase how this bias is present?\n\n- Are the baseline methods also capable of predicting multiple binding sites?\n\t\n- In Table 3, do the baseline methods show variation in other evaluation metrics?\n\t\n- Why were the thresholds 0.5, 0.7, and 0.9 chosen for sequence similarity?\n\n- In line 210, the expression $p_i \\in \\Delta^2$ is shown: Is $p_i$  a scalar? What is $\\Delta^2$ defined?", "limitations": "Yes.", "rating": 2, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 1, "clarity": 1, "significance": 3, "originality": 3, "strengths_and_weaknesses": "*Strengths*\n\t\n- UniSite 1D is a sequence-based method for predicting protein binding sites that performs similarly to other structure-based models.\n\n- The authors identified problems with the current metrics used to assess binding site prediction. The AP metric doesn’t focus only on comparing the centers of the binding sites. Additionally, it addresses the problem of double-counting predictions.\n\n*Weaknesses*\n\nWhile the paper presents an interesting approach, it suffers from several weaknesses that impact its clarity and reproducibility. \n\n- The clarity of the paper could be improved — in particular, Section 3.1 lacks a clear explanation of the loss function, and important terms such as Lmask are introduced without definition until much later (Section 5.1). \n\n- The notation used throughout Section 3 and beyond is often confusing, requiring multiple readings to go through Section 3 and later sections. \n\n- The overall idea of the architecture seems good. However, many important architectural details are missing. For example, the sequence encoder is not described in sufficient detail, making the method non-reproducible. Figure 3 does not help to fill in these details. \n\n- The definition of the binding site center in Section 4 is vague and should be clarified.\n\n- In Issue 1, the authors state that focusing on individual protein–ligand structures introduces statistical bias, but it does not expand much on this claim. It also does not explicitly describe how PDB-centric structures differ from UniSite data; this could be enhanced by describing the instances of each dataset.\n\n- Figure 2 attempts to describe a generalization of the fragmentation in past binding site prediction approaches. However, it is difficult to gain insight into how these pipelines work from the figure and the description in the introduction and Section 3.\n\n- The problems of widely used metrics are well explained. However, the introduced metric could be further described. Given a set of ground truth binding sites and predicted binding sites, it is not possible to compute the metric with the explanation given in the main manuscript.\n\n*Minor comments*\n\n- Line 58: The acronyms DCC and DCA are used without being previously defined. All acronyms should be introduced the first time they appear.\n\n- Line 113: SIFTS annotations are mentioned and referenced, but the acronym SIFTS is not described.\n\n- Section 3: The notation needs refinement. For instance, using the gt in $m^{gt}$ is never explained, and the explanation of this variable can be enhanced to better show that it is binary vector of size $L$. Improving the notation would help readers understand the content.\n\n- Figure 2 (Section 3): This figure does not add relevant insight into understanding the differences between existing methods and the proposed one.\n\n- Line 159: The symbol $\\sigma$ is defined as the bijective matching between sets $z$ and $z_pad^gt$, which could be easily confused with the sigmoid function. This makes it harder to interpret the loss function clearly.\n\n- Line 164: Lmask is referred to as \"binary mask loss\", but its formal definition doesn't appear until Section 5.1 (line 262). It should be mentioned earlier that Lmask combines binary cross-entropy and Dice loss.\n\n- In line 189, GearNet-Edge is mentioned, and details are provided even though no modifications were made. That’s fine, but I would prefer more implementation details for the sequence encoder instead.\n\n- Segmentation module (line 209): It is unclear whether the probabilities are obtained through the MLP. The figure suggests they are, but the text describes a linear classifier followed by a softmax. Additionally, for mask embeddings, the first mask goes through the MLP. This paragraph needs to be clarified.\n - Typo: Seems like it should be \"dot product\" instead of \"Dot-production.\"\n\n- Equations: None of the equations are numbered. They should be labeled sequentially as (Equation 1), (Equation 2), ..., (Equation N).\n\n- DCC/DCA: It is unclear whether this represents a division or something else. The notation should be clarified.\n\n- Lines 276 to 277 attempt to justify the absence of DCC and DCA in Table 1. This justification could be enhanced by means of a specific example. The clarification of the differences between past datasets and UniSite-DS may help in this regard.\n\n- Section 5.1: It is essential to add a justification for the baseline methods used for comparison.\n\n- Line 160: Sentence ends in \"by minimizing the following cost:\" to then show the \"result\" of the minimization $\\hat{\\sigma}$ rather than the loss.", "was_revised": true, "final_justification": "The authors have proposed a valuable new dataset (UniSite-DS) for binding site detection, as well as proposed new models for binding site detection (UniSite-1D and UniSite-3D) and a new metric to assess binding site predictions (AP of IoU).\n\nMy main issue with this manuscript is regarding the description of UniSite-1D and UniSite-3D, I thank the authors for the many clarifications provided. However, Section 3 needs to be rewritten in order to properly explain the architecture of the models to allow for reproducibility. My second concern is the lack of detail about the newly introduced metric. The authors have demonstrated the problems with past metrics very well and empirically shown how the new metric avoids those issues. My problem is that there is no description for the reader to understand what the metric is in the main paper. Finally, my reading of the motivations to build UniSite-DS is that a \"protein-centric\" dataset is the \"solution\" for the statistical bias present in past datasets. However, the response for the reviewers indicates that this statistical bias is a result of the lack of annotated binding sites.", "rebuttal": "We thank reviewer for the constructive comments. We provide our feedback as follows. We hope our answers have addressed your confusion, and we would greatly appreciate it if you could consider raising your rating.\n\n### **A1**\n>**Q1-1:** *Weakness 1: The clarity of the paper could be improved — in particular, Section 3.1 lacks a clear explanation of the loss function …*\n>\n>**Q1-2:** *Minor comment 6: Line 164: Lmask is referred to as \"binary mask loss\", but its formal definition doesn't appear until Section 5.1 …*\n\nThank you for your suggestion. We will move the implementation details about $L_{mask}$ to Section 3.1, line 164.\n\n\n### **A2**\n\n>**Q2-1:** *Weakness 3: The overall idea of the architecture seems good …*\n>\n>**Q2-2:** *Minor comment 7: In line 189, GearNet-Edge is mentioned …*\n\nGiven a protein sequence of length L, the input embedder receives three inputs: (1) the protein sequence represented as a 1D tensor of shape (L); (2) the residue indices as a 1D tensor of shape (L); and (3) the ESM2 embedding of protein sequences as a 2D tensor of shape (L,1280). The residue indices are encoded into sinusoidal positional embeddings of shape (L,32). The protein sequence is embedded using 21 learnable embeddings (20 standard amino acids plus an \"unknown\" category) to produce embeddings of shape (L,64). These three components - the ESM2 embeddings (L,1280), sinusoidal embeddings (L,32), and amino acid embeddings (L,64) - are then concatenated along the feature dimension, and subsequently processed by a 3-layer multilayer perceptron (MLP) to generate the per-residue features of shape (L,256).\n\nWe will add these implementation details of the sequence encoder in the Appendix.\n\n### **A3**\n>**Q3:** *Weakness 4: The definition of the binding site center in Section 4 is vague and should be clarified.*\n\nIn accordance with established methodologies, we employ the geometric center of the ligand as the ground truth binding site center for the DCC metric. And the predicted centers are outputs of each method itself.\n\n### **A4**\n>**Q4:** *Weakness 5: In Issue 1, the authors state that focusing on individual protein–ligand structures introduces statistical bias …*\n\nWe have provided a detailed specific example in Figure 1 of the manuscript to highlight the significant differences between the UniSite-DS and the classic PDB-centric dataset, PDBbind.\n\nSpecifically, for the protein Farnesyl diphosphate synthase (UniProt ID: Q8WS26), the upper-left portion of Figure 1 shows that in the PDBbind dataset, only one ligand binding site is recorded, contributed by PDB ID: 1YHM. In contrast, the upper-right portion of Figure 1 shows that UniSite-DS records 17 ligand binding sites for this protein, sourced from 13 different PDB structures. The protein portions of these PDB structures are highly similar, while their ligand binding sites are notably different. PDB-centric datasets overlook the different binding sites of the same protein across multiple PDB structures, missing a large number of ground truth sites.\n\n### **A5**\n>**Q5-1:** *Weakness 6: Figure 2 attempts to describe a generalization of the fragmentation in past …*\n>\n>**Q5-2:** *Minor comment 4: Figure 2 (Section 3): This figure does not add relevant insight …*\n\nFigure 2 (top) illustrates the discontinuous workflow of conventional learning-based binding site prediction methods. Take the baseline GrASP as a specific example: (1) **Predict Binary Mask.** GrASP first predicts a score for each heavy atom, representing \"the likelihood to be a part of a binding site\". The heavy atoms with scores below 0.3 are discarded. (2) **Clustering.** The remaining atoms are clustered into discrete binding sites through average linkage clustering, which makes overlap between predicted sites impossible. \n\nIn contrast, Figure 2 (bottom) demonstrates our UniSite approach, which directly generates N potentially overlapping binding sites in an end-to-end manner, as detailed in Sections 3.1 and 3.2.\n\n### **A6**\n>**Q6:** *Weakness 7: The problems of widely used metrics are well explained …*\n\nWe have added a detailed pseudocode to explain the calculation of the AP metric, which will be included in the Appendix of the revision. Due to the word limit in the NeurIPS response, please refer to the pseudocode in **our A2 response to Reviewer 4hDu**.\n\n### **A7**\n>**Q7:** *Minor comment 1: Line 58: The acronyms DCC and DCA are used without being previously defined. All acronyms should be introduced the first time they appear.*\n\nWe have already introduced the definitions of both DCC and DCA metrics between the end of line 58 and the beginning of line 61.\n\n### **A8**\n>**Q8:** *Minor comment 2: Line 113: SIFTS annotations …*\n\nSIFTS stands for *Structure Integration with Function, Taxonomy and Sequences resource*. \n\n### **A9**\n>**Q9:** *Minor comment 3: Section 3: The notation needs refinement. For instance, using the gt in $m^{gt}$ is never explained …*\n\nWe have provided a detailed explanation of $m_i^{gt}$ in lines 135-139. Each $m_i^{gt}$ represents a ground truth binding site, where ground truth is well known as gt, and $m_i^{gt}\\in\\lbrace 0,1\\rbrace^L$ is a binary vector of length $L$. We have further elaborated on the meaning of each bit in the vector in lines 137-139: \"Here, $m_{ij}^{gt}=1$ indicates that the $j$-th residue is part of the $i$-th site, while $m_{ij}^{gt}=0$ means it is not.\"\n\n### **A10**\n>**Q10:** *Minor comment 5: Line 159: The symbol $\\sigma$ is defined as the bijective matching …*\n\nThe use of $\\sigma$ to denote permutation or bijective matching is in fact a well-established convention in the literature (e.g., as seen in DETR and MaskFormer). If this notation does make you confused, we would use $\\tau$ rather than $\\sigma$.\n\n### **A11**\n>**Q11:** *Minor comment 8: Segmentation module (line 209) …*\n\nThe description in Section 3.2 (Segmentation Module) is indeed accurate. The N site queries are processed through a linear classifier followed by softmax activation to generate class probabilities, while the N mask embeddings are transformed by a multilayer perceptron (MLP). We will incorporate these corrections in Figure 2.\n\n### **A12**\n>**Q12:** *Minor comment 9: Typo: Seems like it should be \"dot product\" ...*\n\nThank you for your suggestion. We will correct this typo in the revision.\n\n### **A13**\n>**Q13:** *Minor comment 10: Equations: None of the equations are numbered …*\n\nWhile equation numbering is not required by NeurIPS and we don't cross-reference equations, we acknowledge its value for clarity and will add numbering in revision.\n\n### **A14**\n>**Q14:** *Minor comment 11: DCC/DCA: It is unclear whether this represents a division or something else. The notation should be clarified.* \n\nWe have provided detailed explanations of both DCC and DCA as distinct metrics in the manuscript. In this particular context, the “/” was intended to indicate their parallel or alternative applications (i.e., either metric could be used). To prevent any potential ambiguity, we will revise the expression to explicitly state \"DCC or DCA\" in the revision.\n\n### **A15**\n>**Q15:** *Minor comment 12: Lines 276 to 277 attempt to justify the absence of DCC and DCA in Table 1 …*\n\nAs stated in response A5, our dataset includes multiple PDB structure data for each protein entry, where binding sites are contributed by different structures and in different coordinate systems. Traditional DCC and DCA metrics rely on the ligand coordinates, so they are no longer applicable to Table 1.\n\n### **A16**\n>**Q16-1:** *Question 1: In which way can past datasets induce statistical bias? Do the experiments in the manuscript showcase how this bias is present?*\n>\n>**Q16-2:** *Question 3: In Table 3, do the baseline methods show variation in other evaluation metrics?*\n>\n>**Q16-3:** *Question 4: Why were the thresholds 0.5, 0.7, and 0.9 chosen for sequence similarity?*\n\nDue to the word limit in the NeurIPS response, regarding the statistical bias induced by past datasets and the experiments, please refer to Figure 1 of the manuscript, as well as **our A4 and A3 responses to Reviewer cfsU**. For the selection of similarity measures and the additional similarity experiments, please refer to **our A1 response to Reviewer 4hDu**.\n\n### **A17**\n>**Q17:** *Question2: Are the baseline methods also capable of predicting multiple binding sites?*\n\nYes, baseline methods can predict multiple binding sites.\n\n### **A18**\n>**Q18:** *Question 5: In line 210 … What is $\\Delta^2$ defined?*\n\nWe will correct this sentence to \"to generate class probabilities $\\lbrace p_i=(p_i^{site},p_i^{\\emptyset})|p_i^{site},p_i^{\\emptyset}\\in[0, 1]\\rbrace_{i=1}^N$\".\n\n### **A19**\n>**Q19:** *Minor comment 13: Section 5.1: It is essential to add a justification for the baseline methods used for comparison.*\n\nWe selected these baseline methods as they represent the spectrum of methodological approaches for ligand binding site detection. Fpocket represents traditional geometry-based computational algorithms. P2Rank is the most widely-used machine learning approach. DeepPocket, GrASP and VN-EGNN are all deep-learning-based methods. DeepPocket employs 3D convolutional neural networks while GrASP and VN-EGNN utilize graph neural networks. \n\nDue to the page limit of the main text, we currently illustrate and justify the baselines only in Table 1 and Section 5.2. For the revised version, we will add a dedicated appendix section to justify the baseline methods.\n\n### **A20**\n>**Q20:** *Minor comment 14 :Line 160: Sentence ends in \"by minimizing the following cost:\" …*\n\nWe will correct this sentence to \"This bipartite matching is obtained by minimizing matching cost $L_{match}$\".\n\n### **A21**\n> **Q21:** *Weakness 2: The notation used throughout Section 3 and beyond is often confusing, requiring multiple readings to go through Section 3 and later sections.*\n\nThank you for your feedback. We have followed your suggestions and made the improvements to the manuscript accordingly."}, {"review_id": "SqsFhWedCe", "reviewer": "Reviewer_cfsU", "summary": "In this paper, the authors introduce UniSite-DS, a new UniProt-centric protein ligand binding site dataset. They develop UniSite-1D/3D, methods for binding site detection based on this new dataset, and argue for extending the commonly used DCC and DCA metrics for evaluating binding site predictors to include Average Precision based on residue-level IoU. They identify failure modes where DCC and DCA incorrectly classify binding sites, where AP provides a better metric, and show that existing methods have sub-optimal performance on the new UniSite-DS benchmark.", "questions": "Can the authors show some example “case studies” of manual inspection and curation of the data? Making the process and the dataset itself easily inspectable could help the community judge the usefulness of the dataset artifact.\nThe failure cases in Fig 4 are helpful to see - how many such cases are there? How often does IoU AP disagree with DCA and DCC?\nAre there additional, simpler baselines that the authors could consider in addition to the proposed binding site detection method and the methods from literature? In particular to illustrate how a simple approach leveraging the UniSite-DS compares to UniSite-1D/3D", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\nThe focus on dataset curation and preparation, exemplified by the statement: “We manually inspected all UniProt IDs with more than ten ligand binding sites, as well as those where a single protein–ligand complex structure contributed three or more binding sites,” is a strong contribution and potentially very useful for the community.\nBinding site detection is an important and generally understudied problem, compared to binder design / hit discovery, which depends completely on accurate binding site prediction.\nThe illustrations of failure modes of the commonly used metrics and the need to calculate IoU-based AP to more comprehensively evaluate binding site detectors are clear and well-motivated.\n\nWeaknesses\nThe newly curated dataset is central to the work, so it seems to deserve its own figure (perhaps supplemental) describing the dataset curation process, quality control, and manual inspection.\nIn Fig 1 it is maybe unsurprising that UniSite-1D/3D performs best, considering it was trained on UniSite-DS. Results are more mixed for the other benchmark experiments.\nSection 5.4 discusses some limitations with respect to sequence similarity, but I did not see an explicit Limitations section with clearly laid out limitations and future work.", "was_revised": true, "final_justification": "The authors addressed each question and comment during the rebuttal period, and I have raised my score accordingly.", "rebuttal": "We thank reviewer for the constructive comments. We provide our feedback as follows. We hope our answers have addressed your concerns, and we would greatly appreciate it if you could consider raising your rating.\n\n\n\n### **A1**\n> **Q1:** *Weaknesses: The newly curated dataset is central to the work, so it seems to deserve its own figure (perhaps supplemental) describing the dataset curation process, quality control, and manual inspection.*\n\nWe have created an additional figure to illustrate the dataset curation, quality control, and manual inspection process in detail. Due to the official rebuttal constraints set by NeurIPS, we are unable to provide the figure directly at this stage. We would like to walk you through the structure and key elements of the figure as follows.\n\n1. **Curation Process**\n   * Systematic retrieval of all protein–ligand interactions from the PDB database\n   * Identification of binding site residues within 4.5 Å of each ligand\n   * Integration of all ligand binding sites across different PDB structures using UniProt identifiers and SIFTS annotations\n   * Redundancy removal using NMS with IoM ≥ 0.7 and IoU ≥ 0.5 to exclude highly overlapping binding sites\n   * Additional quality control and manual inspection to ensure high-quality dataset\n   \n2. **Quality Control**\n   * Exclude structures with resolution >2.5 Å or determined by non-crystallographic methods\n   * Remove solvent molecules\n   * Discard entries with ≤3 binding site residues to eliminate floating ligands\n   \n3. **Manual Inspection**\n   * Manual inspection of entries with >10 ligand binding sites or with ≥3 sites contributed by a single protein–ligand complex\n      * Entries involving supramolecular assemblies across multiple protein subunits\n      * Entries with large composite cavities jointly formed by multiple ligands, each occupying only part of the cavity\n      \n\nIf you have any further suggestions regarding the design or content of the figure, we would greatly appreciate it.\n\n\n\n### **A2**\n\n> **Q2:**  *Questions: Can the authors show some example “case studies” of manual inspection and curation of the data? Making the process and the dataset itself easily inspectable could help the community judge the usefulness of the dataset artifact.*\n\n**Manual Inspection Case Studies**\n\n1. **Case 1 (Entries involving supramolecular assemblies across multiple protein subunits)**\n   * The Photosystem I-LHCI Supercomplex is a core component of the photosynthetic machinery, consisting of multiple subunits such as UniProt ID: P05310 (Photosystem I P700 chlorophyll a apoprotein A1, PDB: 7dkz_A), Q41038 (Chlorophyll a-b binding protein, 7dkz_B), and Q32904 (Chlorophyll a-b binding protein 3, 7dkz_C).\n   * Although each of these proteins binds a large number of Chlorophyll a molecules, they function as part of a tightly integrated supramolecular assembly. Therefore, they are not suitable to be included as independent entries in the UniSite-DS database and have been excluded in the current version.\n2. **Case 2 (Entries with large composite cavities jointly formed by multiple ligands, each occupying only part of the cavity)**\n   * Trypanothione reductase (UniProt ID: Q389T8) is a key enzyme in *Trypanosoma brucei* that specifically catalyzes the reduction of trypanothione, functionally analogous to glutathione reductase in mammals. The ligands from structures PDB: 5s9x_A, 5s9t_A, and 2wov_C together form a large binding cavity. These ligand binding sites could potentially be merged.\n   * However, since each ligand actually occupies only a portion of the cavity rather than the entire cavity, whether such a composite cavity formed by different ligands should be considered a unified binding site often depends on system-specific definitions in the literature. As a result, entries that require further literature-based validation were excluded from the current version of the UniSite-DS database.\n\n\n\n### **A3**\n\n> **Q3:**  *Questions: The failure cases in Fig 4 are helpful to see - how many such cases are there? How often does IoU AP disagree with DCA and DCC?*\n\nAs we have discussed in Section 4 (*Rethinking the Evaluation Metrics for Binding Site Detection*, lines 217-245), DCC and DCA suffer from two critical limitations:\n\n   * **Limitation 1. Predictions may be double-counted due to the absence of proper matching criteria (Fig 4A).** We quantified the proportion of proteins affected by double counting during evaluation on the widely-used HOLO4K benchmark.\n\n**Table. Double Counting (DC) Rate in Metrics in HOLO4K-sc.**\n\n|Method|DC of $\\text{DCC}_{\\text{top-}n}$|DC of $\\text{DCA}_{\\text{top-}n}$|\n|-|-|-|\n|Fpocket|18.80%|18.31%|\n|Fpocket-rescore|13.23%|13.66%|\n|P2Rank|18.98%|18.31%|\n|DeepPocket|13.53%|14.21%|\n|GrASP|16.29%|14.88%|\n|VN-EGNN|12.80%|11.70%|\n|**UniSite-1D (ours)**|**8.88%**|**8.94%**|\n|**UniSite-3D (ours)**|**9.86%**|**10.04%**|\n\nThe results reveal that DCC and DCA metrics suffer from widespread double counting artifacts, which significantly distort model performance assessment.\n\n   * **Limitation 2.** **DCC/DCA only evaluate the center of binding sites and are ligand-dependent, which leads to evaluation failures in certain scenarios (Fig 4B-D).**\n     * For HOLO4K, we calculated the DCC and DCA metrics of the centroid of ground truth binding residues for each protein. The results show that: **the mean ground truth DCC is 2.15 Å (92.65% < 4 Å), and the mean ground truth DCA is 1.57 Å (98.88% < 4 Å).** However, in principle, both DCC and DCA should ideally be 0 when evaluated using ground truth binding residues, indicating these metrics inherently contain systematic bias.\n     * To address some of DCC's inherent limitations, we defined a corrected metric, DCC-residue, which uses the center of ground truth binding residues rather than the ligand's center for calculation. This modification resolves failure cases caused by ligand diversity in traditional DCC evaluation.\n\n|Method|$\\text{AP}_{0.3}$↑|$\\text{DCC-residue}_{\\text{top-}n}$↑|$\\text{DCC}_{\\text{top-}n}$↑|\n|-|-|-|-|\n|Fpocket|0.2711|0.2982|0.3076|\n|Fpocket-rescore|0.5899|0.5005|0.5183|\n|P2Rank|0.6011|0.4972|0.5300|\n|DeepPocket|0.5415|0.4902|0.4925|\n|GrASP|0.6668|0.5379|0.5131|\n|VN-EGNN|0.2606|0.5997|**0.5861**|\n|**UniSite-1D (ours)**|**0.6867**|**0.6400**|0.5538|\n|**UniSite-3D (ours)**|**0.7091**|**0.6264**|**0.5716**|\n\nOur evaluation on the HOLO4K benchmark demonstrates that the corrected DCC-residue metric shows improved consistency with IoU-based AP in ranking different methods' performance.\n\n\n\n### **A4**\n\n> **Q4:**  *Weaknesses: In Fig 1 it is maybe unsurprising that UniSite-1D/3D performs best, considering it was trained on UniSite-DS. Results are more mixed for the other benchmark experiments.*\n\n1. **Existing benchmark datasets suffer from significant annotation omissions.** These data quality issues greatly affect the reliable evaluation of ligand binding site detection performance, as they overlook numerous ground truth binding sites. \n   *  A representative example is UniProt ID: P28482 (MAPK1), a protein critically involved in cell proliferation, differentiation, survival, and apoptosis. The HOLO4K dataset annotates only one ligand binding site for this protein (derived from PDB: 1pmeA, SB2 binding site). In contrast, UniSite-DS documents 15 ligand binding sites (from PDB entries: 6qa1_A, 6qal_A, 4fv7_A, 6gjd_A, etc.)\n\n2. As described in A3, **traditional DCC and DCA metrics have significant limitations and cannot faithfully reflect the true performance of different methods.** This is exactly the reason why we propose IoU-based AP as a more fair evaluation metric.\n3. **In fact, UniSite demonstrates significantly superior performance when assessed using either the IoU-based AP metric or the corrected DCC-residue measure.**\n\n\n\n### **A5**\n\n> **Q5:**  *Weaknesses: Section 5.4 discusses some limitations with respect to sequence similarity, but I did not see an explicit Limitations section with clearly laid out limitations and future work.*\n\nWe will add a *Limitations and Future Work* section as follows:\n\n1. **Dataset**: Currently, we manually inspect and remove unreasonable data entries. Future work could consider further repairing and reintegrating the excluded data into the dataset.\n2. **Model**: Our current model design aims to demonstrate the effectiveness of the End-to-End ligand binding site learning framework, without incorporating specialized feature engineering. Future work could explore the inclusion of specialized feature engineering to further improve model performance and generalization ability.\n\n\n\n### **A6**\n\n> **Q6:**  *Questions: Are there additional, simpler baselines that the authors could consider in addition to the proposed binding site detection method and the methods from literature? In particular to illustrate how a simple approach leveraging the UniSite-DS compares to UniSite-1D/3D*\n\nIn fact, the baseline methods cannot be re-trained on the UniSite-DS without significant architectural modifications.\n\n* Fpocket is a purely geometric algorithm without trainable parameters.\n* Learning-based methods like P2Rank and GrASP are supervised to predict one binary mask. This  schema is incompatible with UniSite-DS annotations, which contain multiple potentially overlapping ground truth binding site masks.\n\nThe transition from a discontinuous, post-processing-based workflow to an end-to-end learning framework for ligand binding site detection constitutes one of our key contributions."}, {"review_id": "RiQMFpx096", "reviewer": "Reviewer_fcYt", "summary": "This paper introduces an end-to-end discontinuous framework UniSite for protein ligand binding site detection. Besides, they construct and present UniSite-DS, a novel dataset that is \"UniProt-centric\" rather than \"PDB-centric\". For evaluation, beyond traditional evaluation metrics DCC/DCA, the authors advocate for using Average Precision (AP) based on Intersection over Union (IoU) of binding site residues. Experiments are conducted on HOLO4K, COACH420, and UniSite-DS.", "questions": "**Suggestions/Questions:**\n\n1. Providing a comparsion with baselines on proteins with low similarity.\n2. The definition of pocket residue of different baselines should be detailed in the paper for better understanding.\n3. Providing a detailed ablation study of the key module, such as ESM embedding,", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n1. The paper is generally well-written, clearly structured, and easy to follow\n2. The proposed UniSite model is an interesting end-to-end deep learning architectures to the binding site detection problem.\n3. The conceptualization and construction of the UniSite-DS dataset is an interesting contribution. The shift from a PDB-centric to a UniProt-centric view is a well-motivated and important step forward for the field.\n\n**Weaknesses**\n1. The central argument for UniSite's superiority rests heavily on the new IoU-based AP metric. However, this evaluation framework may be inherently biased towards the proposed method. UniSite is explicitly trained to predict residue-level masks, optimizing a loss function that includes IoU-like components (Dice loss). In contrast, many baseline methods were not designed for this specific output format. The authors' decision to define its binding sites as residues within a 9Å radius is an ad-hoc post-processing step, and the paper notes this radius was chosen because it yielded the best AP performance (Appendix E). This constitutes tuning the baseline for the new metric, and it is unclear if this is a fair or optimal representation for that method. Consequently, the superior performance of UniSite on the AP metric might reflect an alignment between the model's architecture and the evaluation metric, rather than a fundamental superiority in identifying binding sites.\n2. When evaluated on traditional, widely-accepted metrics like DCC and DCA, the performance of UniSite is less convincing, as shown in Table 2.\n3. Result in Table 3 reveals a significant performance drop as the sequence similarity between the training and test sets decreases. This suggests a concern about overfitting on the training dataset.  To make a convincing case for its robustness, the paper needs to show how this performance degradation compares to that of the baseline methods under the same conditions. It is possible that simpler, feature-based methods like P2Rank are more robust to such distributional shifts.", "was_revised": true, "final_justification": "All of my concerns have been solved.", "rebuttal": "We thank reviewer for the constructive comments. We provide our feedback as follows. We hope our answers have addressed your concerns, and we would greatly appreciate it if you could consider raising your rating.\n\n### **A1**\n\n> **Q1:** *Weaknesses 1: The central argument for UniSite's superiority rests heavily on the new IoU-based AP metric … rather than a fundamental superiority in identifying binding sites.*\n\n1. **Binding site residues is the standard output for ligand binding site detection task and is critical for downstream applications.** Binding site residues are the primary output of most baselines (details are provided in **A4**). In Appendix A, we have discussed the impact for docking tasks, **and reviewer cfsU also strongly agrees with this perspective**: \n\n   > **reviewer cfsU:** *Binding site detection is an important and generally understudied problem, compared to binder design/hit discovery, which relies entirely on accurate binding site prediction.*\n   \n2. **IoU-based AP is a widely accepted and fair metric for detection tasks like Object Detection.**\n\n3. **Most learning-based baselines are supervised by mask loss.** As discussed in Section 3, most learning-based methods first predict a site score for each residue/atom, and this process is commonly supervised by a mask loss: P2Rank, DeepPocket and GrASP utilize BCE loss and VN-EGNN employs dice loss. To align with the training objective of baselines, we calculate the **Semantic-IoU** metric: On the HOLO4K benchmark, we first merge all ground truth binding sites. For each method, we select the score corresponding to Recall=0.7 at an IoU threshold of 0.3 as the cutoff threshold and aggregate its predictions accordingly. Finally, we compute the IoU between the merged predicted mask and the merged ground truth mask. \n|Method|$\\text{AP}_{0.3}$↑|$\\text{AP}_{0.5}$↑|Semantic-IoU↑|\n|-|-|-|-|\n|Fpocket|0.2711|0.1488|0.1799|\n|Fpocket-rescore|0.5899|0.2847|0.4732|\n|P2Rank|0.6011|0.2625|0.4576|\n|DeepPocket|0.5415|0.2891|0.4368|\n|GrASP|0.6668|0.4126|**0.5536**|\n|VN-EGNN|0.2606|0.1346|0.4344|\n|**UniSite-1D (ours)**|**0.6867**|**0.4595**|0.5360|\n|**UniSite-3D (ours)**|**0.7091**|**0.5446**|**0.5876**|\n\nAs shown above, **our method demonstrates superiority even under this baseline-training-loss-aligned evaluation framework.**\n\n4. **The transition from a discontinuous workflow to an end-to-end architecture for ligand binding site detection constitutes one of our key contributions.**\n\n### **A2**\n\n> **Q2:** *Weakness 2: When evaluated on traditional, widely-accepted metrics like DCC and DCA, the performance of UniSite is less convincing, as shown in Table 2.*\n\n1. **While UniSite shows slightly inferior performance on traditional metrics DCC and DCA, this primarily stems from the fundamental flaws in these metrics** (see Section 4). Reviewer cfsU agreed with the flaws of DCC and DCA: \n\n   > **reviewer cfsU:** *The illustrations of failure modes in commonly used metrics and the proposal to adopt IoU-based AP for more comprehensive evaluation are clearly presented and well-justified. The failure cases in Fig 4 provide particularly compelling evidence.*\n\nSpecifically, DCC and DCA suffer from two critical limitations:\n\n   * **Limitation 1. Predictions may be double-counted due to the absence of proper matching criteria (Fig 4A).** We quantified the proportion of proteins affected by double counting during evaluation on the widely-used HOLO4K benchmark.\n\n**Table. Double Counting (DC) Rate in Metrics in HOLO4K-sc.**\n\n|Method|DC of $\\text{DCC}_{\\text{top-}n}$|DC of $\\text{DCA}_{\\text{top-}n}$|\n|-----------------|-----------------------|-----------------------|\n|Fpocket|18.80%|18.31%|\n|Fpocket-rescore|13.23%|13.66%|\n|P2Rank|18.98%|18.31%|\n|DeepPocket|13.53%|14.21%|\n|GrASP|16.29%|14.88%|\n|VN-EGNN|12.80%|11.70%|\n|**UniSite-1D (ours)**|**8.88%**|**8.94%**|\n|**UniSite-3D (ours)**|**9.86%**|**10.04%**|\n\nThe results reveal that DCC and DCA metrics suffer from widespread double counting artifacts, which significantly distort model performance assessment.\n\n   * **Limitation 2.** **DCC/DCA only evaluate the center of binding sites and are ligand-dependent, which leads to evaluation failures in certain scenarios (Fig 4B-D).**\n      * For HOLO4K, we calculated the DCC and DCA metrics of the centroid of ground truth binding residues for each protein. The results show that: **the mean ground truth DCC is 2.15 Å (92.65% < 4 Å), and the mean ground truth DCA is 1.57 Å (98.88% < 4 Å).** However, in principle, both DCC and DCA should ideally be 0 when evaluated using ground truth binding residues, indicating these metrics inherently contain systematic bias.\n      * To address some of DCC's inherent limitations, we defined a corrected metric, DCC-residue, which uses the center of ground truth binding residues rather than the ligand's center for calculation. This modification resolves failure cases caused by ligand diversity in traditional DCC evaluation.\n\n|Method|$\\text{AP}_{0.3}$↑|$\\text{DCC-residue}_{\\text{top-}n}$↑|$\\text{DCC}_{\\text{top-}n}$↑|\n|-|-|-|-|\n|Fpocket|0.2711|0.2982|0.3076|\n|Fpocket-rescore|0.5899|0.5005|0.5183|\n|P2Rank|0.6011|0.4972|0.5300|\n|DeepPocket|0.5415|0.4902|0.4925|\n|GrASP|0.6668|0.5379|0.5131|\n|VN-EGNN|0.2606|0.5997|**0.5861**|\n|**UniSite-1D (ours)**|**0.6867**|**0.6400**|0.5538|\n|**UniSite-3D (ours)**|**0.7091**|**0.6264**|**0.5716**|\n\nOur evaluation on the HOLO4K benchmark demonstrates that the corrected DCC-residue metric shows improved consistency with IoU-based AP in ranking different methods' performance.\n\n2. **Existing benchmark datasets suffer from significant annotation omissions.** These data quality issues greatly affect the reliable evaluation of ligand binding site detection performance, as they overlook numerous ground truth binding sites. \n   *  A representative example is UniProt ID: P28482 (MAPK1), a protein critically involved in cell proliferation, differentiation, survival, and apoptosis. The HOLO4K dataset annotates only one ligand binding site for this protein (derived from PDB: 1pmeA, SB2 binding site). In contrast, UniSite-DS documents 15 ligand binding sites (from PDB entries: 6qa1_A, 6qal_A, 4fv7_A, 6gjd_A, etc.)\n   \n3. **Both traditional metrics and benchmark datasets exhibit substantial limitations, failing to accurately reflect the true performance of different methods. In fact, UniSite demonstrates significantly superior performance when assessed using either the IoU-based AP metric or the corrected DCC-residue measure.**\n\n### **A3**\n\n> **Q3-1:** *Weakness 3: Result in Table 3 reveals a significant performance drop as the sequence similarity between the training and test sets decreases … feature-based methods like P2Rank are more robust to such distributional shifts.*\n>\n> **Q3-2:** *Question 1: Providing a comparsion with baselines on proteins with low similarity.*\n\nPrevious traditional machine learning methods (such as P2Rank) did not account for robustness testing with sequence similarity splits. To ensure a fair comparison of robustness across different methods, and considering that all methods use HOLO4K as the benchmark testset, we divided the subsets based on different sequence similarity thresholds (0.9, 0.7, 0.5, and 0.3) using the HOLO4K dataset in conjunction with each method's training set.\n\nWe compared the performance at different similarity levels of the classic machine learning methods Fpocket-rescore and P2Rank, the deep learning-enhanced version of P2Rank (GrASP), as well as our methods, UniSite-1D and UniSite-3D, in terms of $\\text{AP}_{0.3}$. The results are as follows:\n\n* Similarity: <1.0 → <0.9 → <0.7 → <0.5 → <0.3\n* **Fpocket-rescore**: 0.5900 → 0.5911 → 0.5925 → 0.5867 → 0.5670\n* **P2Rank**: 0.6011 → 0.5982 → 0.6211 → 0.5987 → 0.5734\n* **GrASP**: 0.6668 → 0.6490 → 0.6729 → 0.6388 → 0.6040\n* **UniSite-1D (ours)**: 0.6867 → 0.6868 → 0.6817 → 0.6156 → 0.5744\n* **UniSite-3D (ours)**: 0.7090 → 0.7053 → 0.7177 → 0.6580 → 0.6534\n\nThe results show that, compared to traditional machine learning methods, deep learning approaches such as UniSite-1D/3D and GrASP experience a performance decline when dealing with proteins of very low sequence similarity. This could be due to deep learning methods' greater sensitivity to the distribution of the training data.\n\n### **A4**\n\n> **Q4:** *Question 2: The definition of pocket residue of different baselines should be detailed in the paper for better understanding.*\n\nIn fact, pocket (binding site) residues are the primary output of most baselines:\n\n(1) Fpocket outputs a \"pocket{index}_atm.pdb\" for each predicted binding site, which records the atoms of binding site in PDB format.\n\n(2) P2Rank generates a \"{name}.pdb_prediction.csv\" file for each protein. The \"residue_ids\" column contains the chain IDs and the residue IDs of binding site residues.\n\n(3) DeepPocket rescores and refines the Fpocket results and keeps the same output format.\n\n(4) GrASP predicts a \"{name}_probs.pdb\" file for each protein, in which the bfactor column contains the predicted scores of the heavy atoms. We filtered and clustered these atoms into different sites following the method in original paper.\n\nIn contrast, VN-EGNN outputs a \"prediction.csv\" file for each protein, which only contains four columns: \"x\", \"y\", \"z\" and \"rank\".\n\n### **A5**\n\n> **Q5:** *Question3: Providing a detailed ablation study of the key module, such as ESM embedding.*\n\nWe conducted an ablation study on the ESM embedding, and the results show that for UniSite-3D, removing the ESM embedding leads to a decrease of approximately 0.03 in the model's AP performance. For UniSite-1D, since it does not incorporate any structural information, removing the ESM embedding from the sequence encoder causes the training to fail to converge.\n\n||$\\text{AP}_{0.3}$↑|$\\text{AP}_{0.5}$↑|\n|------------------|----------|----------|\n|UniSite-3D w/o ESM|0.5359|0.3501|\n|UniSite-3D w/ ESM|0.5603|0.3835|"}, {"review_id": "aa3k4J1VmB", "reviewer": "Reviewer_4hDu", "summary": "This paper introduces novel deep learning approaches, new dataset and benchmark and new evaluation metric for the pocket detection task. The authors recognized that only limited pockets are occupied in most PDB data, and then build a new dataset by aggregating different PDB structures of the same protein. Moreover, by introducing previously well-established techniques in object detection, this paper addresses several existing issues of the evaluation of pocket detection methods, and achieved strong performance with an advanced network architecture.", "questions": "1.\tThe generalization of the model:\nHow good is the model for low homology proteins or even proteins with different fold? P2Rank and Fpocket rescore seems to be a strong baseline with classic ML approaches. How about their performance on low homology test samples?\n2.\tRecall vs precision:\nUnlike the object detection task, personally speaking, recall is more important for pocket detection to locate new biological meaningful pockets. Given a certain IoU threshold, for example 0.9 0.6 and 0.3, how many pockets can be recalled?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1.\tThe paper has provided a useful and valuable new dataset for the pocket detection task, addressing the multi-pocket problem.\n2.\tThe paper has introducing well-established metrics and deep learning approaches of object detection to pocket detection, achieving strong performance.\n\nWeaknesses:\n\n1.\tGeneralization of the model: Although the authors have tested their method on sequence-similarity dataset splits, they should include more strict ones. For example, 30% similarity cutoff or protein family split;\n2.\tThe definition of the AP metric: As the AP metric is newly introduced to the field of pocket detection, the paper should have a formulation or pseudo code to clearly explain the calculation of AP, but not just describe it with natural languages.", "was_revised": true, "final_justification": "This paper makes a valuable contribution to the field of pocket detection by introducing a new benchmark dataset that addresses the multi-pocket problem, a carefully adapted evaluation metric (AP), and a high-performing deep learning architecture. The authors thoughtfully incorporate object detection paradigms into protein structure analysis and provide strong empirical results.\n\nAfter reviewing the rebuttal and follow-up discussions, I find that the authors have satisfactorily addressed my primary concerns:\n\nOn model generalization, the authors provided additional evaluations across varying sequence similarity cutoffs, including the challenging <30% category. Although performance does decline under these strict conditions—highlighting a broader limitation of deep learning models in biology—the authors are transparent about this issue and provide context consistent with other leading works in the field.\n\nOn AP metric clarity, the inclusion of detailed pseudocode significantly improves the reproducibility and interpretability of the evaluation pipeline, especially given the metric’s novelty in this domain.\n\nOn recall vs precision, the detailed breakdown across IoU thresholds and the rationale for balancing recall with ranking precision is compelling. The strong recall of UniSite-3D across all thresholds further strengthens the case for its practical utility.\n\nOverall, this is a technically sound and impactful paper with meaningful improvements in both methodology and benchmarking. The work sets a new standard for pocket detection evaluation and opens promising directions for future research. I maintain my positive evaluation.", "rebuttal": "We thank reviewer for the constructive comments. We provide our feedback as follows. We hope our answers have addressed your questions, and we would greatly appreciate it if you could consider raising your rating.\n\n\n\n### **A1**\n\n> **Q1-1:** *Weaknesses 1: Generalization of the model: Although the authors have tested their method on sequence-similarity dataset splits, they should include more strict ones. For example, 30% similarity cutoff or protein family split;*\n>\n> **Q1-2:** *Questions 1: The generalization of the model: How good is the model for low homology proteins or even proteins with different fold? P2Rank and Fpocket rescore seems to be a strong baseline with classic ML approaches. How about their performance on low homology test samples?*\n\nThank you very much for your suggestion! The reason our initial manuscript did not include the 30% similarity experiment is that we aimed to keep the sizes of the training and testing sets relatively consistent across different similarity splits. However, the 30% similarity split does not yield enough samples for the test set. Moreover, thresholds of 0.5, 0.7, and 0.9 for sequence similarity are commonly used in protein-related tasks to assess generalization performance.\n\nPrevious traditional machine learning methods (such as P2Rank) did not account for generalization testing with sequence similarity splits. To ensure a fair comparison of generalization across different methods, and considering that all methods use HOLO4K as the benchmark testset, we divided the subsets based on different sequence similarity thresholds (0.9, 0.7, 0.5, and 0.3) using the HOLO4K dataset in conjunction with each method's training set.\n\nWe compared the performance at different similarity levels of the classic machine learning methods Fpocket-rescore and P2Rank, the deep learning-enhanced version of P2Rank (GrASP), as well as our methods UniSite-1D and UniSite-3D, in terms of $\\text{AP}_{0.3}$. The results are as follows:\n\n* Similarity: <1.0 → <0.9 → <0.7 → <0.5 → <0.3\n* **Fpocket-rescore**: 0.5900 → 0.5911 → 0.5925 → 0.5867 → 0.5670\n* **P2Rank**: 0.6011 → 0.5982 → 0.6211 → 0.5987 → 0.5734\n* **GrASP**: 0.6668 → 0.6490 → 0.6729 → 0.6388 → 0.6040\n* **UniSite-1D (ours)**: 0.6867 → 0.6868 → 0.6817 → 0.6156 → 0.5744\n* **UniSite-3D (ours)**: 0.7090 → 0.7053 → 0.7177 → 0.6580 → 0.6534\n\nThe results show that, compared to traditional machine learning methods, deep learning approaches such as UniSite-1D, UniSite-3D, and GrASP experience a performance decline when dealing with proteins of very low sequence similarity. This could be due to deep learning methods' greater sensitivity to the distribution of the training data.\n\n\n### **A2**\n> **Q2:** *Weaknesses 2: The definition of the AP metric: As the AP metric is newly introduced to the field of pocket detection, the paper should have a formulation or pseudo code to clearly explain the calculation of AP, but not just describe it with natural languages.*\n\nThank you very much for your suggestion! We have added a detailed pseudocode to explain the calculation of the AP metric, which will be included in the Appendix of the revision. The pseudocode is as follows:\n\n---\n\n**Function** calculate_AveragePrecision(prediction_list, ground_truth_list, iou_threshold)\n\n**Inputs:** \n\n​\t`prediction_list`: A list, where each element represents the predictions for one protein. Each element is of the form $\\lbrace (s_i, m_i)|s_i\\in \\mathbb{R} ,m_i\\in\\lbrace 0,1\\rbrace^L\\rbrace_{i=1}^{N}$, where $m_i$ represents the $i$-th predicted binding site as a binary mask of length $L$, and $s_i$ denotes the $i$-th confidence score.\n\n​\t`ground_truth_list`: A list, where each element represents the ground truth (gt) binding sites for one protein. Each element is of the form $\\lbrace m_i^{gt}|m_i^{gt}\\in\\lbrace 0,1\\rbrace ^L\\rbrace_{i=1}^{N_{gt}}$, where $m_i^{gt}$ represents the $i$-th ground truth binding site as a binary mask of length $L$.\n\n​\t`iou_threshold`: IoU threshold for considering a prediction as True Positive.\n\n**Output:**\n\n​\t`AP`: Average Precision under the given IoU threshold.\n\n---\n\n **\\# Step1: determine TP (True Positive) or FP (False Positive) for each prediction**\n\n**for** `predictions_per_protein`, `ground_truths_per_protein` **in** zip(`predicton_list`, `ground_truth_list`):  \n\n——Set all ground truths in `ground_truths_per_protein` **unused**\n\n——Sort `predictions_per_protein`$=\\lbrace (s_i, m_i)|s_i\\in \\mathbb{R} ,m_i\\in\\lbrace 0,1\\rbrace^L\\rbrace_{i=1}^{N}$ by decreasing confidence score $s_i$\n\n——**for** $i=1,2,\\cdots,N$\n\n———— Find `ground_truth_j` that has the max residue-level IoU, IoU($m_j^{gt}$, $m_i$), with `prediction_i`\n\n———— **if** IoU($m_j^{gt}$, $m_i$) > `iou_threshold` and  `ground_truth_j` is **unused**:\n\n—————— Mark `prediction_i` as TP\n\n—————— Set `ground_truth_j` as **used**\n\n————**else**:\n\n—————— Mark `prediction_i` as FP\n\n **\\# Step 2: Sort all predictions across all proteins by decreasing confidence scores**\n\n`all_predictions` = FLATTEN_ALL(`prediction_list`) \n\n`all_ground_truths` = FLATTEN_ALL(`ground_truth_list`) \n\nSort `all_predictions` by decreasing confidence scores\n\n **\\# Step 3: calculate the precision-recall curve**\n\nSet `cum_TP`, `cum_FP` as 0, `precision_list` and `recall_list` as empty\n\n**for** `prediction_i` **in** `all_predictions`:\n\n—— **if**  `prediction_i` is marked as TP:\n\n———— `cum_TP` += 1\n\n——**else**:\n\n————`cum_FP` += 1\n\n—— `precision_list`.append(`cum_TP` / (`cum_TP` + `cum_FP`))\n\n—— `recall_list`.append(`cum_TP` / LEN(`all_ground_truths`))\n\n**\\# Step 4**\n\nCalculate `AP` as the area under the precision-recall curve\n\n**return** `AP`\n\n\n### **A3**\n> **Q3:** *Questions 2: Recall vs precision: Unlike the object detection task, personally speaking, recall is more important for pocket detection to locate new biological meaningful pockets. Given a certain IoU threshold, for example 0.9 0.6 and 0.3, how many pockets can be recalled?*\n\nThank you very much for your suggestion! We calculated the Recall of different methods at various IoU thresholds on the widely used HOLO4K benchmark dataset, and the results are as follows:\n\n|Method|$\\text{AP}_{0.3}$↑|$\\text{AP}_{0.5}$↑|$\\text{Recall}_{0.3}$↑|$\\text{Recall}_{0.5}$↑|$\\text{Recall}_{0.7}$↑|$\\text{Recall}_{0.9}$↑|\n|-|-|-|-|-|-|-|\n|Fpocket|0.2711|0.1488|**0.8361**|0.5922|0.2130|0.0253|\n|Fpocket-rescore|0.5899|0.2847|0.8361|0.5922|0.2130|0.0253|\n|P2Rank|0.6011|0.2625|0.7814|0.5337|0.1868|0.0089|\n|DeepPocket|0.5415|0.2891|0.7514|0.5824|0.2584|0.022|\n|GrASP|0.6668|0.4126|0.7186|0.5374|0.2537|0.0159|\n|VN-EGNN|0.2606|0.1346|0.7289|0.4874|0.0566|0|\n|**UniSite-1D (ours)**|**0.6867**|**0.4595**|0.8212|**0.6199**|**0.3535**|**0.0824**|\n|**UniSite-3D (ours)**|**0.7091**|**0.5446**|**0.8469**|**0.6901**|**0.4106**|**0.1039**|\n\nThe results indicate that:\n\n* **UniSite-3D achieves state-of-the-art (SOTA) Recall across all IoU thresholds.**\n* Notably, the high Recall score of Fpocket is due to its tendency to output nearly all potential cavities in a protein, rather than accurately identifying the useful pockets. **In fact, Fpocket gives lower scores for true pockets, making it difficult for biologists to identify meaningful pockets.** This is also the reason why many researchers have tried to develop rescore methods for Fpocket.\n* While Recall indicates a method’s potential to identify new biologically meaningful pockets, it doesn’t take into account the score of each predicted pocket. **The AP metric addresses this issue, and therefore, we use it as a fair metric to evaluate different methods.**"}]

Author rebuttal:

Meta-review: This is an exciting and impactful paper that tackles binding site prediction, an important but less studied problem compared to binder design. Existing works typically focus on the single binding site case, ignoring that multiple binding sites may exist across multiple complexes of the same protein.

I, as well as the reviewers, appreciate how the authors took a comprehensive approach to this problem from the perspectives of dataset construction, modeling, and evaluation. The main contributions are:
-A novel UniProt centric dataset that incorporates multiple binding sites for a given sequence. 
-A model (UniSite) that proposes an end-to-end binding detection framework compared to conventional stage-wise pipelines.
-An Average Precision based on Intersection over Union (IoU) metric that address shortcomings of existing evaluation metrics for this problem.

The authors have also worked to thoroughly answer reviewer comments and thus I recommend acceptance of this paper.
Decision: Accept (spotlight)