SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Training and Evaluation. The training process consists of two stages. In the initial warm-up stage, only the native pockets are used for docking. Once the pocket prediction performance reaches a certain threshold (specifically, when the center coordinate distance between the predicted center and ground truth is less than  $4\mathring{\mathrm{A}}$ ), the training progresses to the second stage. During the second stage, the predicted pockets are integrated into the docking training process. The sampling protocol involves a  $75\%$  probability of selecting the ground truth pocket and a  $25\%$  probability of selecting the predicted pocket. Note that the task of pocket prediction is consistently incorporated into the entire training process. Following E3Bind [51], we also apply normalization (divided by 5) and unnormalization (multiplied by 5) techniques on the coordinate and distance. Additionally, to improve the generalization ability of the model, the pocket is randomly shifted from  $-5\mathring{\mathrm{A}}$  to  $5\mathring{\mathrm{A}}$  in all three axes during training. FABind models are trained for approximately 500 epochs using the AdamW [30] optimizer on 8 NVIDIA V100 16GB GPU with batch size set to 3 on each GPU. The learning rate is  $5e - 5$ , which is scheduled to warm up in the first 15 epochs and then decay linearly. To further enforce geometric constraints, we also incorporate local atomic structures (LAS) constraints in the training process by ensuring the distances between ligand atoms  $i$  and  $k$  in the transformed conformer (X) by the model are consistent with those in the initial low-energy conformer (Z) for atoms that are either 2-hop neighbors or in the same ring structure, as proposed in EquiBind [41].
A.3 Ligand and Protein Feature Encoding
As we stated in paper Section 3.1, we construct ligand feature by TorchDrug [52] toolkit and protein feature with the pre-trained ESM-2 model. Here we give a detailed description of the encoding. For ligand compound, the node embedding  $\mathbf{h}_i$  is a 56-dimensional vector containing the following features: atom number; degree; the number of connected hydrogens; total valence; formal charge;
4https://github.com/gcorso/DiffDock
Table 5: PDBBind blind docking on apo proteins. The top half contains results from traditional docking software; the bottom half contains results from recent deep learning based docking methods. The last line shows the results of our FABind. No method received further training on ESMFold generated structures.
Method	Apo ESMFold proteins Top-1 RMSD
%<2	Med.
GNINA	2.0	22.3
SMINA	3.4	15.4
EQUIBIND	1.7	7.1
TANKBIND	10.4	5.4
P2RANK+SMINA	4.6	10.0
P2RANK+GNINA	8.6	11.2
EQUIBIND+SMINA	4.3	8.3
EQUIBIND+GNINA	10.2	8.8
DIFFDOCK (10)	21.7	5.0
DIFFDOCK (40)	20.3	5.1
FABIND	24.9	4.2
whether or not it is in an aromatic ring. For protein target, we directly use the pre-trained 33-layer ESM-2 [28] model<sup>5</sup>, which contains 650M parameters and is trained on UniRef 50M dataset. The node feature in the protein graph is derived from the amino acid feature and the hidden size is 1280.
A.4 Model Architecture Details
Edge Construction. We now introduce how to construct the edges in our FABind layers. For clarity, we use FABind in the pocket prediction module for a demonstration. The cut-offs are set to the same in both pocket prediction and docking. As defined in the paper, we have three types of edges,  $\mathcal{E} := \{\mathcal{E}^l, \mathcal{E}^p, \mathcal{E}^{lp}\}$ , for ligand, protein, and ligand-protein interface, respectively.  $\mathcal{E}^l$  and  $\mathcal{E}^p$  are constructed from independent context, while  $\mathcal{E}^{lp}$  is constructed from external interface. For the independent context of a ligand, we directly refer to chemical bonds as constructed edges  $\mathcal{E}^l$  with the biological insight that a molecule keeps its chemical bonds during the process. For the independent context of a protein,  $\mathcal{E}^p$  is defined as the edges connecting to nodes when the spatial distance is below a cutoff distance  $c_{\mathrm{in}}$ . We set  $c_{\mathrm{in}} = 8.0$  in our work following Kong et al. [22]. Note that independent edges for ligands and proteins differ. For external edges  $\mathcal{E}^{lp}$ , we also add edges with a spatial radius threshold  $c_{\mathrm{ex}} = 10.0$  following TankBind [31].
Global Nodes. MEAN [22] demonstrates that the global nodes intensify information exchange during message passing. Therefore, we insert a global node into each component (ligand or protein) of the complex as well. A global node connects to all nodes in the same component and the other global node. The coordinates are initialized as zero tensors and can be updated during feed-forward.
Iterative Refinement. Iterative structure refinement has been proved as a critical design in structure prediction task [19]. It allows the network to go deeper without adding much computational overhead. Specifically, we update coordinates during all iterations and update hidden representations only in the last iteration. To stabilize the training process and save memories, we stop the gradients except for the last iteration. In our implementation, we also accelerate training speed by randomly sampling an iteration number less than or equal to the configuration  $N$  for each batch, while always refining  $N$  iterations during inference.
5The pre-trained ESM-2 checkpoint can be found at https://dl.fbaipublicfiles.com/fair-esm/models/esm2_t33_650M_UR50D.pt
Methods	Ligand RMSD	Centroid Distance
Percentiles ↓	% Below ↑	Percentiles ↓	% Below ↑
25%	50%	75%	Mean	2Å	5Å	25%	50%	75%	Mean	2Å	5Å
FABIND	1.7	3.1	6.7	6.4	33.1	64.2	0.7	1.3	3.6	4.7	60.3	80.2
NO SCHEDULED SAMPLING	1.7	3.5	7.0	6.4	28.7	63.4	0.7	1.5	3.5	4.5	60.3	82.6
COORD LOSS ONLY	2.6	4.1	7.3	6.9	16.3	60.9	1.0	1.7	4.5	4.7	53.9	77.4
COORD LOSS + DIST. LOSS (1)	1.9	3.5	7.4	6.5	26.2	63.4	0.7	1.5	4.1	4.5	56.7	80.0
NO CROSS-ATTENTION	2.2	4.2	7.0	6.4	21.4	59.9	0.9	1.9	4.1	4.5	52.3	80.4
NO INDEPENDENT MP	2.1	3.8	7.0	6.6	22.6	61.7	0.8	1.5	3.3	4.6	59.8	82.6
NO ITERATIVE	2.2	4.1	7.2	6.6	22.5	58.5	0.9	1.9	4.7	4.7	52.3	80.4
NO ESM-2	1.9	3.6	8.0	6.3	27.5	61.2	0.8	1.6	3.8	4.4	56.2	79.1
WITH POST OPTIMIZATION	1.8	3.5	7.2	6.6	30.9	61.4	0.7	1.4	3.5	4.7	59.8	80.7
Table 6: Results of full ablation study.
B Apo-Structure Docking
As stated in DiffDock [7], the PDBBind benchmark primarily focuses on evaluating the ability of docking methods to dock ligand to its corresponding receptor holo-structure, which is a simplified and less realistic scenario. However, in real-world applications, docking is often performed on apo or holo-structures that are bound to different ligands. To address this limitation, DiffDock proposed a new benchmark that combines the crystal complex structures of PDBBind with protein structures predicted by ESMFold [28]. In order to validate the efficacy of our FABind in the apo-structure docking scenario, we also evaluated its performance under the same settings with DiffDock. In order to validate the efficacy of our FABind in the apo-structure docking scenario, we implement the same experimental setup as DiffDock. For 363 test samples, we first extract their sequences from PDBBind and use esmfold_v1 to predict their structures, from which process we assume that the apo protein structures are obtained. One sample (PDB: 6OTT) is filtered out due to out-of-memory error. Then we align the rest of the samples with PDBBind, where 12 samples are further excluded due to memory limitations, the same as DiffDock. The results in Table 5 demonstrate that FABind outperforms DiffDock, achieving an RMSD of less than  $2\AA$  on  $24.9\%$  of the complexes generated by ESMFold. This demonstrates the strong predictive capacity of FABind for apo-structure predictions.
C.1 Full Ablation
The comprehensive ablations are listed in Table 6. We can observe that each of the components contributes to the good performance of our FABind. Firstly, the scheduled training strategy, when removed, leads to a slight decrease in performance for challenging cases (e.g., RMSD  $75\%$ ). This indicates that the scheduled training strategy contributes positively to handling difficult scenarios. Regarding loss construction, the inclusion of the distance map loss is crucial, and solely utilizing the first term of distance map loss (i.e., the distance loss between  $D_{ij}$  and  $\widetilde{D}_{ij}$  in  $\mathcal{L}_{dist}$ , which we call "coord loss + dist. loss (1)" in Table 6) does not yield optimal results. The architecture design also has a substantial impact. The removal of the cross-attention update and independent message passing severely impair the model's ability to handle favorable cases (e.g., RMSD  $25\%$ ). This suggests that cross-attention update and independent message passing are both vital for capturing important structural dependencies. Furthermore, iterative refinement is found to be indispensable for most structure prediction models, including ours. In terms of feature representation, the utilization of ESM-2 features for residues proves to be most beneficial for challenging cases. It enhances the model's capability to handle difficult scenarios effectively. Lastly, post-optimization does not significantly affect the overall performance but ensures that the generated ligand conformations are chemically more rational.
C.2 Iterative Refinement
In this section, we investigate the impact of iterative refinement on our model. We utilize our best-performing model and evaluate its performance using different iterations during inference. The
Iteration	Ligand RMSD	Centroid Distance	Average
Percentiles ↓	% Below ↑	Percentiles ↓	% Below ↑
25%	50%	75%	Mean	2Å	5Å	25%	50%	75%	Mean	2Å	5Å	Runtime (s)
ITERATION=1	3.1	4.8	7.9	7.3	5.2	51.8	1.2	2.0	5.3	5.3	49.9	74.1	0.03
ITERATION=2	2.4	3.8	7.7	6.8	20.1	60.1	0.9	1.5	4.6	4.9	56.5	77.1	0.05
ITERATION=4	1.8	3.3	7.2	6.5	28.4	63.6	0.7	1.4	3.8	4.8	59.5	78.8	0.07
ITERATION=6	1.8	3.2	7.0	6.4	30.9	64.5	0.7	1.3	3.7	4.7	60.6	79.6	0.10
ITERATION=8	1.7	3.1	6.7	6.4	33.1	64.2	0.7	1.3	3.6	4.7	60.3	80.2	0.12
ITERATION=10	1.7	3.1	6.6	6.4	32.5	65.3	0.7	1.3	3.5	4.7	60.6	80.4	0.16
ITERATION=12	1.7	3.1	6.6	6.4	32.5	65.6	0.7	1.4	3.6	4.7	61.7	80.4	0.19
Table 7: Inference on different iterations.
(a) PDB: 6N93
Figure 3: Additional case studies. Pose prediction by FABind (green), DiffDock (wheat), E3Bind (magenta), TankBind (cyan), and EquiBind (orange) are placed together with protein target structure, and RMSD to ground truth (red) are reported. (a) For unseen protein (PDB 6N93), FABind, E3Bind, and TankBind successfully identify the pocket, among which FABind predicts the most precise binding pose with the lowest RMSD  $2.7\AA$ , while the other methods are all off-site. (b) For PDB 6JB4, all deep learning models find the right pocket, among which FABind predicts the most precise binding pose with the lowest RMSD  $1.9\AA$ .
RMSD from
ground truth (A)
FABind: 2.7
DiffDock: 55.5
E3Bind: 6.8
TankBind: 7.4
EquiBind: 34.0
RMSD from
ground truth (A)
FABind: 1.9
DiffDock: 2.2
E3Bind: 3.4
TankBind: 3.4
EquiBind: 5.7
(b) PDB: 6JB4
results are reported in Table 7. From the table, we can find that (denote the number of iterations as  $i$ ,  $1 \leq i \leq 12$ ): (1) the performance improves as  $i$  increases. (2) The results tend to be stable when  $i$  increases to some extent. (3) When  $i = 8$ , the results are generally optimal. However, the results are similar when  $4 \leq i \leq 12$ . Thus, a smaller value of  $i$  can be used for better efficiency.
C.3 More Cases
Here we show more cases on test sets to further verify the ability of FABind in finding the correct pocket for unseen protein and docking at the atom level. From Fig. 3(a), in PDB 6N93, the protein is unseen in the training set, and only the predictions of FABind, E3Bind and TankBind are in the right pocket, among which FABind predict the most accurate binding pose (RMSD  $2.7\AA$ ). From Fig. 3(b), in PDB 6JB4, though every method correctly finds the native pocket, FABind predicts the most accurate ligand conformation (RMSD  $1.9\AA$ ).
D Broader Impacts and Limitations
Broader Impacts. Developing and maintaining the computational resources necessary to conduct AI-based molecular docking requires considerable resources, which may lead to a waste of resources.
Limitations. In FABind, we represent the protein structure at the residue level, assuming the rigidity of the protein. While many existing molecular docking methods adopt a similar protein modeling strategy, we believe that employing atom-level protein modeling and incorporating protein flexibility into the modeling process could yield improved results. FABind is not optimally designed to accommodate scenarios with multiple binding sites, wherein a ligand has many potential binding
sites on a protein. Additionally, it is not well-suited for situations where a ligand can adopt multiple binding conformations within a specific pocket of a protein. Generative modeling [7] (sample-based method) presents a reasonable approach to address these two scenarios. However, due to the scope and limitations of our current work, we have decided to defer these aspects to future research.