SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Our pre-training of the molecule and pocket encoders is based on the methodology proposed by UniMol [55]. Similar to BERT [5], we utilize a masked token prediction task. In the context of molecule or pocket data, this task involves predicting masked atom types. To augment the complexity of the pre-training task and extract valuable insights from 3D coordinates, we introduce an additional task called position denoising. Specifically, we add random uniform noise within the range of  $[-1\mathring{\mathrm{A}}, 1\mathring{\mathrm{A}}]$  to  $15\%$  of the atom coordinates. Two tasks are incorporated to restore the original positions. Firstly, the model needs to predict the original distance between two corrupted atoms. Secondly, the model needs to estimate the original coordinates of a corrupted atom using the SE(3)-Equivariance coordinate system.
B.5 Contrastive Learning Training Details
We train our model using the Adam optimizer with a learning rate of 0.001. The other hyper-parameters are set to their default values. We have a batch size of 192, and we use 4 NVIDIA A100 GPU cards for acceleration. We train our model for a maximum of 200 epochs. To avoid overfitting, we use the CASF-2016 dataset as a validation set and select the epoch checkpoint with the best  $\mathrm{BEDROC}_{85}$ . For more detailed training configurations, please refer to the code.
For the model used for human evaluation(DrugCLIP-L), we use dot product as the distance metric. For other models we use cosine similarity.
C.1 Evaluation on Target Fishing
Since DrugCLIP has the ability to learn the matching between proteins and molecules, it could be also used for target fishing, another important task in drug discovery, which entails the identification of the target from a pool of candidate targets that have the potential to bind to a specific molecule. We establish a benchmark using the CASF-2016 dataset. For each molecule, we test whether the model can correctly find its corresponding pocket from all other pockets. As shown in Table 6, DrugCLIP exhibits superior accuracy in the top 1 to 5 predictions as compared to docking software, i.e. Glide, and Vina. Conversely, DrugBA performs much poorer, with results comparable to random guessing.
Note: In this benchmark, we are unable to use the CASF-2016 dataset as both the test set and the validation set. Therefore, we split our training set in a 9 to 1 ratio and allocate the latter portion as the validation set.
Table 6: Result of Target Fishing Task on CASF-2016 dataset
Accuracy
@1	@2	@3	@4	@5
Vina [41]	3.38	5.26	7.52	9.02	10.15
Glide [11]	14.98	22.85	30.34	35.58	39.33
DrugBA	0.37	0.74	1.11	2.22	2.22
DrugCLIP	24.07	42.96	51.11	59.26	62.59
C.2 Global and Local interactions
Table 7: Performance Comparison on DUD-E and LIT-PCBA Datasets by adding atom-level interactions
DUD-E
AUROC %	BEDROC %	EF@1%
Global only	80.93	50.52	31.89
with last	78.87	44.72	28.65
with second	82.79	50.57	32.45
As shown in Table 7, using atom embeddings from the last transformer layer yields worse performance. However, marginal improvement is observed when utilizing embeddings from the second last layer. Selecting the appropriate transformer layer is crucial for obtaining effective atom embeddings and enhancing model performance, and should be considered as future work.
Table 8: Results of Human Expert evaluation. During the experiments, experts are free to choose their preferable computational tools to evaluate the screened molecules.
5kdt	6g2o	1n5x	7ksi	8etr
Glide [11]	2	2	4	7	4
DrugCLIP	8	8	6	3	6
C.3 Evaluation on Kinase Screening
In order to evaluate whether our data augmentation has negative effects on screening for similar protein pockets, we evaluate our methods on a carefully curated kinase dataset. We build a novel dataset that provides protein structures of the kinase and the molecule structures of the inhibitors. To assemble this dataset, we meticulously collected 154 kinase structures sourced from the KLIFS database (https://klifs.net/). Complementing this, we harnessed the power of data mining techniques to derive a collection of 9423 inhibitor structures from the ChEMBL database with reported bioactivity data. We further took the subset by intersecting our dataset with FDA-approved drugs, as the protein binding of drugs has been extensively explored by researchers. In our experiments, the goal is to identify the correct kinase within the top-ranked pocket structures for the given inhibitor structure input.
It's noteworthy that DrugCLIP surpasses the performance of both the molecular docking method (AutoDock Vina) and the Unimol binding affinity prediction model. We selected Unimol as a representative due to its being open-sourced and its achievement of state-of-the-art excellence in various ligand-pocket tasks, including the affinity prediction task, outperforming alternatives such as Planet.
C.4 GPCR
In this section, we demonstrate the ability of our model to pair all known human GPCR proteins with 31,422 human metabolites using AlphaFold2 predicted models. We aim to identify unrevealed GPCR ligands to facilitate functional studies, as certain GPCR proteins may have unexpected functions. For example, hOF17-4, an olfactory receptor, locates on sperms and contributes to egg localization. To achieve this, we utilized Fpocket
Table 9: Results for kinase screening
Method	Top5 acc(%)	Top10 acc(%)	Top15 acc(%)	Top20 acc(%)	Time
Vina	8.51	14.89	24.47	34.04	4h10min
Unimol Regression	7.45	18.09	23.40	32.98	<1min
DrugCLIP	14.89	23.40	28.72	37.23	<1min
(b)
Figure 6: Visualization of the docking pose of OR2T5 and 2-nonenal complex. The 2D interaction pattern is generated with LigPlot+. Interactions between OR2T5 and 2-nonenal are mainly hydrophobic interactions.
for ligand-binding pocket detection on GPCR protein surfaces and obtained 17,702 pockets. Evaluating more than  $5 \times 10^{8}$  pocket-ligand pairs would typically take around one CPU year with cutting-edge active-learning-assisted docking; however, our model can rank these pairs within minutes.
We manually evaluated top-ranked pairs and predicted their binding poses using commercialized docking software GLIDE in the Schrodinger Suite. Our findings revealed several particularly interesting pairs, including three kidney-enriched olfactory GPCRs, OR2T5, OR2T11, and OR4C3, which were predicted to bind known metabolic wastes. The kidney-expressed olfactory system has long been known to influence urine production. Additionally, the presence of olfactory G protein,  $G_{olf}$ , and olfactory-related adenylate cyclase AC3 was detected in the distal convoluted tubule. When olfactory signaling was blocked via AC3 knock-out, creatinines accumulated in the blood, indicating defective renal function.
Our model identified OR2T5 paired with 2-nonenal, OR2T11 paired with p-cresol, and OR4C3 paired with D-lactic acid. Docking poses revealed potential hydrophobic interactions, hydrogen bonds, and  $\pi -\pi$  interactions between pockets and ligands. As previous studies reported, 2-nonenal is a uremic toxin; p-cresol is an intermediate of tyrosine metabolism; and D-lactic acid is a widely distributed waste product. These molecules are highly toxic and require timely cleaning/recycling by either the excretory system or cellular processes. Our findings suggest that olfactory receptors in the kidney can sense metabolic wastes and regulate the excretion process as a feedback loop. Visualizations are shown in Figure 6,7,8.
D Disscussion on the Impact of Molecule Library
Besides library size, the molecule library choice also influences the process of identifying true binders. However, discussions on expanding the chemical library and its relation to finding potential drugs have been ongoing. Previous work [25] compared billion-size and million-size libraries, showing a log-linear improvement in predicted affinity scores with library size, and reduced bias towards reported drugs in larger libraries. Other research [19] also found that using an ultra-large library enhances the quality of screened hits compared to using small libraries which might have been fully reported by patents and articles. These studies indicate that ultra-large libraries not only offer improved hit quality due to their scale but also due to their reduced bias and
(b)
Figure 7: Visualization of the docking pose of OR2T11 and p-cresol complex. The 2D interaction pattern is generated with LigPlot+. Tyr192 of OR2T11 and O1 of p-cresol form a hydrogen bond. Tyr256 could have potential  $\pi -\pi$  interaction with p-cresol.
(a)
Figure 8: Visualization of the docking pose of OR4C3 and D-lactic acid complex. The 2D interaction pattern is generated with LigPlot+. Ser16 and Asp82 interact with D-lactic acid via hydrogen bonds.
lesser prior exploration, in contrast to smaller libraries. As a conclusion, an ultra-fast virtual screening is of great importance, because it offers the opportunity to utilize those ultra-large libraries.
The major limitation of our paper lies pertains to its interpretability. Although our model demonstrates enhanced effectiveness and efficiency, it falls short in terms of interpretability compared to traditional docking methods. These conventional approaches offer visualizations that elucidate the binding mechanism between a pocket and a molecule, providing clear explanations.
F Negative Societal Impacts
While our method has the potential to greatly expedite the drug discovery process, which is undoubtedly advantageous, it is important to consider the potential implications it may have on drug auditing and clinic trials. The increased speed and efficiency may inadvertently create additional pressures and challenges for regulatory bodies responsible for ensuring the safety and efficacy of new drugs.