D EEP L EARNING M ODEL FOR F LEXIBLE AND E FFICIENT P ROTEIN -L IGAND D OCKING
A BSTRACT
Protein-ligand docking is an essential tool in structure-based drug design with applications ranging from virtual high-throughput screening to pose prediction for lead optimization.
Most docking programs for pose prediction are optimized for re-docking to an existing co-crystalized protein structure ignoring protein flexibility.
In real-world drug design applications, however, protein flexibility is an essential feature of the ligand-binding process.
Here we present a deep learning model for flexible protein-ligand docking based on the prediction of an intermolecular
Euclidean distance matrix (EDM), making the typical use of search algorithms obsolete.
Our method introduces a new approach for the reconstruction of ligand poses in Cartesian coordinates, utilizing EDM completion and restrained energybased optimization.
The model was trained on a large-scale dataset of proteinligand complexes and evaluated on standardized test sets.
Our model generates high quality poses for a diverse set of protein and ligand structures and outperforms comparable docking methods.
I NTRODUCTION
Protein-ligand docking is widely used in structure-based drug design and throughout early drug development.
Molecular docking enables the identification of key binding interactions which can be optimized for affinity or selectivity.
Furthermore, docking can be applied to virtual screening (VS) protocols, wherein a large library of compounds are screened for potential target binding (Pereira et al., 2016; Fan et al., 2019).
Docking can also be applied in the context of inverse (or reverse) docking in which one compound is screened against a large set of potential targets.
This technique is particularly useful in drug repurposing, polypharmacology and side-effect prediction (Kharkar et al., 2014).
Despite the broad applicability of docking, several challenges still exist with traditional docking methods.
One challenge is the computational cost of the sampling or search algorithms.
Docking programs often generate millions of potential poses and attempt to rank the strongest binding ones towards the top.
This creates a burdensome computational requirement and often makes molecular docking of large chemical libraries infeasible or requires the use of simplified and less accurate scoring metrics.
Another limitation of many docking methods is the use of a rigid receptor, which neglects the induced-fit effect known to play a critical role in protein-ligand recognition and binding (Koshland Jr, 1958; Savir & Tlusty, 2007).
Simplified scoring functions which neglect protein flexibility have detrimental effects on the performance of docking algorithms.
Finally, there are still protein systems where state-of-the-art docking algorithms fail to generate any correct pose.
1. R ELATED W ORK
Over recent years, several different methods employing deep learning models have been applied to molecular docking.
These models can be divided into two predominant approaches: reranking and generation.
In the reranking approach, an ensemble of docked poses are first generated using a traditional docking method.
Then the deep learning model is trained to rerank the ensemble such that the top-ranked poses have the lowest RMSD to the native pose obtained from experiment.
A number of neural network architectures have been applied to this approach including convolutional neural networks(Ragoza et al., 2017; Mahmoud et al., 2020b; Zheng et al., 2019) and graph neural networks (Wang et al., 2021).
While this approach has proven to be powerful in improving the ranking of poses, they do not address the sampling problem.
In the second approach, deep learning has been employed to generate the docked poses directly.
This approach has been less extensively studied with only a few methods developed in recent years (McNutt et al., 2021; Mahmoud et al., 2020a;
St¨ark et al., 2022).
Our work falls into the later of these categories.
Our contributions to the existing methods are two-fold:
• We present an equivariant graph neural network (EGNN) model for the accurate prediction of protein-ligand distance matrices.
• We provide a method for the reconstruction of the binding poses in Cartesian space based on the predicted Euclidean distance matrix which respects the physical constraints and energy of the pose.
[Figure]
In this section, we propose a model architecture for the prediction of protein-ligand distance matrices.
The model, depicted in figure 1, consists of two independent graph neural networks (GNNs) (one for ligand, one for binding site) and stack of fully connected layers.
The model predicts distance matrices as an intermediate representation due to it’s inherent invariance to rigid transformations such as translation and rotation.
The GNN model used in this work, the Equivariant Graph Neural
Network (EGNN) from Satorras et al.
(2021) is also unaffected by rigid transformations.
The EGNN was chosen for its ability to learn information about a nodes environment and 3D geometry.
The nodes of the binding site graph are defined by the C α atoms of residues with any atom within 8 ˚A of the co-crystallized ligand.
Therefore, each node represents an entire residue centered on the
C α atom.
As a result, the model is inherently flexible as it disregards the side chain conformation entirely.
The nodes of the ligand are defined by the ligand heavy atoms and input coordinates are given from a random conformation generated by RDKit (Landrum, 2021).
Given a graph G =
( V , E ) with nodes v i ∈ V and edges e ij ∈ E an EGNN can be defined as a set of equations which updates V in successive layers.
Each node v i is associated with a set of node features h i ∈
R f where f is the number of features, and a set of n-dimensional coordinates x i ∈ R n (here: n = 3) .
Each EGNN layer l is provided with these node features h l (with h
0 i = h i ), coordinates x l (with x 0 i = x i ), and edge information E and outputs updated h l +1 and x l +1 .
The equations defining this update are as follows:
[Equation]
Both EGNNs have the same structure, each with stacks of three layers which successively transform the input features into feature embeddings containing chemical and geometric information.
These feature embeddings are denoted as h lig and h bs for the ligand and binding site graph respectively.
These two sets of feature embeddings are concatenated for each pair of protein and ligand nodes to form a tensor B ∈ R N × M × f with
[Equation]
for i ∈ { 1 , 2 , ..., N } and j ∈ { 1 , 2 , ..., M } where N is the number of ligand nodes and M is the number of binding site nodes.
This tensor B ∈ R N × M × f is finally transformed into the distance matrix D ∈ R N × M between ligand and protein atoms using a multi-layer perceptron (MLP)
Φ d
[Equation]
The distance matrix D can then be used to generate an ensemble of poses in Cartesian coordinates by the reconstruction process described in section 2.5.
2. D ATASETS 2.2. T RAINING AND V ALIDATION
During early experimentation, a significant improvement in performance was observed with increasing training set size.
Therefore, we chose the large-scale BioLiP dataset of protein-ligand structures for training and validation (Yang et al., 2012).
The complete set contains nearly 300k protein-ligand structures.
However, many ligands are non-drug-like and had to be excluded from the training set.
The refinement process resulted in about 53k protein-ligand structures used for training and validation.
The complete refinement process is described in figure 7 in the supplementary information.
The validation set was then created from the refined BioLiP dataset.
The importance of having an unbiased validation set has been demonstrated numerous times in the literature (Davis et al., 2020;
Wu et al., 2018; Wallach & Heifets, 2018; Tran-Nguyen et al., 2020).
Unfortunately, most of these recent unbiasing approaches rely on activity data, which is not available for the BioLiP dataset.
Therefore, we decided to apply the Kennard-Stone (KS) algorithm to generate a robust validation set (Kennard & Stone, 1969; Xu & Goodacre, 2018).
The KS algorithm works by selecting the two most distant samples first.
Then selecting the third sample based on the distance from the first two, and so on.
KS produces a uniform selection of samples across the dataset, including samples from the boundary of the dataset.
RDKit fingerprints were generated for every ligand in the dataset and a sample-wise distance matrix was created using the Tanimoto similarity metric.
KS was performed using this matrix and the size of the desired validation set, 5,233, which represents 10% of the full dataset.
2.2. I NDEPENDENT T EST S ETS
The PDBbind core set is a subset of 285 high resolution, manually curated protein-ligand structures used to validate docking methods (Su et al., 2018).
We chose to use the latest 2016 PDBbind core set to evaluate our method for the task of re-docking.
Because these structures are also contained in the larger BioLiP dataset, we excluded them from the training and validation sets.
In addition to re-docking, our method was also evaluated on the task of cross-docking, wherein a ligand is docked to an independent protein structure which hasn’t been influenced by the ligand.
Cross-docking is more difficult but also more rigorous in evaluating docking methods with regards to real-world applications.
To this end, the DISCO cross-docking dataset was employed as a second test set (Wierbowski et al., 2020).
The dataset features 94 targets with an average of 50 ligands per target.
As none of these complexes were included in the BioLiP dataset, nothing had to be removed from the training or validation sets.
2. F EATURIZATION
As the ligand and binding site graphs are processed by the model separately, and are have different levels of granularity, different featurization schemes were used for each.
A combination of embeddings generated using pre-trained, self-supervised neural networks and hand-crafted features were used.
The ligand embeddings were generated using a self-supervised graph transformer known as GROVER (Rong et al., 2020).
Hand-crafted features for the ligand describe the physical and chemical nature of each atom.
The binding site embeddings were generated using a state-of-the-art protein sequence transformer model known as ESM1b (Rives et al., 2021).
Protein hand-crafted features describe the sequence and structure of each residue.
All hand-crafted features were encoded and normalized as needed.
A complete description of the featurization is included in section 6.2 of the supplementary information.
2. M ODEL T RAINING
The model was trained to minimize the mean squared error between the predicted and known distances.
Training was done using the Adadelta optimizer (Zeiler, 2012) with a learning rate of 0.01.
The Adadelta optimizer was found to have better stability and performance compared to Stochastic
Gradient Descent or Adam optimizers.
The model was trained on the full training set (47,095 structures) for a total of 100 epochs.
The validation loss was calculated following each epoch and was used for the selection of the best model weights.
The training and validation loss improved significantly during training, and good correlation between the two indicates there was no overfitting of the training data.
2. R ECONSTRUCTION
In order to reconstruct the Cartesian coordinates, first the predicted distance matrix, that contains only intermolecular protein-ligand distances, must be completed into the full square EDM.
This is accomplished through tiling the ligand distance matrix D lig , binding site distance matrix D bs , and the predicted protein-ligand distance matrix D pred as depicted in equation 7.
[Equation]
The exact distance matrix of the binding site is computed from the original X-ray structure, as the backbone atoms (including C α atoms) are constrained during pose generation.
However, the exact distance matrix of the ligand is not known prior to having the docked pose.
Using the native pose would bias the reconstruction towards the correct result and would show higher performance than actually achieved.
Therefore, the ligand distance matrix used in reconstruction is generated from an unbiased conformation generated with RDKit.
In order to generate an ensemble of different poses, our method relies on randomly sampling from the predicted protein-ligand distances and only using these samples to reconstruct the pose.
Sampling 50% of distances was found to be a good balance between increased diversity while maintaining accuracy.
Although, this sampling introduces a new problem; The EDM is now sparse and the missing values will lead to a deformed structure.
Thus, an EDM completion algorithm called
Semi-Definite Relaxation (SDR) was used to complete the matrix, resulting in a valid EDM.
Given a precise EDM, Multi-Dimensional Scaling (MDS) is capable of reconstructing the Cartesian coordinates exactly.
MDS uses an intermediate representation called a Gram matrix in order to accomplish this conversion (Hoffmann & No´e, 2019).
The relationship between an EDM D and its corresponding Gram matrix G is defined in equation 8.
[Equation]
Because the Gram matrix G is real symmetric matrix, it can be decomposed into eigenvalues and eigenvectors denoted in equation 9 as S and U respectively.
[Equation]
The Cartesian coordinates X ∈ R 3 N can then be reconstructed from U and S using equationsand 11 where N is the total number of protein and ligand nodes. [Equation]
[Equation]
where the last three row vector of matrix V are used.
However, even when small levels of noise are present in the distance matrix, as is the case with distances predicted by a neural network, reconstructing the coordinates without distortion becomes considerably harder.
MDS can attempt to reconstruct the coordinates, but will often lead to unrealistic molecular conformations with extremely unfavorable bonds and angles.
Therefore, we chose to develop a new approach which uses an energy-based model to refine the atomic coordinates into a reasonable docked pose which satisfies both the predicted protein-ligand distances as well as physical constraints described by a molecular force field.
The approach is to utilize the predicted distances as an additional harmonic restraint force between atoms of the protein and atoms of the ligand during energy minimization.
This allows us to perform a correction on the initial reconstruction so that both the predicted distances are satisfied and the potential energy of the protein-ligand complex is minimized.
This can be formalized with the following equations, where U represents the potential energy as a combination of U ff , the potential energy of the protein-ligand system described by the ff14SB (Maier et al., 2015) and gaff-2.11 (Wang et al., 2004) force fields, and U pred , the harmonic restraints imposed by the predicted distances. [Equation]
[Equation]
U pred is a harmonic potential based on the difference between current distances R ij (during energy minimization) and the predicted distances D ij .
Additionally, a cutoff C is introduced so that restraints are only introduced for distances smaller than a prescribed value.
Typically, harmonic forces have a constant spring coefficient k defined by the two bonded atom types.
However, in our case we chose a spring constant that is inversely proportional to the predicted distance.
This signifies that small distances represent important, strong interactions between protein and ligand, e.g. hydrogen bonds.
Those interaction distances should be strongly confined to the predicted values and therefore have a larger force constant than larger distances.
The C α atoms of the protein were harmonically restrained with a spring constant of 2 kJ mol − 1 ˚A − 2 during minimization.
This prevented large changes to the backbone, while sidechains were allowed to move freely.
Minimization was performed until a tolerance of 0.5 kJ/mol was reached.
The hyperparameters C and k were optimized using a grid search with a smaller subset of the training set, and were set to be 10.0 ˚A and 2.0 kJ mol − 1 ˚A − 2 respectively.
Finally, the ensemble of poses were ranked according to the final potential energy U .
also deformed
In the unminimized position ligand is generally
However some bonds and angles are unrealistic
The minimized ligand has corrected bond lengths and is in better
Additionally some of nearby adopt a conformation similar
2. B ENCHMARK C OMPARISONS
In order to compare our results to traditional docking methods, we employed two different docking software packages for benchmarking.
GeauxDock is a high-performance docking software developed to be used on CPU or GPU.
It is based on a Monte Carlo search algorithm and has a scoring function composed of physical-based energy terms and knowledge-based potentials (Fang et al., 2016).
Importantly, GeauxDock only considers C α atoms of the receptor, thereby making it implicitly flexible like our method.
Autodock Vina is another docking method which is widely used in the context of protein-ligand docking(Koes et al., 2013; Pagadala et al., 2017).
For this work, a fork of AutoDock Vina called
Smina was used to run the protocol for its ease-of-use and performance improvements over the original program(Koes et al., 2013).
As opposed to our method and GeauxDock, Autodock Vina uses all atoms of the receptor.
The search space for AutoDock Vina was defined using the same protocol as the original paper (Koes et al., 2013).
The box was centered on the binding site residues with sides extending 8 .
˚A from the native ligand with a minimum length of 22 . 5 ˚A per side.
No flexible residues were used and all other settings were left as their default value.
In the context of re-docking, using all atoms of the receptor imposes a bias towards the final pose and can lead to deceptively good results (Jain, 2008; 2009; Jain & Nicholls, 2008).
Therefore,
AutoDock Vina is not a suitable tool for comparison for the re-docking task.
This bias is not an issue when using methods which use C α atoms alone to represent the receptor, as the induced side chain conformation is discarded entirely.
For this reason, GeauxDock was selected as the primary benchmark for the re-docking task.
In the context of cross-docking, ligands are docked to a protein structure resolved independently from the ligand.
This provides a more realistic benchmark, better aligned to the real-world use of docking programs in structure-based drug design.
Moreover, crossdocking lessens the bias imposed by using all atoms of the receptor.
For this reason, AutoDock Vina was selected as the primary benchmark for the cross-docking task.
R ESULTS 3. R E - DOCKING
First, we summarize the results for the re-docking task.
A random sample of docked poses from the
PDBbind core test set is shown in figure 4 superimposed with their native pose.
In comparison to
GeauxDock, our model had superior performance with respect to RMSD and rate of success among the top-1, top-3, and top-5 ranked poses (Figure 5).
In terms of RMSD, our method outperformed
GeauxDock with an average improvement of 1.3 ˚A 1.1 ˚A and 0.6 ˚A for the top-1, top-3, and topposes respectively.
In terms of rate of success, our method had a 2.3, 1.7, and 1.3-fold improvement over GeauxDock for the top-1, top-3, and top-5 poses respectively.
Success for the re-docking task was defined as having a docked pose with less than 2.0 ˚A RMSD from the native pose.
Another comparison using a higher cutoff of 3.5 ˚A is provided in figure 9 of the supplementary information.
Although AutoDock Vina is not a suitable comparison for the re-docking task, it was still run for the
PDBbind core test set.
The results exceeded our method in several categories.
This test, however, is not a fair comparison due to the artificial enhancement of AutoDock Vina results by using all receptor atoms in their native conformation in a re-docking setting.
Nonetheless, there were about 50 systems where our method outperformed Autodock Vina in the top-1.
While the improvement in most of these cases was modest, a handful of examples showed significant improvement of 7 ˚A or more.
These examples, shown in figure 10 in the supplementary information, failed entirely using
Vina but succeeded with our method.
3. C ROSS - DOCKING
Next, we summarize the results for the cross-docking task.
As with re-docking, performance was assessed in terms of RMSD and rate of success among the top-1, top-3, and top-5 ranked poses.
Among the top-1 poses, our method was successful in 52% of systems, an improvement from 40% with AutoDock Vina.
Improvement among the top-3 and top-5 poses was more modest.
In terms of RMSD, our method performed exceptionally well in the top-1 with around 1.0 ˚A improvement in mean RMSD.
The performance was also examined for each system.
Some difficult targets, such as Adenosine A2a receptor, Farnesyltransferase, HIV integrase, and Histone deacetylase 8 all had greater than 2-fold improvements in RMSD.
A full overview of the cross-docking results by system is included in tables 3 and 4 of the supplementary information.
GeauxDock was not run for the cross-docking dataset.
D ISCUSSION
Despite the excellent results, there are still a couple limitations to our method that should be pointed out.
First, the diversity of the generated poses is not as high as traditional docking methods.
The distance prediction model only generates a single distance matrix and thus the reconstruction procedure often leads to very similar poses.
As such, our method performs exceptionally well in the toprank but only improves modestly when considering lower ranked poses.
Another limitation is the ranking procedure.
The ranking used in this study is simple and involves no additional scoring function.
The results do show some improvement when considering the lower ranked poses.
Therefore, there is an opportunity to boost performance even further by adding an additional scoring function, either based on traditional scoring functions or a deep neural network.
We hope that future work can address these limitations and develop our method further.
C ONCLUSION
In this study, we proposed a novel docking method capable of generating high quality poses for a diverse set of protein and ligand structures.
The method is based on a combination of two independent equivariant graph neural networks for protein and ligand, combined by a multi-layer perceptron model to predict the EDM between the two entities.
Pose generation is based on EDM completion, reconstruction based on MDS and energy-based models.
The method was rigorously evaluated on two independent test sets, covering both re-docking and cross-docking tasks.
Moreover, two independent docking programs, GeauxDock for re-docking and AutoDock Vina for cross-docking were selected as comparison.
Our method showed superior performance in both tasks in terms of RMSD and rates of success.
Additionally, our methods performance is uncoupled from a time-consuming search algorithm and the need to enumerate many possible docked poses.
Together, these factors show that deep learning models are capable of flexible and efficient protein-ligand docking.
This approach is a powerful new paradigm which will be investigated further and used to accelerate modern structure-based drug discovery.
S UPPLEMENTARY I NFORMATION 6. D ATASET P REPARATION
Proteins with a sequence longer than 1024 were removed as they could not be processed by the model in contact with multiple chains were removed with were
6. F EATURIZATION D ETAILS 6.2. B INDING S ITE F EATURES
We employed two methods for featurizing the binding site nodes from different perspectives.
In the first approach, we obtained sequence-based embeddings generated by the Evolutionary Scale
Modeling (ESM) deep neural network (Rives et al., 2021).
In the second approach, we collected several handcrafted features which describe the geometry and chemical environment of each residue.
The ESM-1b model was used to generate the sequence embeddings.
This model contains roughly 650M parameters in 33 transformer layers.
The model was trained on a diverse dataset of 27M unique protein sequences.
It is considered a general-purpose protein language model and has shown state-of-the-art performance when applied to structure prediction tasks (Rives et al., 2021).
To generate the embeddings, we provide the complete protein sequence to the ESM-1b model.
The sequence is transformed through the stack of layers and is retrieved after the last hidden layer.
The resulting embeddings have a rank of 1280 and are generated for each letter in the sequence.
Embeddings specific to the binding site residues were then extracted.
The second featurization method combines several hand-crafted features which describe the geometric and chemical environment of each residue.
The selected features were used in previous works on deep learning of protein structures (Jing & Xu, 2021; Cao et al., 2019).
The residue type was onehot encoded as one of the twenty standard amino acids with an additional category for non-standard residues.
DSSP was used to assign the secondary structure and calculate Solvent Accessible Surface Area (SASA) for each residue (Kabsch & Sander, 1983).
The secondary structure was one-hot encoded as one of eight types.
Hydrogen bonding of the protein was calculated using HBPLUS (McDonald & Thornton, 1994).
The hydrogen bonds were segmented into eight categories: one for each possible combination of donor/acceptor, backbone/sidechain, and occupied/accessible.
The position of the residue within the full sequence is included and normalized by the sequence length.
Finally, the backbone dihedral angles phi, psi, and omega are included.
The periodicity of the angles are handled by transforming with sin and cos functions.
This set of features is summarized in table 1.
6.2. L IGAND F EATURES
Similar to the binding site features, we also chose to explore multiple approaches for the featurization of ligand atoms.
In the first method, we use another trained self-supervised transformer model,
GROVER (Rong et al., 2020), to generate abstract atom-level embeddings.
In the second method, we again include a collection of handcrafted features including several known to have a significant influence on protein-ligand binding.
The GROVER model is another large-scale transformer model.
However, rather than working on a sequence representation, it works directly on the graph representation of molecules.
The model contains roughly 100M parameters and was trained on 10M small, drug-like molecules.
The hand-crafted ligand features includes descriptors known to be important for binding.
The atomic element was one-hot encoded into eight possible classes (C, O, N, P, B, S, Halogens, and Metals).
Different halogens and metal elements were grouped together due to their sparsity in the dataset and similar chemical characteristics.
Still, individual halogens and metal elements are able to be distinguished based on the other features such as atomic weight and radii.
The hybridization type was also one-hot encoded into one of seven possible classes (S, SP, SP2, SP3, SP3D, SP3D2, and unknown).
The aromaticity of ligand atoms were indicated by a single boolean value.
Several other atom-level properties including atomic weight, atomic radii, partial charge, polar surface area (PSA), accessible surface area (ASA), and logP, were calculated using RDKit (Landrum, 2021).
The hand-crafted ligand features are summarized in table 2.
[Table] [Table]
[Figure]
[Table] [Table] [Table]
[Table]