MetaDTA: Meta-learning-based drug-target binding affinity prediction
Abstract
We propose a meta-learning-based model for drug-target binding affinity prediction (MetaDTA), for which no information on the protein structures or binding sites is available.
We formulate our method based on the Attentive Neural Processes (ANPs) (Kim et al., 2019), where the binding affinities for each target protein are modeled as a regression function of the compounds.
Known drug-target binding affinity pairs are used as support set data to determine the regression function.
We designed few-shot prediction experiments with a small number of support set data, similar to the typical situations in actual drug discovery processes.
Experimental results showed that the proposed method outperforms the sequence-based baseline models with the same amount of limited data.
Introduction
In the early stage of drug discovery, accurate prediction of the binding behavior between the target protein and drugs is crucial for discovering the candidate molecule with decent potency and selectivity (Hughes et al., 2011).
The drug-target binding affinity (DTA) prediction is a regression problem that aims to predict experimentally measured binding affinity values, which is helpful for ranking and optimizing the compounds.
Although it is usually more difficult than the drug-target interaction (DTI) prediction problem, which is a binary classification of the active/inactive compounds, the DTA prediction has been actively tackled based on the recent advances in deep learning (Ragoza et al., 2017; StepniewskaDziubinska et al., 2018; Jim´enez et al., 2018;
Zhang et al., 2019; Jones et al., 2021; Abbasi et al., 2020; ¨Ozt¨urk et al., 2018; Abbasi et al., 2020; Nguyen et al., 2020b; 2021).
DTA prediction becomes challenging when the exact structural information of the target protein and the binding location are not precisely determined.
This often occurs, especially when the drug discovery program aims to conquer disease by controlling a novel target protein that has not been studied much.
For those cases, structure-free binding affinity prediction methods are required.
Recently, the DTA prediction methods based on the protein sequence were proposed (¨Ozt¨urk et al., 2018; Huang et al., 2021) which bypasses the use of three-dimensional structural information of the target proteins.
Those methods successfully predict binding affinity without using the structural information. However, the performance is degraded when the number of training data samples for the target protein is limited (Huang et al., 2021).
In the hit identification and lead optimization procedure, designing, synthesizing, and measuring the potency of molecules are time-consuming and expensive processes.
These limit the number of molecules to be tested for a target protein, reducing the size of the dataset.
In
[Figure]
𝑓 𝑥 ! ,𝑆 " 𝑆 ! =
{(𝒙 " , 𝑦 " )} 𝑥 !
𝑥 # ∈ 𝑄 !
[Figure]
𝑥 # 𝑦 #
[Figure]
such a small data set, the performance of the deep learning approaches mentioned above is reduced, so developing a model that can predict well even in a few-shot manner is challenging.
We can think of this as a few-shot learning problem where a binding affinity prediction function is determined with a small number of given data (Figure 1). Instead of using the intrinsic protein features such as the sequences, we use the known (compound, binding affinity) pairs for a target protein as the features. Based on this setting, we propose a meta-learning model for drug-target binding affinity prediction (MetaDTA), where the regression functions that map between the given compounds to corresponding binding affinities are modeled with Attentive Neural Processes (ANPs) (Kim et al., 2019).
Each regression function for each task represents the binding affinities for a specific target protein.
By stochastically splitting the compounds into support and query sets, we generate various episodes for a task and use them for training the model.
ANPs have many advantages to be applied for binding affinity predictions.
First, ANPs are few-shot learning models specialized for regression problems.
Next, the model architecture, consisting of a cross-attention-based deterministic path and a global latent path, represents the prediction strategies well.
The deterministic path can serve as a ligand similarity-based prediction (Altae-Tran et al., 2017), and the latent path can be viewed as a target protein feature extraction.
Finally, ANPs do not underfit the given context data, which fits well for the binding affinity prediction problem where the context data is usually the most reliable experimental result.
Numerical experiments on the binding affinity dataset were performed to compare the performance with existing sequence-based prediction models (¨Ozt¨urk et al., 2018;
Huang et al., 2021).
The advantages of the MetaDTA are summarized as follows:
• The accuracy of MetaDTA is better than the sequence-based models, especially when a small number ( ≤ 100) of data pairs are available.
( § 4.2) • MetaDTA can accurately predict the binding affinity even when support set data is used only in test time.
( § 4.1.2) • The accuracy of MetaDTA is gradually improved when the training data contains more diverse proteins, even though the model does not directly exploit structural information.
Related Works 2. Drug-Target Binding Affinity Prediction
Machine learning has been studied as a promising approach for drug-target binding affinity prediction.
Structure-based methods use the experimentally measured 3D structure of the protein-ligand binding complexes (Ragoza et al., 2017; Stepniewska-Dziubinska et al., 2018;
Jim´enez et al., 2018; Zhang et al., 2019; Jones et al., 2021).
However, obtaining such structural information in high resolution is still a challenging task, and virtual generation of the combined structure from docking simulations often produces inaccurate results; hence the prediction accuracy degrades in these cases.
Structure-free methods try to predict the affinity without using the exact 3D binding structures.
Instead, the models use simple but abundant and diverse protein features, such as amino acid sequences (¨Ozt¨urk et al., 2018;
Abbasi et al., 2020; Nguyen et al., 2020b; 2021; Huang et al., 2021), the motifs (¨Ozt¨urk et al., 2019), Protein Sequence Composition (PSC)(Feng et al., 2018), or Position-Specific Scoring Matrix (PSSM)
features(Mousavian et al., 2016).
These methods achieve somewhat promising results.
However, learning proper protein embeddings and interactions is difficult, especially when only limited data for the target protein is available.
Most models try to learn the embedding of drugs and target proteins, as well as the interaction between them (Lim et al.
(2021) and the references therein).
In this case, the model for predicting the binding affinity y between drug d and target t from drug information x d and target information x t is formulated as
[Equation]
where g ϕ represents the embedding function for drugs, h ψ represents the embedding function for targets, and f θ represents the interaction function between the drugs and targets.
Although there exists millions of binding affinity pair data, the learned model does not generalize well for the protein targets with a small number of samples (Huang et al., 2021).
2. Few-shot learning for DTI prediction
Meta-learning, which is often referred to as ’learning to learn,’ aims to improve the ability to deal with a new task (Koch et al., 2015;
Vinyals et al., 2016;
Finn et al., 2017; Garnelo et al., 2018a; Hospedales et al., 2021).
Few-shot learning is a representative problem of metalearning, which makes predictions for the target task from only a few available examples.
Altae-Tran et al.
(2017) proposed a few-shot learning method for DTI prediction.
The binary interaction label was predicted as the weighted sum of context labels, where the weights were computed from the similarity between the learned embeddings of the drugs.
Few-shot learning showed adequate performance for various molecular property prediction tasks, including DTI prediction (Nguyen et al., 2020a) and drug response prediction (Ma et al., 2021).
Recently, a benchmark dataset (Stanley et al., 2021) was proposed for the few-shot DTI prediction problem.
Existing few-shot learning methods focused on the DTI problems, and to the best of our knowledge, few-shot learning methods specialized for the
DTA prediction problem have not been proposed.
Method 3. Few-shot Learning for Binding Affinity Prediction
We consider a problem of binding affinity prediction as a few-shot learning problem.
To this end, we consider a task T t as predicting the binding affinities for a specific target protein t .
A support set S t
= {( x i , y i )} for the task is the set of ligand x i and binding affinity y i pairs, which represents the target protein.
A query set Q t for the task is the set of ligands whose binding affinities sholud be predicted by the model.
Then, the problem of predict the binding affinity y of a query ligand x q ∈ Q t is formulated as
[Equation]
Ligand S
Ligand S Ligand S
Ligand Sn …
[Figure]
layers
Y S
Y S Y S
Y Sn
Ligand Q
S upp o r t s e t
Z (a)
Q u e r y s e t
Ligand Q2 …
Multi-headSelf-Attention N -layers
Q
K
V
R
Y Q
Y Q2 D
e c o d e r
Ligand S Ligand S
Ligand Sn …
Y S
Y S Y S
Y Sn
S e l f - A tt e n t i o n
M ~
Z (b)
S uppo r t s e t
Ligand S
[Figure]
𝑆 ! 𝑆 !
𝑄 !
Note that this formulation does not directly use the target protein information.
Instead of extracting information from the target protein, the target-specific binding affinity function f
θ is determined from available data pairs.
Various and abundant episodes for the task T t are generated by splitting available training data pairs into support and query sets during training.
The number of possible episode combinations is much greater than the number of training data pairs, so the meta-learning formation has the potential to overcome the over-fitting for a limited amount of training data pairs which is conventional in the methods following (Eq. 1).
3. MetaDTA Model
We employ the ANPs to model the binding affinity prediction functions.
As in the Neural
Processes (NPs) (Garnelo et al., 2018b), ANPs consist of deterministic and latent paths.
The deterministic part models the conditional distributions conditioned on the support set
S t to model the query set Q t , which is formulated as
[Equation]
where a query-specific representation r q ∶ = a W ( S t , x q ) is modeled with a cross-attention mechanism between the query and the items in the support set.
We employ the multi-head, scaled dot-product attention (Vaswani et al., 2017),
[Equation]
In the cross-attention module, Q is the feature of the query set ligands, and K is the feature of the support set ligands.
V is the embedded vector of the support set binding affinities.
The differences between our model and ANPs are in this cross-attention module.
We used the embedded vector of y i as V to focus on the binding affinities, while ANPs used the embedded vector of concatenated ( x i , y i ) .
Based on the intuition of the ligand-based prediction (Altae-Tran et al., 2017), the attention which represents the relationship between the support and the query ligands is used as a weight distribution of support binding affinities.
We constructed deeper models by adding cross-attention modules or additional self-attention modules after cross-attention modules (Figure 2).
The latent path uses a global latent variable z to model different realizations of regression functions, which is written by
[Equation]
where the prior function q ξ models the latent variable conditioned on the support set.
In the binding affinity prediction problem, the latent path can be considered to extract the representation of target protein from the given support set S t and use it to predict the binding affinities.
Considering both paths lead to the following conditional distribution:
[Equation]
Here the functions a W ( S t , x q ) and q ξ ( S t ) forms the encoder, and the likelihood p
ρ
( y q ∣ x q , r q , z ) corresponds to the decoder.
The training of the model is done by maximizing the following ELBO
[Equation]
by using stochastically splitted query and support sets.
Experiments 4. Experimental Setting
The primary objective of our work is to develop a binding affinity prediction method for the practical drug-discovery scenario, where limited binding affinity measurements are only available for the target protein without three-dimensional protein structure information.
To show the behavior of the proposed model, we designed two sets of experiments to answer the following questions:
• Does MetaDTA predict binding affinity well with small test samples?
• Does MetaDTA generalize well for the drugs/targets not contained in the training data?
• How much data is required for good generalization performance?
The first set of experiments tests the model performances for a limited number of few-shot test data (4.2), and the second one tests for a limited amount of training target diversities (4.3).
4.1. Dataset
We used the BindingDB (Gilson et al., 2016) for the experiments, since it is one of the largest databases of experimentally measured binding affinities.
We used the measurements labeled with the half-maximal inhibitory concentration (IC 50 ), representing the compound concentration required for 50% inhibition.
The BindingDB contains 1,390,284 inhibition affinities with 5,960 proteins, for 689,301 small molecules.
Since IC 50 values distribute over several orders of magnitude, we used pIC 50 = − log 10 ( IC 50 ) values for the prediction targets.
The outlier detection using interquartile (IQR) method was applied to filter out extraordinarily large or small values.
The multiple measurements for single molecule are averaged out, but if the standard deviation of the duplicated values is more than 0.5, the corresponding drug-target pair is discarded.
Only the targets with the binding affinities with more than 20 ligands were used.
As a result of the process, 1,040,153 inhibition affinities with 2,968 proteins and 662,438 small molecules have remained in the dataset.
For the ligand representation, we used the extended-connectivity fingerprints (ECFPs) (Rogers & Hahn, 2010) with 2,048 dimensions and a diameter of 6.
The simplified molecularinput line-entry system (SMILES) string was used for the baseline models as a ligand representation.
MetaDTA did not use the explicit protein features, but the support set data pairs were used to characterize the target protein.
The pIC 50 values were converted to 32dimensional vectors using a Gaussian kernel to increase model sensitivity towards affinity values.
The baseline methods used protein amino acid sequences as protein features. [Figure] [Figure] Random
Scaffold
[Figure]
Number of ligands in Support set
DeepDTA
MolTrans
MetaDTA(I)
MetaDTA(T)
4.1. Baselines and metrics
We compared our MetaDTA model with two baseline models, DeepDTA(¨Ozt¨urk et al., 2018) and MolTrans(Huang et al., 2021), which do not rely on the three-dimensional structure of target proteins.
Both methods use the protein sequences as protein features and SMILES strings as compound features.
DeepDTA uses convolutional neural networks (CNNs), and
MolTrans uses the Transformers to extract information of the target protein and compound features.
MolTrans was originally proposed for DTI prediction methods, so we modified the final activation of MolTrans to a linear function.
For MetaDTA, we can use the few-shot data for test tasks in two different ways.
The first is the inference mode ( MetaDTA(I) ), which does not use the few-shot data pairs in training, but only uses them in the test tasks as support sets.
The other is the training mode ( MetaDTA(T) ), in which the few-shot data were used to train the model and predict the test task as support sets.
The baseline methods used the few-shot data to train the model with other train data.
The metrics for model evaluation of regression problems are Root Mean Squared Error (RMSE), Mean Absolute Error(MAE), Concordance Index (CI), and r 2 m , and details are described in A.1.
Averaged results of 5-fold cross-validation were provided as a model performance.
4. Binding affinity prediction with limited data
In this experiment, we evaluated model performances when the number of available data for test targets was limited.
A limited number of data points were selected as the few-shot data for test targets.
The maximum sizes of the few-shot data for a target were 10, 20, 30, 50, and 100.
The models were evaluated for two different settings.
The overall ligands were not explicitly split into the training and test types in Random setting.
In Scaffold setting, the ligands were split into training and test types based on the Murcko scaffold (Bemis &
Murcko, 1996).
The similar ligands were either entirely in the training or test data in the latter case.
Scaffold setting is much more difficult because the model should predict with a limited number of previously unseen ligands.
In general, the measured MAE of the models was improved as the size of few-shot data increased (Figure 3).
Except for some cases, MetaDTA(I) outperforms the baseline models,
[Table] [Table]
even though it does not update the model parameters using the few-shot data of the test tasks.
MetaDTA(T) consistently showed the best performance for all cases.
In Scaffold setting, the prediction errors increased than that of Random setting.
The tendency is the same for the other metrics (Table 1, 2).
The performance of r 2 m > 0 .
5 is generally believed to be predictive, and MetaDTA(T) achieved this level with over 50 few-shot ligands in Random setting and 100 ligands in Scaffold setting (Appendix.
Table 5).
4. Protein cluster split
Although the above experiments were conducted for the target proteins not contained in the training data, there is still a possibility of bias due to the structural similarity of the target proteins, which could result from very similar affinity patterns for ligands.
For an appropriate train/test split of target proteins, Ragoza et al.
(2017) exploited protein sequence similarity and binding site similarity.
However, sequence similarity does not provide a complete split because some proteins have similar structures, but their sequences are different.
Clustering based on binding site similarity excludes the protein targets whose binding sites are not determined, resulting in a decrease in the size of the dataset.
To obtain an appropriate protein split for a larger number of protein targets, we exploited AlphaFold (Jumper et al., 2021) predicted protein structures to cluster the protein domains.
All available structure models for human proteins (23,391 structures as of Feb 10, 2022) were downloaded from the AlphaFold Protein Structure Database’s FTP site (Varadi et al., 2022).
We divided a single protein into domains by identifying long disordered regions and only used 1,778 proteins that have a single domain and exist in bindingDB.
The application calculates the number of inter-residue contacts using a distance cutoff of 4 ˚A for every residue.
The starting position of a domain boundary is determined as the first residue of a 10-residue window when all the consecutive residues in the window have inter-residue contacts and the ending position as the first residue of a 5-residue window with no inter-residue contact under the condition of the defined starting position.
Domains were extracted from each AlphaFold structure model using a set of the starting and end positions (i.e., domain boundaries).
We performed an all-against-all
Structural similarities between two domains were measured using TM-align (Zhang & Skolnick, 2005) and normalized by the length of the smaller domain.
The similarity score, TM-score (Zhang & Skolnick, 2004) has the value in (0, 1], where 1 indicates a perfect match.
The proteins are grouped by hierarchical clustering into 32 different classes with respect to the distances ( 1 − TM-score ) .
[Figure] [Figure]
[Figure] [Figure]
[Figure] [Figure]
[Table]
We applied 5-fold cross-validation by using the 32 protein clusters and varied the number of protein clusters from 5 to 25 for each training fold.
We limited the number of ligands in the few-shot data to 50.
MetaDTA models showed steady improvements in test accuracy as the number of training protein groups increased (Figure 4).
For the MetaDTA, the training dataset’s diversity helps to improve the model performance for the unseen, structurally dissimilar protein targets.
Although MetaDTA(I) performances were relatively poor at small training protein clusters, they became better than the baseline models with sufficiently diverse training data.
The model’s performance is generally expected to improve with more diverse training data.
However, such improvement was not evident in the baseline models.
For sufficiently diverse protein data given (25 clusters), MetaDTA(T) showed the best performance overall metrics (Table 3).
Conclusion
We proposed a meta-learning model for drug-target binding affinity prediction (MetaDTA).
The model predicts the binding affinity for query compounds by applying attention mechanisms to the support compounds and known affinity data.
Numerical experiments showed that the proposed method achieves higher prediction accuracy than the baseline models without additional information about the target protein.
In the experiments using the AlphaFold-predicted structure-based protein clustering , the proposed method showed gradual performance improvement for more diverse protein information. In contrast, the other methods did not show such improvements
A Appendix A. Evaluation Metrics
We evaluate the performance of the proposed model using most common evaluation indexes
Root Mean Squared Error (RMSE), Mean Absolute Error (MAE) and Concordance Index (CI) (G¨onen & Heller, 2005) for regression problems.
Equation for CI is described in 10 and 11.
The CI index measures whether the predicted values of two random binding affinities are predicted in the same order as the true values.
r 2 m index also be used to evaluate the predictive performance of QSAR models and model with r 2 m > 0 .
5 for the test set was considered acceptable (Pratim Roy et al., 2009).
The metric for r 2 m is described in Equation 12 where r 2 and r 20 are the squared correlation coefficients with and without intercept, respectively. [Equation]
[Equation]
A. Ablation Study
We present the results of the ablation study for proposed model in Table 4.
We evaluated the performance of cross-attention module (C.A) , self-attention module (S.A) and latent encoder path of MetaDTA model on benchmark dataset with support set length 100.
In this dataset, cross-attention alone has strong predictive performance.
The use of latent path and self-attention achieves similar performance with cross-attention and model configuration would be better to optimized in differenct dataset and different condition.
[Table]
r 2 m ndexes for few-shot scenario ligands experiment
[Table]