Predicting Drug-Drug Interactions using Deep Generative Models on Graphs
Abstract
Latent representations of drugs and their targets produced by contemporary graph autoencoder-based models have proved useful in predicting many types of node-pair interactions on large networks, including drug-drug, drug-target, and target-target interactions.
However, most existing approaches model the node’s latent spaces in which node distributions are rigid and disjoint; these limitations hinder the methods from generating new links among pairs of nodes.
In this paper, we present the effectiveness of variational graph autoencoders (VGAE) in modeling latent node representations on multimodal networks.
Our approach can produce flexible latent spaces for each node type of the multimodal graph; the embeddings are used later for predicting links among node pairs under different edge types.
To further enhance the models’ performance, we suggest a new method that concatenates
Morgan fingerprints, which capture the molecular structures of each drug, with their latent embeddings before preceding them to the decoding stage for link prediction.
Our proposed model shows competitive results on two multimodal networks: (1) a multi-graph consisting of drug and protein nodes, and (2) a multigraph consisting of drug and cell line nodes.
Our source code is publicly available at https://github.com/HySonLab/drug-interactions .
Introduction
Investigations on biomedical multimodal graphs consisting of drugs, proteins, or cells and their interrelations are essential in drug repurposing as they can help discover innovative node-pair interactions, which may be helpful in medical treatments, on these networks.
Most approaches to tackling drug repurposing problems estimate drug side effects and drug responses to cell lines to provide important information about the use of those drugs.
In particular, polypharmacy is the practice of treating complex diseases or co-existing health conditions by using combinations of multiple medications [1].
Drug combinations consisting of many drugs affecting distinct targeted proteins can effectively modulate the process of severe diseases by facilitating the diverse biological process of different proteins [2].
Albeit commonly applied, polypharmacy is one of the major underlying issues that cause adverse medical outcomes, also known as side effects caused by drug combinations [3].
Understanding drug-drug interactions (DDIs) is pivotal in predicting the potential side effects of multiple co-administered medications.
Moreover, accurate estimation of drug responses to targeted cell lines is another major factor in providing precision medicines to patients.
However, it is unfeasible to render clinical testing of all drug combinations due to the tremendous number of relations between drug pairs or drugs and their targets (e.g., proteins).
Machine learning models have been widely applied to provide tractable solutions in predicting potential drug interactions.
Data-driven techniques (e.g., machine learning or deep learning) have proved their capabilities for dealing with complex patterns in data.
Such methods have produced remarkable results in various fields, such as computer vision, natural language processing, and audio processing.
Therefore, it is reasonable to employ machine learning approaches to investigate the interactions among complex combinations of medications.
Previous work on DDIs [4–6] used various hand-crafted drug features (e.g., chemical properties or Morgan fingerprints) to compute the similarity of each drug pair, and drug-drug interactions can be predicted based on the drug-pair proximity scores.
Nevertheless, using fixed drug representations can result in sub-optimal outcomes since they do not sufficiently capture the complex interrelations of drugs.
Recent approaches rely on graph neural networks (GNNs) [7] to directly learn drug and protein representations from structural data (i.e. graphs) augmented with additional information for node features.
Zitnik et al.
[3] proposed a graph autoencoder to predict polypharmacy side effects on a multimodal graph consisting of drug and protein nodes with various edge types.
Similarly, Yin et al.
[8] introduced DeepDrug combining graph convolutional neural networks (GCN) [9] and convolutional neural networks (CNN)[10] to learn the structural and sequential representations of drugs and proteins.
In addition, Wang et al.
[11] suggested using several graph attention (GAT) [12] layers to learn on hand-crafted features of drugs to predict drug-pair interactions.
Nyamabo et al.
[13] use multiple GAT layers to compute the drug representations and aggregate them using co-attention to produce final predictions.
These mentioned GNN-based models effectively learn the latent representations of drugs and proteins, which can be used in several downstream tasks.
Predicting drug-drug or drug-target interactions can be regarded as a link prediction task on graphs.
We argue that latent spaces produced by the aforementioned approaches are disjoint and not interpretable; hence, they are not capable of generating new links on some graph benchmarks.
In this paper, we suggest using deep graph generative models to make the latent spaces continuous and more interpretable; as a result, they are superior to traditional graph autoencoders (GAE) in predicting new links on biomedical multimodal graphs.
Instead of yielding deterministic latent representations, variational graph autoencoders (VGAE) [14] use a probabilistic approach to compute the latent variables.
In this work, we aim to use VGAE to learn the representations of drugs and proteins on a multimodal graph; then, we predict several node-pair interactions (e.g., drug-pair polypharmacy side effects, drug-cell line response, etc.)
using the learned representations.
Our contributions are two-fold as follows:
• We demonstrate that using VGAE can attain superior performance in predicting drug-drug interactions on multimodal networks, outperforming GAE-based architectures and handcrafted feature based methods.
• We leverage drug molecular representations (i.e. Morgan fingerprints) concatenated with drug latent representations before the decoding stage to further enhance the model’s performance in predicting drug-pair polypharmacy side effects.
Related work
Link prediction is one of key problems in graph learning.
In link-level tasks, the network consists of nodes and an incomplete set of edges between them, and partial information on the graph are used to infer the missing edges.
The task has a wide range of variants and applications such as knowledge graph completion [15], link prediction on citation networks [16], and protein-protein interaction prediction [17].
Graph neural networks (GNNs) are deep learning models that learn to generate representations on graph-structured data.
Most GNN-based approaches use a message passing paradigm proposed in [18] [19] wherein each node aggregates vectorized information from its neighbors and updates the messages to produce new node representations.
Kipf and Welling [9] introduced a scalable method that approximates first-order spectral graph convolutions, which are equivalent to one-hop neighborhood message passing neural networks.
Besides, Veliˇckovi´c et al.
[12] proposed graph attention networks (GATs) that use soft attention mechanism to weight the messages propagated by the neighbors.
Hamilton et al.
[20] introduced GraphSAGE that efficiently uses node attribute information to generate representations on large graphs in inductive settings.
Deep generative models aim to generate realistic samples, which should satisfy properties in nature.
Different from traditional methods that rely on hand-crafted features, data-driven approaches can be categorized into variational autoencoders (VAEs) [21], generative adversarial networks (GANs) [22], and autoregressive models.
In the field of deep generative models on graphs, several works ([23, 24] aim at adding stochasticity among latents to make the models more robust to complicated data distributions.
Simonovsky and Komodakis [25] proposed a regularized graph VAE model wherein the decoder outputs probabilistic graphs.
In addition, Bojchevski et al.
[26] introduced a stochastic neural networks that are trained by Wassterstain GAN object to learn the distribution of random walks on graphs.
For autoregressive methods, Liao et al.
[27] proposed a attention-based GNN network to generate graph nodes and associated edges after many decision steps in a generation process.
The objective of our work is to apply a probabilistic framework in predicting missing links on biomedical multimodal networks.
We extend varational graph autonecoders (VGAE)
[14] to multimodal networks such that separate posterior latent distributions are computed for each node type after several message passing and aggregation layers among different node and edge types.
Moreover, we also incorporate molecular structures into the latent variables of drug nodes to capture more inter-intra information across the medications, resulting in superior performance in drug-pair prediction.
Approach 3.1 Problem Setup
Given a multimodal graph, our task is to predict whether two nodes of the same or different types are connected or not under a specific edge type.
This can be regarded as a link-level prediction on graphs.
In our work, the graph G consists of two node types (e.g., drug and protein), and each side effect is represented as an edge type.
In addition to polypharmacy interactions, G also involves edges representing drug-protein and protein-protein interactions.
Formally, let denote G “ p V, E, X q , where V “ V d Y V p is a union of two node sets of different types (i.e. V d is the set of drug nodes and V p is the set of protein nodes), E is a set of edges, and X “ X d ‘ X p is a concatenated matrix denoting the node features of different node types.
Each edge in E is a triplet p v i , e, v j q in which node v i interacts with node v j under a specific edge type e .
In case of a binary classification setting, we perform negative sampling to generate non-edge labels to make the model robust to negative examples.
Thus, the objective is to learn a function f : E Ñ T , where f predicts the value of a particular triplet p v i , e, v j q ; T can be either t 0 , 1 u or R .
3.2 Model Architecture
Our proposed model has three main components: an encoder, a latent encoder, and a decoder.
They are described as follows:
Encoder a two-layer graph convolutional neural network operating message passing scheme [19] on G and producing node embeddings for each node type (e.g., drugs and proteins).
Each layer in the network has the following form:
[Equation]
where N ie denotes the neighbor set of node x i under the edge type e .[-W 
-]
W e P R d k ˆ d is a edge-type specific transformation matrices that map x i P R d i and its neighbors x j P R d j into d k -dimensional vector spaces, resulting in h i P R d k .
It is worth noting that d i and d j are not necessarily the same because x i and x j can be nodes of two different node types (i.e. d i “ d j when x i and x j are in the same node type). c i and c j indicates the degree node i and j on the network.
Also, ϕ denotes a nonlinear activation function; we use rectified linear unit (ReLU)
[28] in our experiments.
Figureillustrates an overview of the encoder in our framework.
Latent Encoder For each node type v , there are two multilayer perceptrons (MLPs) receiving the node embeddings from the encoder and computing the predicted mean µ and logarithm of the standard deviation log σ of the posterior latent distribution:
[Equation]
v | X, E q“ | V v | ź i “q v p z iv | X, E q [Figure] q v p z iv | X, E q“ N p z iv | µ iv , diag pp σ iv q 2 q denotes the posterior distribution of a node a type.
Here, µ v and log σ v are computed as follows:
[Equation]
where W iµ v P R d k ˆ d , W iσ v P R d k ˆ d are the weight matrices, µ v and log σ v are the matrices of mean vector µ iv and logarithm of standard deviation vector log σ iv , respectively.
Figure 2 demonstrates how we integrate molecular structures of drug nodes with their latent representation to improve the performance in side effect link prediction.
Decoder a tensor factorization using latent embeddings to predict the node interactions on G .
We follow the approach proposed in [3] to design the decoder:
[Equation]
v j are a protein and a drug, or vice versa.
where D e , R, M e P R d ˆ d are learnable parameters.
R denotes the global matrix representing all drugdrug interactions among all polypharmacy side effects; M e is a edge-type-specific matrix modeling drug-protein and protein-protein relations.
Also, D e is a diagonal matrix, and its on-diagonal entries model the significance factors of z i and z j in multiple dimensions under the side effect type e .
Finally, the probability of edge p v i , e, v j q is calculated via a sigmoid function σ :
[Equation]
Experiments
Our implementation is done with the PyTorch Geometric library [29, 30].
The code base is developed based on the implementation of VGAE published in the library’s repository.
We release our implementation at https://github.com/HySonLab/drug-interactions .
[Table]
4.1 Polypharmacy side effects
We conduct experiments on the dataset introduced in [3].
It is a multimodal graph consisting of 19, protein and 645 drug nodes.
There are three types of interaction including drug-drug, drug-protein, and protein-protein.
In particular, the dataset contains 964 commonly occurring polypharmacy side effects, resulting in 964 edge types of drug-drug interaction.
Protein-protein and drug-protein interactions are regarded as two other edge types; therefore, the multimodal graph has 966 edge types in total.
Additionally, total numbers of edges of each interaction are displayed in Table 4.
We randomly split the edges into training, validation, and testing sets with a ratio of 8:1:1 .
Then, the edges in the training edge set are randomly divided into 80% edges for message passing and 20% edges for supervision.
We use Adam optimizer [31] to minimize the loss function L showed in
Equation 1 in 300 epochs with a learning rate of 0 . 001 .
We run the experiments with six different random seeds.
[Equation]
where p v p Z v q denotes the latent space’s prior distribution of the node type v .
The first term of L denotes the cross-entropy loss of the probabilities of positive and negative edges which is sampled by choosing a random node v n for each node v i under a specific edge type e , while the second term is the weighted sum of KL-divergence of each node type v ; t λ v u are hyper-parameters.
In our experiments, λ d and λ p equal to 0 .
and 0 .
9 for drug and protein nodes, respectively.
Finally, we use one-hot representations indicating node indices as features for both drug and protein nodes.
4.2 Anticancer drug response
In addition to drug-pair polypharmacy side effects, we evaluate our approach for the anticancer drug response prediction problem; the task is to predict the interactions between drug and cell line on a multimodal network.
Integrated information between drugs and cell lines are an effective approach to calculate anticancer drug responses using computational methods; various type of information can be exploited such as drug chemical structures, gene expression profiles, etc., to provide more accurate predictions.
In this work, we evaluate the performance of VGAE on the CCLE dataset proposed in [32] which contains comprehensive information of drugs and cell lines.
Three types of pairwise similarities are provided for each modality, resulting in nine types of combination used in computation among drug and cell line pairs; in addition, pairwise logarithms of the half-maximal inhibitory concentration (IC50) scores between drugs and cell lines are given as values to be predicted.
The combinations are detailed in Table 3.
To apply VGAE in predicting the IC50 scores among drug-cell line pairs, we construct an undirected multimodal graph consisting of drug and cell line nodes with the similarity scores indicating the edge weights under three types of relation, including drug-drug, cell line-cell line, and drug-cell line.
The problem is regarded as a link prediction task in which an amount of edges connecting drug and cell line nodes are masked, and the objective is to compute real values w e denoting the weights of these missing links.
The implementation of VGAE is almost the same as detailed in 3.2 with modifications of the decoding stage in which representations of two nodes are concatenated before being preceded to a simple multilayer perceptron with two hidden layers of size 16, ReLU nonlinearity, and a linear layer on top to predict a final score denoting the edge weights among them.
We randomly split the edges with a ratio 7:1:2 for training, validation, and testing purposes, respectively.
In this task, the models are trained to reconstruct a weighted adjacency matrix and regularized by KL-divergence of the node latent distributions with hyperparameters λ
d and λ c denoting the coefficients for drug and cell nodes, respectively; λ d “ λ c “ 0 .
001 in our experiments.
We train the models in 500 epochs with a learning rate of 0.01 to minimize the loss function L as follows:
[Equation]
s e p v i , v j q ´ s e p v i , v j qq 2 ´ ÿ v λ v D KL p q v p Z v | X, E q| p v p
Z v qq (2) where p s e p v i , v j q indicates the predicted edge weights between node v i and v j , whereas s e p v i , v j q are their ground truths.
Finally, experiments are run with 20 different random seeds.
Results 5.1 Polypharmacy side effects prediction
[Table]
We compare the performance of VGAE to alternative approaches. In addition to VGAE
[Table]
we also implement a graph autoencoder and train the model in the same setting detailed in Sec. 4.
The baseline results are taken from [3] including: RESCAL Tensor Factorization [33], DEDICOM
Tensor Factorization [34], DeepWalk Neural Embeddings [35, 36], Concatenated Drug Features [Table] and
Decagon [3].
It is worth noting that Decagon is also a GAE-based model with two GCN layers, yet
Zitnik et al.
[3] use side information (e.g., side effects of individual drugs) as additional features for drug nodes.
By constrast, our VGAE and GAE are trained on one-hot representations for drug and protein nodes.
To examine the effects of Morgan fingerprints in drug-drug interaction decoding, we augment the Morgan fingerprint information into the latent vectors of drug nodes before preceding them to tensor-based decoders to produce final predictions.
Table 2 shows the results of all approaches in predicting the polypharmacy side effects.
The models are evaluated based on three metrics, including area under ROC curve (AUROC), area under precision-recall curve (AUPRC), and average precision at 50 (AP@50).
The scores reveal that
VGAE without augmenting Morgan fingerprints outperforms traditional tensor-based approaches by a large margin, resulting in up to 24 .
8% (AUROC), 32 . 0% (AUPRC), and 40 .
3% (AP@50).
We also compare VGAE with two machine-learning-based methods.
The model achieves a 25 . 8% gain over DeepWalk neural embeddings and 18 . 0% over Concatenated drug features in AP@ scores.
Furthermore, albeit trained on one-hot feature vectors, vanilla VGAE can achieve competitive performance with Decagon, outperforming the baseline by 6 . 0% (AP@50).
This indicates the effectiveness of VGAE in recommending potential side effects in a featureless drug-protein multimodal network.
Finally, VGAE with additional molecular information at the decoding stage provides the best performance across the approaches.
The method achieves the highest scores among all three metrics and outperforms the others by a large margin.
The results reveal that using molecular fingerprints is a simple yet effective solution in predicting drug-drug interactions as suggested in [37].
Visualization of side effect embeddings To further demonstrate the capability of VGAE in learning to model drug-pair polypharmacy side effects, we plot their representations in 2D dimension using the t-SNE [38] method.
The embedding of each side effect is derived by taking on-diagonal entries of the tensor D e to create a d -dimensional vector which is accordingly projected into 2D space.
Figure 3 illustrates a vector space in which side effects’ representations establish a clustering structure.
5.2 Anticancer drug response prediction
In this task, the models are evaluated using four criteria including root mean square error (RMSE), coefficient of determination ( R 2 ), Pearson correlation coefficient (PCC), and fitness which is computed as:
[Equation]
Table 3 shows the experimental results of drug response prediction on nine combinations of pairwise similarities of drugs and cell lines.
VGAE trained on copy number variation with KEGG-pathways as similarity scores for cell lines and drugs achieve the best performance, yielding the lowest RMSE at 0.46 ˘ 0.02 and highest scores in R 2 = 0.67 ˘ 0.03, PCC = 0.85 ˘ 0.01, and fitness = 1.05 ˘ 0.06.
Moreover, Table 4 demonstrates the comparisons between VGAE and other baselines which are taken from [32] including CDRscan [39], CDCN [40], SRMF [41], CaDRRes [42], ADRML [32] [Table] and k-nearest neighbors (KNN).
The results reveal that compared with the baselines, VGAE has competitive performance, especially in the fitness score determining the effectiveness of the approaches in the drug response prediction task
[Table]
[Table]
[Table]
Conclusion
In this work
[Table]
we evaluate the effectiveness of variational graph autoencoders in predicting potential polypharmacy side effects on multimodal networks.
The results reveal that VGAE trained on one-hot feature vectors outperforms other approaches.
Moreover, augmenting Morgan fingerprints before the decoding stage helps boost the performance of VGAE.
This suggests further examination of the use of molecular fingerprints in drug-drug interaction problems.