Predicting Drug-Drug Interactions using Deep Generative Models on Graphs
Anonymous Author(s)
AffiliationAddress
Abstract
Latent representations of drugs and their targets produced by contemporary graphautoencoder-based models have proved useful in predicting many types of node-pairinteractions on large networks, including drug-drug, drug-target, and target-targetinteractions.
However, most existing approaches model the node’s latent spacesin which node distributions are rigid and disjoint; these limitations hinder themethods from generating new links among pairs of nodes.
In this paper, we presentthe effectiveness of variational graph autoencoders (VGAE) in modeling latentnode representations on multimodal networks.
Our approach can produce flexiblelatent spaces for each node type of the multimodal graph; the embeddings are usedlater for predicting links among node pairs under different edge types.
To furtherenhance the models’ performance, we suggest a new method that concatenates
Morgan fingerprints, which capture the molecular structures of each drug, with theirlatent embeddings before preceding them to the decoding stage for link prediction.
Our proposed model shows competitive results on two multimodal networks: (1) amulti-graph consisting of drug and protein nodes, and (2) a multi-graph consistingof drug and cell line nodes.
Introduction
Investigations on biomedical multimodal graphs consisting of drugs, proteins, or cells and theirinterrelations are essential in drug repurposing as they can help discover innovative node-pairinteractions, which may be helpful in medical treatments, on these networks.
Most approaches totackling drug repurposing problems estimate drug side effects and drug responses to cell lines toprovide important information about the use of those drugs.
In particular, polypharmacy is the practiceof treating complex diseases or co-existing health conditions by using combinations of multiplemedications [1].
Drug combinations consisting of many drugs affecting distinct targeted proteinscan effectively modulate the process of severe diseases by facilitating the diverse biological processof different proteins [2].
Albeit commonly applied, polypharmacy is one of the major underlyingissues that cause adverse medical outcomes, also known as side effects caused by drug combinations[3].
Understanding drug-drug interactions (DDIs) is pivotal in predicting the potential side effects ofmultiple co-administered medications.
Moreover, accurate estimation of drug responses to targetedcell lines is another major factor in providing precision medicines to patients.
However, it is unfeasibleto render clinical testing of all drug combinations due to the tremendous number of relations betweendrug pairs or drugs and their targets (e.g., proteins).
Machine learning models have been widely applied to provide tractable solutions in predictingpotential drug interactions.
Data-driven techniques (e.g., machine learning or deep learning) haveproved their capabilities for dealing with complex patterns in data.
Such methods have producedremarkable results in various fields, such as computer vision, natural language processing, and audioprocessing.
Therefore, it is reasonable to employ machine learning approaches to investigate theinteractions among complex combinations of medications.
Previous work on DDIs [4–6] used varioushand-crafted drug features (e.g., chemical properties or Morgan fingerprints) to compute the similarityof each drug pair, and drug-drug interactions can be predicted based on the drug-pair proximity scores.
Nevertheless, using fixed drug representations can result in sub-optimal outcomes since they do notsufficiently capture the complex interrelations of drugs.
Recent approaches rely on graph neural networks (GNNs) [7] to directly learn drug and proteinrepresentations from structural data (i.e. graphs) augmented with additional information for nodefeatures.
Zitnik et al.
[3] proposed a graph autoencoder to predict polypharmacy side effects ona multimodal graph consisting of drug and protein nodes with various edge types.
Similarly, Yinet al.
[8] introduced DeepDrug combining graph convolutional neural networks (GCN) [9] andconvolutional neural networks (CNN)[10] to learn the structural and sequential representations ofdrugs and proteins.
In addition, Wang et al.
[11] suggested using several graph attention (GAT)[12] layers to learn on hand-crafted features of drugs to predict drug-pair interactions.
Nyamaboet al.
[13] use multiple GAT layers to compute the drug representations and aggregate them usingco-attention to produce final predictions.
These mentioned GNN-based models effectively learn thelatent representations of drugs and proteins, which can be used in several downstream tasks.
Predicting drug-drug or drug-target interactions can be regarded as a link prediction task on graphs.
We argue that latent spaces produced by the aforementioned approaches are disjoint and not interpretable; hence, they are not capable of generating new links on some graph benchmarks.
In thispaper, we suggest using deep graph generative models to make the latent spaces continuous and moreinterpretable; as a result, they are superior to traditional graph autoencoders (GAE) in predictingnew links on biomedical multimodal graphs.
Instead of yielding deterministic latent representations,variational graph autoencoders (VGAE) [14] use a probabilistic approach to compute the latentvariables.
In this work, we aim to use VGAE to learn the representations of drugs and proteins on amultimodal graph; then, we predict several node-pair interactions (e.g., drug-pair polypharmacy sideeffects, drug-cell line response, etc.)
using the learned representations.
Our contributions are two-fold as follows:
• We demonstrate that using VGAE can attain superior performance in predicting drug-drug
interactions on multimodal networks, outperforming GAE-based architectures and
hand-
crafted feature based methods.
• We leverage drug molecular representations (i
e. Morgan fingerprints) concatenated with
drug latent representations before the decoding stage to further enhance the model’s perfor-
mance in predicting drug-pair polypharmacy side effects.
Related work
Link prediction is one of key problems in graph learning.
In link-level tasks, the network consists ofnodes and an incomplete set of edges between them, and partial information on the graph are used toinfer the missing edges.
The task has a wide range of variants and applications such as knowledgegraph completion [15], link prediction on citation networks [16], and protein-protein interactionprediction [17].
Graph neural networks (GNNs) are deep learning models that learn to generate representations ongraph-structured data.
Most GNN-based approaches use a message passing paradigm proposedin [18] [19] wherein each node aggregates vectorized information from its neighbors and updatesthe messages to produce new node representations.
Kipf and Welling [9] introduced a scalablemethod that approximates first-order spectral graph convolutions, which are equivalent to one-hop neighborhood message passing neural networks.
Besides, Veliˇckovi´c et al.
[12] proposed graphattention networks (GATs) that use soft attention mechanism to weight the messages propagatedby the neighbors.
Hamilton et al.
[20] introduced GraphSAGE that efficiently uses node attributeinformation to generate representations on large graphs in inductive settings.
Deep generative models aim to generate realistic samples, which should satisfy properties in nature.
Different from traditional methods that rely on hand-crafted features, data-driven approaches can becategorized into variational autoencoders (VAEs) [21], generative adversarial networks (GANs) [22],and autoregressive models.
In the field of deep generative models on graphs, several works ([23, 24]aim at adding stochasticity among latents to make the models more robust to complicated datadistributions.
Simonovsky and Komodakis [25] proposed a regularized graph VAE model whereinthe decoder outputs probabilistic graphs.
In addition, Bojchevski et al.
[26] introduced a stochasticneural networks that are trained by Wassterstain GAN object to learn the distribution of random walkson graphs.
For autoregressive methods, Liao et al.
[27] proposed a attention-based GNN network togenerate graph nodes and associated edges after many decision steps in a generation process.
The objective of our work is to apply a probabilistic framework in predicting missing links on biomedical multimodal networks.
We extend varational graph autonecoders (VGAE)
[14] to multimodalnetworks such that separate posterior latent distributions are computed for each node type after severalmessage passing and aggregation layers among different node and edge types.
Moreover, we alsoincorporate molecular structures into the latent variables of drug nodes to capture more inter-intrainformation across the medications, resulting in superior performance in drug-pair prediction.
Approach
3.1 Problem Setup
Given a multimodal graph, our task is to predict whether two nodes of the same or different types areconnected or not under a specific edge type.
This can be regarded as a binary classification problemon the graph’s edges.
In our work, the graph G consists of two node types (e.g., drug and protein),and each side effect is represented as an edge type.
In addition to polypharmacy interactions, Galso involves edges representing drug-protein and protein-protein interactions.
Formally, let denote
G “ p V, E, X q , where V “ V d Y V p is a union of two node sets of different types (i.e. V d is theset of drug nodes and V p is the set of protein nodes), E is a set of edges, and X “ X d ‘ X p is aconcatenated matrix denoting the node features of different node types.
Each edge in E is a tripletp v i , e, v j q in which node v i interacts with node v j under a specific edge type e .
As our task is in abinary classification setting, we perform negative sampling to generate non-edge labels to make themodel robust to negative examples.
Thus, the objective is to learn a function f : E ˆ E Ñ t 0 , 1 u ,where f predicts the label of a particular triplet p v i , e, v j q .
3.2 Model Architecture
Our proposed model has three main components: an encoder, a latent encoder, and a decoder.
Theyare described as follows:
Encoder a two-layer graph convolutional neural network operating message passing scheme[19, 20] on G and producing node embeddings for each node type (e.g., drugs and proteins).
Eachlayer in the network has the following form:
[Equation]
˘ where N ie denotes the neighbor set of node x i under the edge type e .
W
e P R
d
k ˆ d 1 , W 2 e P R
d k ˆ d 2 aretwo edge-type specific transformation matrices that map x i P R d 1 and its neighbors x j P R d 2 into d kdimensional vector spaces, resulting in h i P R d k .
It is worth noting that d 1 and d 2 are not necessarilythe same because x i and x j can be nodes of two different node types.
Also, ϕ denotes a nonlinear
[Figure]
activation function; we use rectified linear unit (ReLU)
[28] in our experiments.
Figure 1 illustratesan overview of the encoder in our framework.
Latent Encoder For each node type v , there are two multilayer perceptrons (MLPs) receivingthe node embeddings from the encoder and computing the predicted mean µ and logarithm of thestandard deviation log σ of the posterior latent distribution:
[Equation]
q v p z iv | X, E q“ N p z iv | µ iv , diag pp σ iv q 2 q denotes the posterior distribution of a node of a specific nodetype.
Here, µ v and log σ v are computed as follows:
[Equation]
[Equation]
where W iµ v P R d k ˆ d , W iσ v P R d k ˆ d are the weight matrices, µ v and log σ v are the matrices of meanvector µ iv and logarithm of standard deviation vector log σ iv , respectively.
Figure 2 demonstrateshow we integrate molecular structures of drug nodes with their latent representation to improve theperformance in side effect link prediction.
Decoder a tensor factorization using latent embeddings to predict the node interactions on G .
Wefollow the approach proposed in [3] to design the decoder:
[Equation]
where D e , R, M e P R d ˆ d are learnable parameters.
R denotes the global matrix representing all drugdrug interactions among all polypharmacy side effects; M e is a edge-type-specific matrix modeling drug-protein and protein-protein relations.
Also, D e is a diagonal matrix, and its on-diagonal entriesmodel the significance factors of z i and z j in multiple dimensions under the side effect type e .
Finally,the probability of edge p v i , e, v j q is calculated via a sigmoid function σ :
[Equation]
Experiments
Our implementation is done with the PyTorch Geometric library [29, 30].
The code base is developedbased on the implementation of VGAE published in the library’s repository.
4.1 Polypharmacy side effects
We conduct experiments on the dataset introduced in [3].
It is a multimodal graph consisting of 19,085protein and 645 drug nodes.
There are three types of interaction including drug-drug, drug-protein,and protein-protein.
In particular, the dataset contains 964 commonly occurring polypharmacyside effects, resulting in 964 edge types of drug-drug interaction.
Protein-protein and drug-proteininteractions are regarded as two other edge types; therefore, the multimodal graph has 966 edge typesin total.
Finally, the network consists of 4,651,131 drug-drug, 18,596 drug-protein, and 715,612protein-protein interactions.
We randomly split the edges into training, validation, and testing sets with a ratio of 8:1:1 .
Then,the edges in the training edge set are randomly divided into 80% edges for message passing and20% edges for supervision.
We use Adam optimizer [31] to minimize the loss function L showed in
Equation 1 in 300 epochs with a learning rate of 0 . 001 .
We run the experiments with six differentrandom seeds.
[Equation]
where p v p Z v q denotes the latent space’s prior distribution of the node type v .
The first term of Ldenotes the cross-entropy loss of the probabilities of positive and negative edges which is sampled bychoosing a random node v n for each node v i under a specific edge type e , while the second term is theweighted sum of KL-divergence of each node type v ; t λ v u are hyper-parameters.
In our experiments, λ d and λ p equal to 0 .
and 0 .
9 for drug and protein nodes, respectively.
Finally, we use one-hotrepresentations indicating node indices as features for both drug and protein nodes.
4.2 Anticancer drug response
In addition to drug-pair polypharmacy side effects, we evaluate our approach for the anticancer drugresponse prediction problem; the task is to predict the interactions between drug and cell line on amultimodal network.
Integrated information between drugs and cell lines are an effective approachto calculate anticancer drug responses using computational methods; various type of informationcan be exploited such as drug chemical structures, gene expression profiles, etc., to provide moreaccurate predictions.
In this work, we evaluate the performance of VGAE on the CCLE datasetproposed in [32] which contains comprehensive information of drugs and cell lines.
Three types ofpairwise similarities are provided for each modality, resulting in nine types of combination used incomputation among drug and cell line pairs; in addition, pairwise logarithms of the half-maximalinhibitory concentration (IC50) scores between drugs and cell lines are given as values to be predicted.
The combinations are detailed in Table 2.
To apply VGAE in predicting the IC50 scores among drug-cell line pairs, we construct an undirectedmultimodal graph consisting of drug and cell line nodes with the similarity scores indicating the edgeweights under three types of relation, including drug-drug, cell line-cell line, and drug-cell line.
Theproblem is regarded as a link prediction task in which an amount of edges connecting drug and cellline nodes are masked, and the objective is to compute real values w e denoting the weights of thesemissing links.
The implementation of VGAE is almost the same as detailed in 3.2 with modificationsof the decoding stage in which representations of two nodes are concatenated before being precededto a simple multilayer perceptron with two hidden layers of size 16, ReLU nonlinearity, and a linearlayer on top to predict a final score denoting the edge weights among them.
We randomly split the edges with a ratio 7:1:2 for training, validation, and testing purposes, respectively.
In this task, the models are trained to reconstruct a weighted adjacency matrix and regularizedby KL-divergence of the node latent distributions with hyperparameters λ
d and λ c denoting thecoefficients for drug and cell nodes, respectively; λ d “ λ c “ 0 .
001 in our experiments.
We train themodels in 500 epochs with a learning rate of 0.01 to minimize the loss function L as follows:
[Equation]
where p s e p v i , v j q indicates the predicted edge weights between node v i and v j , whereas s e p v i , v j q aretheir ground truths.
Finally, experiments are run with 20 different random seeds.
Results
5.1 Polypharmacy side effects prediction
[Table]
fingerprints
[Table]
We compare the performance of VGAE to alternative approaches.
In addition to VGAE, we alsoimplement a graph autoencoder and train the model in the same setting detailed in Sec. 4.
Thebaseline results are taken from [3] including: RESCAL Tensor Factorization [33], DEDICOM
Tensor Factorization [34], DeepWalk Neural Embeddings [35, 36], Concatenated Drug Features, and
Decagon [3].
It is worth noting that Decagon is also a GAE-based model with two GCN layers, yet
Zitnik et al.
[3] use side information (e.g., side effects of individual drugs) as additional features fordrug nodes.
By constrast, our VGAE and GAE are trained on one-hot representations for drug andprotein nodes.
To examine the effects of Morgan fingerprints in drug-drug interaction decoding, weaugment the Morgan fingerprint information into the latent vectors of drug nodes before precedingthem to tensor-based decoders to produce final predictions.
Table 1 shows the results of all approaches in predicting the polypharmacy side effects.
Themodels are evaluated based on three metrics, including area under ROC curve (AUROC), areaunder precision-recall curve (AUPRC), and average precision at 50 (AP@50).
The scores reveal that
VGAE without augmenting Morgan fingerprints outperforms traditional tensor-based approachesby a large margin, resulting in up to 23 .
4% (AUROC), 36 .
1% (AUPRC), and 75 .
4% (AP@50).
We also compare VGAE with two machine-learning-based methods.
The model achieves a 26 .
7%gain over DeepWalk neural embeddings and 17 .
2% over Concatenated drug features in AP@50scores.
Furthermore, albeit trained on one-hot feature vectors, vanilla VGAE can achieve competitiveperformance with Decagon, outperforming the baseline by 4 . 0% (AP@50).
This indicates theeffectiveness of VGAE in recommending potential side effects in a featureless drug-protein multimodal network.
Finally, VGAE with additional molecular information at the decoding stage providesthe best performance across the approaches.
The method achieves the highest scores among allthree metrics and outperforms the others by a large margin.
The results reveal that using molecularfingerprints is a simple yet effective solution in predicting drug-drug interactions as suggested in [37].
Visualization of side effect embeddings To further demonstrate the capability of VGAE in learningto model drug-pair polypharmacy side effects, we plot their representations in 2D dimension using the t-SNE [38] method.
The embedding of each side effect is derived by taking on-diagonal entries ofthe tensor D e to create a d -dimensional vector which is accordingly projected into 2D space.
Figure3 illustrates a vector space in which side effects’ representations establish a clustering structure.
5.2 Anticancer drug response prediction
In this task, the models are evaluated using four criteria including root mean square error (RMSE), coefficient of determination ( R 2 ), Pearson correlation coefficient (PCC), and fitness which is computedas:
[Equation]
Table 2 shows the experimental results of drug response prediction on nine combinations of pairwisesimilarities of drugs and cell lines.
VGAE trained on copy number variation with KEGG-pathwaysas similarity scores for cell lines and drugs achieve the best performance, yielding the lowest RMSEat 0.46 ˘ 0.02 and highest scores in R 2 = 0.67 ˘ 0.03, PCC = 0.85 ˘ 0.01, and fitness = 1.05 ˘0.06.
Moreover, Table 3 demonstrates the comparisons between VGAE and other baselines whichare taken from [32] including CDRscan [39], CDCN [40], SRMF [41], CaDRRes [42], ADRML[32], and k-nearest neighbors (KNN).
The results reveal that compared with the baselines, VGAEhas competitive performance, especially in the fitness score determining the effectiveness of theapproaches in the drug response prediction task.
[Table] [Table]
Conclusion
In this work
[Table]
we evaluate the effectiveness of variational graph autoencoders in predicting potentialpolypharmacy side effects on multimodal networks.
The results reveal that VGAE trained on one-hotfeature vectors outperforms other approaches.
Moreover, augmenting Morgan fingerprints before thedecoding stage helps boost the performance of VGAE.
This suggests further examination of