GENN: Predicting Correlated Drug-drug Interactions with Graph Energy Neural Networks

1 INTRODUCTION . The use of drug combinations is common and often necessary for treating patients with complex diseases . However , it also increases the risk of drug-drug interactions ( DDI ) . DDIs are pharmacological interactions between drugs that can alter the action of either or both drugs and cause adverse effects . Overall DDIs result in a large number of fatalities per year and incur ∼ $ 177 billion DDI associated cost annually ( Giacomini et al. , 2007 ) . To mitigate these risks and costs , accurate prediction of DDI becomes a clinically important task . While DDI knowledge is expensive to gather , several deep learning approaches have been proposed to search the large biomedical data for predicting potential DDIs ( Zitnik et al. , 2018 ; Ma et al. , 2018 ; Ryu et al. , 2018 ) . Among them , graph neural networks ( GNN ) that consider DDI prediction in the graph setting have obtained great performance . In DDI graphs , drugs are represented as nodes and embedded into a low-dimensional space . Further prediction can be made based on casting DDI prediction as a link prediction problem where DDI types are represented as link types ( Kipf & Welling , 2016 ; Zitnik et al. , 2018 ) . However , existing works rarely exploit the correlations between these link types ( e.g. , DDI types ) despite of their importance . For example , in Fig . 1 Warfarin is a drug for treating blood clots . Its combined use with antibiotic or nonsteroidal anti-inflammatory drugs can cause multiple DDI types including nhibition of clotting , gastrointestinal bleeding , and hemorrhage . These DDI types are correlated since they are all bleeding related DDIs caused by the increase of Warfarin ’ s effect . Explicit modeling such correlations can help infer unseen DDI types . To fill the gap , we propose GENN , a new deep architecture that predicts correlated DDIs based on graph neural networks and energy based models . Specifically , we leverage the dependency structures among DDI types and formulate multi-type DDI detection as a structure prediction problem ( Lafferty et al. , 2001a ; LeCun et al. , 2006 ; Belanger & McCallum , 2016b ) . Here we use an energy based approach to incorporate such dependency structures . To summarize , GENN is enabled by the following technical contributions . 1 . Modeling link type correlations . GENN bridges graph neural networks and structure prediction to directly capture link type correlations for more accurate link prediction in graphs through minimizing an energy function . 2 . A new graph neural network based energy function . Inspired by a family of structure prediction model called structured prediction energy networks ( SPENs ) ( Belanger & McCallum , 2016a ) , we design a new graph energy function based on graph neural networks to capture the global dependencies among link types in DDI graphs . This is more flexible and expressive than previous approaches which used CRF-type energy definitions Qu et al . ( 2019 ) . 3 . Efficient semi-supervised training . We designed one cost-augmented inference network to approximate the output in training and one test inference network to approximate the output in testing . Different from the previous inference method for SPENs , we propose a new semi-supervised joint optimization procedure to adapt to our problem on graphs . The new inference method optimizes the training-set structured hinge loss together with the test-set energy with respect to the parameters for both inference networks and the energy function . We evaluated GENN on two real world DDI datasets with both quantitative and qualitative study . Results demonstrated GENN outperformed basic graph neural networks with 13.77 % and 5.01 % PR-AUC improvement on the two datasets , respectively . 2 RELATED WORK . 2.1 DDI PREDICTION . To predict unseen DDIs based on known ones , drug similarity has been learned via nearest neighbor approaches ( Zhang et al. , 2017 ) , and using random walk methods including label propagation ( Zhang et al. , 2015 ; Wang et al. , 2010 ) , and unsupervised methods ( Wang et al. , 2014 ; Angione et al. , 2016 ) . Recently , deep graph neural networks have been demonstrated to provide much improved performance in DDI prediction . Among them , ( Ma et al. , 2018 ) integrates different sources of drug-related information with heterogeneous formats into an coherent and informationpreserving representation using attentive multi-view graph auto-encoders ( Kipf & Welling , 2016 ) . Decagon ( Zitnik et al. , 2018 ) develops a new graph auto-encoder approach , which allows us to develop an end-to-end trainable model for link prediction on a multi-modal graph . As the experimental results in Zitnik et al . ( 2018 ) show graph neural networks achieved significantly improved performance than shallow models such as tensor decomposition ( Nickel et al. , 2011 ; Papalexakis et al. , 2017 ) , random walk based methods ( Perozzi et al. , 2014 ; Zong et al. , 2017 ) , and non-graph neural fingerprinting ( Duvenaud et al. , 2015 ; Jaeger et al. , 2018 ) , DeepDDI ( Ryu et al. , 2018 ) . None of the existing DDI prediction works consider the correlation among multiple DDI types . 2.2 STRUCTURE PREDICTION AND ENERGY BASED MODELS . Structure prediction is an important problem in various application domains where we want to predict structured outputs instead of independent labels , e.g. , structured labels prediction for object detection ( Zheng et al. , 2015b ) , semantic structure prediction ( Belanger et al. , 2017 ) or part-ofspeeches tagging ( Ma & Hovy , 2016 ) . For structure prediction problems , feed-forward networks is insufficient since they can not directly model the interaction or constraints of the outputs . Instead , energy based models ( LeCun et al. , 2006 ) provide one way to address the challenge by defining a much more flexible and expressive energy score , and the prediction is conducted by minimizing the energy function . One of the well-known structure prediction approaches is the conditional random fields ( CRF ) ( Lafferty et al. , 2001b ) which show success in application areas such as named entity recognition ( Sato et al. , 2017 ) and image segmentation ( Zheng et al. , 2015a ) . But as a structured linear model , CRF has limited representation ability . To enhance the flexibility of structure prediction models , Belanger & McCallum ( 2016b ) proposed Structured Prediction Energy Networks ( SPENs ) which uses arbitrary neural networks to define the energy function and optimizes the energy over a continuous relaxation of labels . Thus it can capture high-arity interaction among output labels and approximate the optimization of energy function via gradient descent and repeated “ loss-augmented inference ” . Belanger et al . ( 2017 ) further developed an “ end-to-end ” method that unroll the approximate energy optimization to a different computation graph . However , after learning the energy function , they still have to use gradient descent for test-time inference . Later , Tu & Gimpel ( 2018 ) replaced the gradient descent with a neural network trained to do inference directly . However , it separates the training of cost-augmented inference network and the fine-tuning of another inference network for testing , which makes it inefficient . In addition , how to optimize the SPENs on graphs for semi-supervised learning remains challenging . In this paper our method improves the algorithm of Tu & Gimpel ( 2018 ) to solve this problem . Very recently , GMNN ( Qu et al. , 2019 ) combines the statistical relational learning ( SRL ) and graph neural networks . It includes a CRF to model the joint distribution of object labels conditioned on object attributes , and then it is trained with a variational EM algorithm . The model has been used for node classification and link classification . However , it can not be easily extended for prediction of missing links , since they have to use links as nodes in a “ dual graph ” for link classification , but the missing links generally can not be modeled as nodes . 3 PRELIMINARY . In this section , we first summarize our task in Section 3.1 , then a basic message passing graph neural network ( MPNN ) is described for embedding the DDI graph in Section 3.2 . 3.1 TASK FORMULATION . Definition 1 ( DDI Graph ) Given a DDI graph G = { V , E } , V is the node set which contains N drug nodes with node features X ∈ RN×D , and E is the edge set which contains all DDIs between drug pairs . For a specific drug pair xi ∈ RD , xj ∈ RD , the DDIs of this pair ei , j can have multiple types , i.e . ei , j ∈ { 0 , 1 } L , where D is the feature dimension and L is the total number of DDI types . And these DDI vectors e form the edge features ( or labels ) Y ∈ { 0 , 1 } |E|×L . Problem 1 ( DDI Prediction ) We cast the DDI prediction into a multi-type link prediction problem . We assume there are some missing edges YU in the graph , i.e . Y = { YL , YU } . Given the node features X and some known DDI links YL , the goal is to predict the unknown DDIs YU . 3.2 MESSAGE PASSING NEURAL NETWORKS FOR DDI PREDICTION . Recently graph neural networks ( GNNs ) have been successfully applied to DDI prediction ( Zitnik et al. , 2018 ; Ma et al. , 2018 ) . To predict the unknown DDIs in a graph , a GNN first learns the node embeddings of all the drug nodes and then make prediction based on these embedding . In this section , we describe a basic message passing based graph neural network ( Gilmer et al. , 2017 ) for drug node embedding . Considering that our graph contains different types of edges , we select a schema including edge information in the message passing . Given the graph G with node features X , for each node with hidden state hti and its neighborhood N ( i ) , we define the message passing layer to update the node hidden state as follows : ht+1i = Wsh t i + ∑ j∈N ( i ) htj · f ( ei , j ) ; ∀i ∈ V , h0i = W0xi ( 1 ) where ei , j ∈ RL is the edge ( DDI type vector ) between nodes xi , xj ∈ RD , Ws ∈ RM×M , W0 ∈ RM×D are both learnable weight , and f ( · ) ∈ RM×M is a neural network , i.e . MLP . The message passing layer can then be stacked to get the final state hi for each node , i.e . the node embedding . Then we can use these final states to predict the probability of DDI types for each edge : ŷi , j = Sigmoid ( W [ hi||hj ] + b ) ( 2 ) where || indicates concatenation , W ∈ RL×2M and b ∈ R2M are parameters and Sigmoid is the activation function commonly used for multi-label prediction . The model is trained on the partially known DDI edges YL by optimizing cross entropy loss LCE ( ŶL , YL ) = ∑ ei , j∈YL LCE ( ŷi , j , ei , j ) between the predictions and the true edges . After training , it can be directly used for prediction of unknown DDIs YU . 4 THE GENN METHOD We present our GENN framework in this section ( Fig . 2 ) . We first discuss the design of energy based framework for DDI prediction and the design of our graph energy functions based on MPNN . Then we propose a joint learning strategy with a semi-supervised learning setting in Section 4.3 . Obviously the aforementioned message passing neural network does not explicitly consider the DDI correlation in the prediction . However , in practice , the labels are often correlated as we stated in the introduction section . So we re-formulate our problem as a structure prediction problem and infer the labels by optimizing an energy function . In order to get extremely powerful representation for the energy model , we follow the ideas of SPENs to formulate the energy function as a neural network .

Summary: This paper presents a graph neural network for drug-to-drug interaction (DDI) prediction, which explicitly models link type correlation. Basically, the drug-to-drug interaction prediction problem is a specific type of link prediction task, with drugs as vertices and interaction as edges, and the authors propose a graph neural network with an energy-based formulation where the link types are encoded as the graph edges. The authors validate their method against feedforward GNNs on two DDI prediction datasets, and achieve significantly improved performances.