# A Knowledge Graph Embedding Approach for Polypharmacy Side Effects Prediction
## 2 Methods
### Learning Objective and Loss Function  
The common goal of learning in the embedding and prediction modules was to predict the side effect label as accurately as possible. For this purpose, the occurrences of side effects induced by multidrug combinations were used as labels for learning. To calculate the learning loss, we used a binary cross entropy function. Another goal of learning in the embedding module was to appropriately reflect the connectivity strength (\(cs\)) between a hypothetical multidrug node and its related proteins shown in the knowledge graph. To this end, we assumed that the \(cs\) between the multidrug combination \(md\) and the protein \(p\) in the graph is the largest \(cs\) value between the constituent drugs of \(md\) and protein \(p\) in the initial embedding space. It is to place importance on and preserve the information about the strongest influence that \(md\) can have on \(p\). Specifically, the \(cs\) between \(md\) and \(p\) was calculated as follows:  
Figure 2: Overall structure of the proposed method for multidrug side effects prediction. For a given multidrug, the model yields an embedding vector representing this multidrug and generates its prediction scores for k targeted side effects from this vector. Initial embedding space:  
\[cs^{I}(md,\ p)=\max_{d\in md}\mathit{CosSim}\Big{(}v_{d}^{I},v_{p}^{I}\Big{)} \tag{1}\]  
Unified embedding space:  
\[cs^{U}(md,\ p)=\mathit{CosSim}\Big{(}v_{md}^{U},v_{p}^{U}\Big{)} \tag{2}\]  
where \(\mathit{CosSim}\) is the cosine similarity, \(v_{a}^{I}\) is the initial embedding vector for a node \(a\), \(d\) is a drug constituting \(md\), and \(v_{x}^{U}\) is a unified embedding vector representing \(x\). \[\sum_{md\in\mathit{MD}_{labeled}}\mathit{BCE}\big{(}L_{md},\hat{L}_{md}\big{)} +\alpha\sum_{md\in MD_{labeled}\cup MD_{unlabeled}}\mathit{MSD}(md,P) \tag{3}\]  
The final loss function of the proposed model is thus given as follows: where \(MD_{labeled}\) is a set of labeled multidrug combinations, \(MD_{unlabeled}\) is a set of unlabeled virtually generated multidrug combinations, \(\mathit{BCE}\) is the binary cross entropy loss function, \(L_{md}\) is the list of the actual labels of \(md\), \(\hat{L}_{md}\) is a list of prediction scores for \(md\), \(\alpha\) is a hyperparameter, and \(p\) is a protein set. \(\mathit{MSD}(md,P)\) is the mean squared difference of connectivity strengths from \(md\) to proteins in the initial and unified embedding spaces, which was calculated by  
\[\mathit{MSD}(md,\ P)=\frac{1}{|P|}\sum_{p\in P}\Big{(}cs^{I}(md,\ p)-cs^{U}(md,\ p)\Big{)}^{2}. \tag{4}\]  
Here, \(\mathit{MSD}\) serves as a regularized term for calculating the final loss. In the previous expression, \(\alpha\) indicates the role of tuning the impact of the regularization term. To strengthen the generalizability of the prediction model, virtually generated multidrug combinations were additionally considered for regularization. All embedding vectors of multidrug combinations (including single drugs), virtual multidrug combinations, and proteins were obtained by minimizing the loss function. Thus, these vectors are present in the same embedding space. The learning process for our proposed model is summarized in Algorithm 1. ```
\(G\): The knowledge graph consisting of drug and protein nodes. \(Md\): A multidrug (i.e., a set of one or more drugs). \(MD_{labeled}\): Training data of multidrug combinations labeled with side effects. \(MD_{unlabeled}\): Training data of multidrug combinations virtually generated. \(v_{a}^{I}\): An initial embedding vector of a node \(a\). \(v_{md}^{I}\): A set of initial embedding vectors for given nodes. \(v_{md}^{I}\): A unified embedding vector of a multidrug \(md\). \(L_{md}\): A set of side effect labels of a multidrug \(md\). \(EmbNN\): Neural networks of embedding module. PredNN: Neural networks of prediction module.