Predicting Drug Repurposing Candidates and Their Mechanisms from A Biomedical Knowledge Graph
Authors: Chunyu Ma
Zhihan Zhou
Han Liu
David Koslicki
Venue: Submitted to ICLR 2023 (2023)
Keywords: Drug Repurposing
Reinforcement Learning
Biomedical Knowledge Graph

Abstract: Computational drug repurposing is a cost- and time-efficient method to identify new indications of approved or experimental drugs/compounds. It is especially critical for emerging and/or orphan diseases due to its cheaper investment and shorter research cycle compared with traditional wet-lab drug discovery approaches. However, the underlying mechanisms of action between repurposed drugs and their target diseases remain largely unknown, which is still an unsolved issue in existing repurposing methods. As such, computational drug repurposing has not been widely adopted in clinical settings. In this work, based on a massive biomedical knowledge graph, we propose a computational drug repurposing framework that not only predicts the treatment probabilities between drugs and diseases but also predicts the path-based, testable mechanisms of action (MOAs) as their biomedical explanations. Specifically, we utilize the GraphSAGE model in an unsupervised manner to integrate each entity’s neighborhood information and employ a Random Forest model to predict the treatment probabilities between pairs of drugs and diseases. Moreover, we train an adversarial actor-critic reinforcement learning model to predict the potential MOA for explaining drug purposing. To encourage the model to find biologically reasonable paths, we utilize the curated molecular interactions of drugs and a PubMed-publication-based concept distance to extract potential drug MOA paths from the knowledge graph as ”demonstration paths” to guide the model during the process of path-finding. Comprehensive experiments and case studies show that the proposed framework outperforms state-of-the-art baselines in both predictive performance of drug repurposing and explanatory performance of recapitulating human-curated DrugMechDB-based paths.

Under review as a conference paper at ICLR 2023
PREDICTING DRUG REPURPOSING CANDIDATES AND THEIR MECHANISMS FROM A BIOMEDICAL KNOWL- EDGE GRAPH
Anonymous authors Paper under double-bling review
ABSTRACT
Computational drug repurposing is cost- and time-efficient method to iden- tify new indications of approved or experimental drugs/compounds. cially critical for emerging and/or orphan diseases due to its cheaper investment and shorter research cycle compared with traditional wet-lab drug discovery ap- proaches. However, the underlying mechanisms of action between repurposed drugs and their target diseases remain largely unknown, which still an un- solved issue in existing repurposing methods. As such, computational drug re- purposing has not been widely adopted in clinical settings. In this work, based on a massive biomedical knowledge graph, we propose computational drug re- purposing framework that not only predicts the treatment probabilities between drugs and diseases but also predicts the path-baseds testable mechanisms of action (MOAs) as their biomedical explanations. Specifically, we utilize the GraphS model an unsupervised manner to integrate each entity's s neighborhood infor- mation and employ Random Forest model to predict the treatment probabilities between pairs of drugs and diseases. Morcover, we train an adversarial actor- critic reinforcement learning model to predict the potential MOA for explaining drug purposing. To encourage the model to find biologically reasonable paths, we utilize the curated molecular interactions of drugs and based concept distance to extract potential drug MOA paths from the knowledge graph "demonstration paths" to guide the model during the process of path- finding. Comprehensive experiments and case studies show that the proposed framework outperforms of-the-art baselines in both predictive performance of drug repurposing and explanatory performance of recapitulating DrruMMecDDB-ben paths.
is espe-
INTRODUCTION
Traditional drug development is a time-consuming process (from initial chemical identification to clinical trials and finally FDA approval) that takes around 10-15 years and also comes along with billions-of-dolle investments and high failure rates (Berdigaliyev & Aljofan 2020). Considering the rapid pace of novel disease evolution, is urgent find more efficient and economical drug discovery method. Fortunately, it has been observed that a single drug can often be effective in treat- ing multiple diseases. For example, thalidomide was originally used an anti-anxiety medication Miller 1991). and was later found to have the potential for the treatment of cancers (Singhal et al.) 1999.. Hence, drug repurposing, also known as the identification of new uses for the approved or ex- perimental drugs/compounds, might bring us the hope to address this urgent need with the advantage of shorter research cycle, lower investments, and more pre-existing safety tests.
Existing drug repurposing approaches can roughly be categorized into three groups: experimental- based approaches (e.g., binding affinity assays, phenotypic screening), clinical approaches (e.g., off-label drug use analysis), and computational-baase proaches) (Dhir et al. 2020). Due to the advancement of techniques, more and more publicly avail- able biomedical data can be freely accessed in different databases such as DrugBank Wishart al.) 2017), ChEMBL (Gaulton et al. 2012). HMDB (Wishart et al., 2018), which makes the computa- tional approaches seem to be more cost-efficient, particularly when the goal is prioritize repur-
approaches (e.g., network-based ap-

---

Under review as a conference paper at ICLR 2023
posed targets for followup experimental investigation. One of the computational drug repurposing methods commonly used in recent years is to integrate existing biomedical relations from databases or literature into a so-called biomedical knowledge graph (BKG) where unknown drug-disease treatment relationships are predicted via different knowledge graph (KG)-based machine learning models (Himmelltein al. 2017 [oannidis et al., 2020b; Zhang et al. 2021 ZZang & Chel 2021). Although these KG-based models are demonstrated to have good predictive performance for drug repurposing, they struggle to explain why some drugs can be useful for treating a given disease in an intuitive and unndeerstdd fashion. To solve the "black-box" concern for drug repurposing prediction, some methods are proposed to leverage KG-based paths as explanations, as illustrated in Figure [ However, these existing models cannot be efficiently applied to a large and general BKG without additional weighted edge information (Sosa et al., (020) or pre-defined meta-paths derived from domain experts or inefficient computational methods (e.g., degree- -eeghteweeghted path count "DWPC"") (Liu et al. 202I).
In this study, we customize large and stan- dardized biomedical knowledge graph and pro- pose a computational drug repurposing frame- work that predicts not only the treatment prob- abilities between drugs and diseases but also the KG-based mechanism of action (MOA) (Davis 2020) paths their biomedical expla- nations based on the treatment predictions. For drug repurposing predictions, we first calcu- late attribute embedding as the initial feature of each node and employ the GraphSAGE model in an unsupervised manner to further capture the neighborhood information for each node, then Random Forest model is utilized to pre- dict the treatment probability of drug-diseass pairs based on their embeddings. To predict the MOA paths, we employ the ADversarial Actor- Critic (ADAC) reinforcement learning (RL) model (Zhao et al. 2020) to perform path-finding on the knowledge graph. To encourage the RL model to find paths that are biologically reasonable, we amplify it with knowledge-and- aawaaaddlliaaton--aaaaa drug can treat a disease. Although the underlying mechanisms of action between repurposed drugs and their target diseases largely remain vague, this study, we assume that a repurposed drug fol- lows similar molecular mechanisms as the known MOAs to treat different diseases. Based on this assumption, we define demonstration paths based on the known drug-target interactions from a cu- rated drug database (e.g., DrugBank v5.1 (Wishart et al. (017)) and chemical-knowe -eo---kloeooeettttiti data provider (c.g., Molecular Data Provider v v1.2 as well as an adjusted PubMed- libeed-publication- based version of Normalized Google Distance (NGD) (Cilibrasi & Vitanyi 2007). In summary, the main contributions are summarized as follows:
+
Gene1
Blo- plo-process1
Treats
?
Drug
Disease
Gene2
Bio-process2
Figure I: Drug repurposing prediction and path- based explanation.
"demonstration paths", paths that explain why
We propose novel computational model framework that both accurately predict how likely a drug can be used to treat disease and also predict its corresponding knowledge graph- based mechanism of action path as the explanation of the predicted treatment.
We innovative in using knowledge-basse based and publication- based method to extract demonstration paths from a BKG and leverage it to guide the RL model to identify biolog- ically reasonable paths. Empirical results demonstrate the great effectiveness of it.
By comparing with the existing popular KG-based models and evaluating the predicted paths with an expert-curated path-based drug MOA database DrugMechDB (Mayers et al. 2020), we show that this proposed model framework outperforms the state-of-the-art baseline models on both the pre- dictive performance of drug repurposing and the explanatory performance of recapitulating human- curated MOA paths provided by DrugMechDB. In further case studies, by comparing the model predictions with the real regulatory networks, we show that the proposed framework is effective in identifying biologically reasonable KG-based paths for woal-world applications.
com/ http://////titiooooo//o///aoo/ra
Mo 1ocecul ar--aata-aPooo
2

---

Under review as a conference paper at ICLR 2023
2 RELATED WORK
Biomedical knowledge graphs A biomedical knowledge graph (BKG) is normally defined heterogeneous, semantic graph that integrates biomedical information from multiple data sources (Nicholson & Greene 2020). Many open-source BKGs have been developed for different biomed- ical research purposes. These existing BKGs can be categorized into three groups: database-based BKGs, literature- based BKGs, and mixed BKGs. The database- based BGKs (e.g.. Hetioner (Him- melstein et al. 2017), BioKG (Walsh et al. 2020), CBKH (Su et al. 202I)) are constructed with biomedical data and their relations stored existing biological databases. The literature-based BKGs are obtained by leveraging Natural Language Processing (NLP) techniques to extract the se- mantic relations from a large amount of available biomedical litcrature and electronic health record (EHR) data. They are more disease-specific if the text sources are only focused on specific dis- eases. For example, Zhang et al. (202T) recently constructed a literature knowledge graph via the collections of Covid-T9 literature from the Semantic MEDLINE Database (SemMedDB) (Kilicoglu jet al. 2012) as well as other Covid-19 19 datasets for Covid- 19 drug predictions. The mixed BKGs (e.g.. CKG (Santos al., 2020). DRKG (Ioannidis et al. 2020al) are generated by combining the knowledge collected from both existing biological databases and available text-based data.
KG-based computational methods for drug repurposing KG-based computational approaches have been widely used in drug repurposing recent years (Zhang al.) 2021, Al-Saleem et al.) 1 Yan et al. 202T). These approaches treat drug repurposing as a link prediction task by apply- ing different machine learning models to biomedical knowledge graphs. These models can mainly models, translation-based models, and neural network-basee models (Zeng et al. 2022). The tensor-factorizatio based models (Nickel et al. 2011 Yang et al. 2014) consider KG cubic tensor in which three dimensions spectively represent head entity, tail entity, and their relations and then leverage tensor- factorizatt
be classified into three groups: consor-factorizztion--
methods (Rabanser et al. (017) to recover the missing relations in KGs. The translation-based mod-
els (Bordes et al.] 20133] Lin et al. 2015) Sun et al.) 2019) regard the semantic relations in KGs as a "translation" process, that is, given triple (head entity, relation, tail entity) the head entity can be translated to the tail entity via the relation- relation-specific "translation". The neural network- based models [Ioannidis et al. 2020b; DDttmers al. (017) leverage graph convolutions or 2D convolutions to learn the underlying relationships between two entities.
Explanation of drug repurposing Although drug repurposing prediction accelerate the pro- cess of drug discovery, one of the greatest concerns in computational drug repurposing is the lack of biologically reasonable explanations, which hinders its wide adoption in clinical settings. Currently, there are few computational models designed for drug repurposing explanations. A common and intuitive explanation for drug repurposing leverages the semantic KG-based paths between given drug-disease pairs. To the best of our knowledge, existing models either utilize known statistics scores or use reinforcement learning models to find the most likely KG-based paths for explaining drug repurposing. For example, [Sosa et al.] (2020) applied a graph embedding model UKGE (Chen) al. 2219)), which utilizes the confidence scores of relation edges a literature- based KG GNBR, to identify new indications of drugs for rare diseases and then explain the results via the highest- ranking paths based on confidence scores. [iu et al. (202T) developed a Reinforcement Learning- based model "PoLo" that utilizes the biological meta-pate -atth identified in Himmelstein et al. (2017) via a computationally inefficient method "DWPC" to supervise path searching for drug repurposing.
3 KNOWLEDGE GRAPH
To accommodate biomedical-reasonabbee predictions of drug repurposing and mechanism of ac- tion, the ideal knowledge graph should accurately represent comprehensive and diverse interactions among known biological entities. Thus, we utilize the canonicalized version of the Reasoning Tool X Knowledge Graph 2 (RTX-KG22
(Wood al.) 2022). one of the largest open-source biomedical knowledge graph (BKG) that has been widely used in the Biomedical Data Translator Project (Trans-
lator Consortium
2019b a). Compared to other commonly used open-source BKGs mentioned in
3

---

Under review as a conference paper at ICLR 2023
Drug Repurposing Prediction
Mechanism Of Action (MOA) Prediction
Treat or Not
(Drug Bio-Process
Random Forest Classifier
Demonstration Paths:
2. (Drug 5, Gene N, Bio-Process Diseasn A)
Embedding
Actor Network
Current Location
Critic Network
0.55
Drug
Gene X
(Drug, Gonel
Meta Path Diseriminator
[Drag A, Gene X]
Path
X
Figure 2: Illustration of the drug repurposing prediction model (left, see Sec. 4.1 for more details)
and mechanism of action (MOA) prediction model (right, Sec. for more details).
Sec. RTX-KG2c is biolink-model-basel Saaaaaasddddddddriiiiaaaaaaaa standardized (Unni et al. 2022 and eegularly-updated The version 2.7.3 of RTX- that we use aggregates data from 70 public knowledge sources into large graph where all biological concepts (e.g.. "ibuprofen") (e.g., "ibuprofen increases activity of GPIBA gene") are presented edges. We customized it for drug repurposing purpose with four principles: nodes whose categories are irrelevant to drug repurposing explanation (e.g., "GeographicLocatee and "Device"); 2). filtering out the low-quality edges based on our criteria; 3). removing the hier- archically redundant edges; 4). excluding all drug-diseas, edges (see more details in Appx. [.. After this, 3,659, 165 nodes with 33 distinct categories and 18,277 edges with 74 distinct types are left in our customized knowledge graph, which is used for downstream model training.
represented as vertices and all
excluding the
PROPOSED METHOD
In this section, we describe our proposed model framework (see Figure 2 that predicts drug re- purposing and identifies the possible KG-based mechanism of action (MOA) paths its biological explanations. The proposed framework consists of two components: drug repurposing prediction with GraphSAGE (Hamilton al. (017)) plus Random Forest (Sec. and MOA prediction with an ad adversarial actor-critic reinforcement learning (RL) (Sec.
Notation: Let {V,E} be directed knowledge graph, where each node v E V represents particular biomedical entity (e.g., specific drug, disease, or gene, etc.) and each edge E & represents biomedical relationship (e.g., interacts-with), We use ydrug to represent all the drug nodes and ydisease to represent all the disease nodes (defined in item 4 in Appx. A.
4.1 DRUG REPURPOSING PREDICTION
Drug repurposing aims to identify new indications of approved or experimental drugs. We solve as a link prediction problem on the knowledge graph G. Specifically, given any drug-disease pair (v,, where V E ydrug and By E dissease we predict the probability that drug i can be used to treat disease j. We first use GraphSAGE to calculate the embedding for each node. Idcally, the node embeddings should contain two kinds of information: node attributes and node neighborhoods. optimize GraphSAGE to encourage neighbor nodes
To capture the neighborhood information, to have similar embeddings and non-neighbor nodes to have distinct embeddings. Specifically, we perform random walks for each node collect its neighborhood information and train the model to maximize a node's similarity with its neighbor nodes. For node u, the loss is calculated as:
Jz(2,) = log 2,)) E, (P) log(a zy,,))
Biolink model hhttps://biol:: http::th....o.ooooooo..ooo model is standardized BKG ontol- ogy framework
4

---

Under review as a conference paper at ICLR 2023
where Zu. 2e are respectively the embeddings of nodes u, v, is the sigmoid function, e is node that co-occurs with u in fixed-length random walks, Pn represents negative sampling distribution, and & indicates the number of negative samples (nodes not in u's fixed-length neighborhood).
To capture the node attributes information, we utilize the PubMedBERT model (Gu et al., 2022). a pre-trained language model designed for biomedical texts, to generate node attribute embedding for each node based on the concatenation of the node's 's name and category. We further compress the embeddings to 100 dimensions with Principal Components Analysis (PCA) to reduce memory usage and use them as the initial node feature for GraphSAGE, In this way. the final GraphSAGE embedding of each node contains the information of both graph topology and node attributes. We concatenate the GraphS AGE embeddings of drug-disease pairs and use them as input into a Random Forest model to classify each drug-diseas, pair into one of the "not treat", "treat" and "unknown" classes. We obtain "treat" and "not treat" drug-disease pairs from biomedical datasets (see Appx. A.3.I and generate "unknown" drug-disease pairs through negative sampling (Mikolov et al. 2013).
4.2
MECHANISM OF ACTION (MOA) PREDICTION
When potential indications of given drug are identified by the proposed drug repurposing model, a natural yet important question is: can we biologically explain the predictions? We solve this by employing a reinforcement learning (RL) model predict the KG-based MOA paths, which are essentially the paths on the knowledge graph from drug nodes to disease nodes. These KG-based MOA paths can semantically describe an abstract biological process of how drug treats a disease.
4.2.1 DEMONSTRATION PATHS
To encourage the RL agent to terminate the path searching at the expected diseases through biolog- ically reasonable path, we leverage so-called demonstration paths, a set of biologically likely paths (e.g., -er---ene--rrotenn--iieeee a disease. We extract the demonstration paths by using the known drug-target interactions collected from two curated biomedical data sources: DrugBank (v5.1) and Molccular Data Provider (v1.2) along with a literature-based source that utilizes the adjusted Normalized Google Distance (NGD) applied to concepts appearing in PubMed publication abstracts (see Appx. A.2 for more details).
that explains the underlying reasons of why a drug can treat
4.2.2 ADAC-BASED REINFOR LEARNING MODEL
We formulate the MOA prediction path-finding problem and adapt the ADversarial Actor-Critic (ADAC) Reinforcement Learning model to solve it. The reinforcement learning is defined as a Markov Decision Process (MDP) which contains:
States: Each state is defined as = (Várug: Ut, (.).,, ((-(K-))) where edrug E ydrus is given starting drug node; 1 E V represents the node where the agent locates at time 4; the tuple (u--x, (i-KK&eg (--(K1))) represents the previous Kthh node and (K 1)th predicate. For the initial state so, the previous nodes and predicates We concatenate the embedding of all nodes and predicates of st to get the state embedding, where the node embeddings are node attribute embeddings generated with the PubMedBERT model (see Sec. 4.1 and the predicate embeddings employ one-hot vectors.
substituted by a special dummy node and predicate.
The action space A, of each node Vt includes self-loop action and the actions to reach its outgoing neighbors in the graph G. Due to memory limitation and extremely large out- degree of certain nodes in the knowledge graph, we prune the neighbor actions based on the PageRank scores if a node has neighbors more than a certain threshold. Specifically, we let A, vcat(aself ......, .z..... ........aa. ,nn.).,nn..) where veat indicates the vertical concatenation and ne, is out-degree of node v, Ur E V. For each action a, (ve, Ec), we concatenate its node and predicate embeddings to obtain action embeddings. We learn two embedding matrices EN xd and E.N, xd respectively for nodes and predicates where d represents the embedding dimension, N, represents the number of nodes, and Np represents the number of predicate categories.
Actions:
Rewards: During the path searching process, the agent only receives a terminal reward Re, from the environment (Re, 0, vt < T). Let vr be the last node of the path, and Narag be the known
Fach sub-network uses separate embedding matrices.
5

---

Under review as a conference paper at ICLR 2023
diseases that drug Vdrug can treat, the terminal reward is calculated by using the proposed drug repurposing model (Sec. 4.1 via:
1,
if or E &Narug-
if UT $ Narugi UT E Vdiseaue and f f(druu:: vT) is predicted as "treat". VT $ Ndrugi UT E ydisease and [(druu: UT) is not predicted as "treat". oT & Vdisease.
Re,T 0,
Ptreats
where Ptreat is the "treat" class probability predicted by the drug repurposing model f.
The ADAC-based RL model consists of four sub-networks that share the same model architecture MLP' (note that i represents the id of each sub-network described later, such as for actor network, e for critic network, etc.) but with different parameters:
(X) f(f(XW] b))3 + 66) + +
where {Wi, W3, W3. bj, b2, b9} are the parameters and biases of linear transformations, f represents a batch normalization layer followed by an ELU activation function.
Actor network: The actor network learns path-finding policy uy (note that 0 represents all param- eters of actor network) to guide the agent to choose an action d, from the action space A, based on current state sc:
(a Sec A.) softmax( A, O Ms)"(s())
where @ represents the dot product. Here, Te (axa/1s, As) represents the probability of choosing action 0r at time / from the action space A, given the state 8t-
The critic network (Lillicrap et al. 2015) estimates the expected reward Q (Se, at) (note that @ represents all parameters of critic network) if the agent takes the action a, at state st by:
Critic network:
Qs(5,, a²) = MsP"(s:) O aa
(4)
Path discriminator network: Since the RL agent only receives a terminal reward indicating whether it reaches an expected target, to encourage the agent to find biologically reasonable paths and provide intermediate rewards, further guide it with demonstration paths. This network is essentially a binary classifier that distinguishes whether a path segment (#t, ae) is from demonstra- tion paths generated by the actor network. We treat all the known demonstration path segments (sP,aP) as positive samples and all actor-generated non-demonstration path segments (st, az) as negative samples. The path discriminator network Dp(st,a at) sigmoid(MLPP"(s at)), where @ represents the concatenation operator, is optimized with:
Ip Po. (lgg a))] P(..)>>PPoogl (, a))]
where PD and PA respectively represent the demonstration path segment distribution and the actor- generated non-emmonttration path segment distribution. Based on the probability (,(st, at), the path-discriminator based intermediate reward Rp,, is calculated as:
Rp.t = log (s, az)) log(1 - (p(ss,ae))
(6)
Meta-Path discriminator network: Similar to the path discriminator, this network aims to judge whether the meta-path of the actor-generated paths is similar to that of demonstration paths. The meta-path is the path of node categories (e.g.. ['Drug' -> "GGnee"
"=' I)- Similarly, the meta- meta-ppth discriminator Dm (M) sigmoid(MLPP (M)) is also binary classifier where the meta- peet-ppats of demonstration paths are treated as positive samples while others are negative samples. We optimize it with the following loss:
Lm = EEM~py [log( (Dm
E EMPP [log(1 Dm. (M))
where M a meta-path embedding vector defined as the concatenation of learned category embed- dings of all nodes that appear in the path; PM and PM respectively represent the demonstration meta-path distribution and the actor-generatee non-demonstratton meta-path distribution. The inter- mediate reward Rm,t generated by the meta-path discriminator is calculated by:
Rm, log( (m, (M)) log(1 Dm (M)).
(8)
The final intermediate reward R, at time t is then calculated as:
R₂ Opp Rp.t + am Run, + (1 Opp am ) T-t Re.T
(9)

---

Under review as a conference paper at ICLR 2023
where dp [0, 1] and a' [0, 1 apl are hyperparameters, fy is the decay coefficient, and Re,r is defined in the "Rewards" section above. To optimize the critic network, we minimize the Temporal Difference (TD) error (Sutton 1988) with loss:
= TD [R,, + Qo(s ((+1)) Qo a,)]²
(10)
Since the goal of the actor network is to achieve the largest expected reward by learning an optimal actor policy, we optimize the actor network by maximizing J(Ø) = EanTh Ea a)]. We use the
(st,
REINFORCE algorithm (Williams, 1992) to optimize the parameters. To encourage more diverse exploration in finding paths, we use the entropy of T as a regularization term and optimize the actor network with the following stochastic gradient of the loss function La:
VoLa (0) Ex V log No [e(((=)))] V peeetroppy(
(11)
where xy is the action probability distribution based on the actor policy and a is the entropy weight.
Following Zhao et al. (2020), train the ADAC-based RL model in multi-stage way. First, initialized the actor network using the behavior cloning method (Pomerleau) 1991) in which the training of demonstration paths used to guide the sampling of the agent with Mean Square Error (MSE) loss. Then, in the first epochs, we freeze the parameters of the actor network and the critic network and respectively train the path discriminator network and meta-path discriminator network by minimizing T,p and T.m- After 2 epochs, we unfreeze the actor network and the critic network and optimize them together by minimizing joint loss Lijoint = L.g I.c.
EXPERIMENTS
In this section, we introduce training data used in this study (Sec. 5.1). evaluation setup (Sec. 5.2) as well as evaluation results (Sec. 5.3 & 5.44) respectively for drug repurposing prediction and MOA prediction. Implementation details of the proposed models are presented in Appx. B
5.1 TRAINING DATA
Training data of drug repurposing model consists of drug-disease pairs in three categories: "treat", "not treat", and "unknown". For the first two categories, we collect data from four different data sources (Appx. A.3.I. The raw data are pre-processed and separated into training, validation, and test sets with ratios of [0.8, 0.1, 0.1] for downstream model training (Appx. A.3.2
5.2 EVALUATION METRICS AND METHODS
We evaluate the proposed models two types of tasks: predicting drug-disease "treat" probability (i.e., drug repurposing prediction) and identifying biologically reasonable KG-based MOA paths from all candidates (i.e., MOA prediction). These two tasks are evaluated based on classification accuracy based metrics (e.g., accuracy, macro fl score) and ranking based metrics (e.g., mean per- centile rank (MPR), mean reciprocal rank (MRR), and Hit@ K) defined as follows:
MPR
Σ pr
1 MRR
Σ PER
Hit@K
Eirsh R
(12)
PR
R
PrEPR
PER
where PR and are respectively a list of percentile ranks and list of ranks of all true positive drug-disease pairs ("treat" category) in the evaluation dataset (combination of validation and test).
Drug repurposing prediction: Due the extremely long running time of certain baselines (e.g., SVDP we calculate the metrics MRR and Hit@K K based on the rank of each true pos- itive drug-disease pair among 1000 random drug-diseass pairs (500 with drug id replacement and 500 with discase id replacement) and compare them across all models. We also show the comparison results with MRR and Hit K based on "all nodes" replacement for the models that can be imple- mented in reasonable time (Appx. C.2). In addition, since our drug repurposing model does 3-class classification while other baselines do 2-class classification, for a fair comparison, we re-calculate "accuracy" and "macro fl score" for our drug repurposing model by excluding the "unknown" class.
MOA prediction: For the evaluation of MOA prediction (Sec. 4.2 we use the DrugMechDB Mayers et al. 2020), expert-curated path-based drug MOA database, to obtain the verified MOA
7

---

Under review as a conference paper at ICLR 2023
paths ground-truth data. We match each biological concepts in these verified MOA paths to the biological entities used in our customized knowledge graph (Sec. (3 and generate the corresponding KG-based MOA paths (Appx. A.44). The matched KG-based MOA path is defined a 3-hop KG-based path of which all four nodes appear in the verified MOA paths. For each drug-disease pair not in the training set, calculate the path scores for all 3-hop KG paths between drug and disease with the path-finding policy learned from the ADAC-based RL model by using equation: path score = 2i=1 8t-1 x log( (P: x No). where k is the number of hops in this path; 8 ís a decay coefficient (we set it to 0.9 in this study); P. represents the probability of choosing action as in the ith hop following this path based
the trained ADAC-based RL model: N, is the number of possible actions in the ith hop. With these path scores, we obtain the ranks of the matched KG- based MOA paths and calculate the ranking-based metrics mentioned above. For those drug-disease pairs with multiple KG-based MOA paths, we use the highest ranks of their paths. We compare our ADAC-based RL model with the baseline models based on these metrics. In addition, we further perform some case studies to evaluate the effectiveness of the proposed method and present them in Appx.
5.3 DRUG REPURPOSING PREDICTION EVALUATION
Baselines: We compare our proposed drug repurposing prediction model against several state-of- the-art (SOTA) KG-based models and the variants of our proposed model. TransE (Bordes et al.) 2013). TransR (Lin et al., 2015). RotatE (Sun et al., 2019) are the conslation-distanee-- models that regard a relation
translation/rottaioo from a head entity to tail entity. MistMult Yang et al. 2014) is a bilinear model that measures the latent semantic similarity of a triple with trilinear dot product. ComplEx (Trouillon et al., 2016) and ANALOGY (Liu et al.] (017)
the
extensions of Di DistMult that consider more complex relations. S SimpLE (Kazemi & Poole 2018) censor-factorization-aasss model learn the semantic relation of a triple. GAT (Veličkovid
eet al. 2018) is popular graph attention model. Besides these SOTA baselines, we use the vari- ants of our proposed model (e.g., pure GraphSAGE for link prediction GraphSAGE-link, the GraphSAGE+ Logisttttiitsti ic model, the GraphSAGE+ SVM model, and the GraphSAGE+RA model for 2-class classification Implementation details of these baselines are presented in Appx. B
Evaluation results: Table I shows the evaluation results of the proposed drug repurposing model and other baselines based on the metries described in Sec. [..2] As shown in the table, on the one hand, the proposed model outperforms almost all baselines in classification-base metrics even though it is a bit worse than the GAT model, which indicates its effectiveness in classifying known "treat" and "not treat" drug-diseass pairs based on their attribute and neighborhood information on the knowledge graph (note that we conduct an ablation experiment to show the effectiveness of node attribute embeddings in Appx. C.T On the other hand, the best performance of our model in ranking-basse based metrics shows its capability in identifying new indications of existing drugs out of all the possible drug-disease pairs with relatively low false positives, which is of great importance iaaaa
for guiding clinical research. Besides, comparing 2 c-class GraphSAGE + RF with the vanilla GraphSAGE model (e.g.. GraphSAGE- link), the results demonstrate the effectiveness of the Random Forest model over a neural network classifier in this task. Comparison between 2-class and 3-class GraphSAGE+RF models indicates the importance of using negative sampling to gen- erate "unknown" drug-disease pairs for model training. With the "unknown" drug-disease pairs, the 3-cl ass GraphSAGE+R? model achieves significant improvement in ranking-based met- rics, which is essential when applying to real- world drug repurposing because it can reduce the false positives.
5.4 MECHANISM OF ACTION (MOA) PREDICTION EVALUATION
Baselines: For fair comparison, we choose the Mult t Hop model (Lin et al. 2018) (Implemen- tation details in Appx. B) as a baseline since it allows using a self-defined reward shaping strategy in its reward function as what we do in our proposed ADAC-based RL model. We don't 't compare with other existing models that either require krew edge importance information (see Sec. 2 or cannot be trained within reasonable time (e.g., within two weeks) on the customized knowledge graph. To show the importance of demonstration paths, we also compare with an ablated version of only considers true positive and true negative.
8

---

Under review as a conference paper at ICLR 2023
Table I: Comparing evaluation results between the proposed drug repurposing model with baselines.
The values with inside the parenthesis are the results after excluding the "unknown" category.
Model
Accuracy
Macro F1 score
MRR
Hit@1
Hit@3
Hit@5
TransF:
0.706
0.706
0.275
0.111
0.295
0.446
TransR
0.858
0.855
0,307
0.130
0,350
0.517
RotatE
0.707
0.707
0.255
0.077
0.287
DistMult
0.555
0.495
0,172
0,040
0.144
0.258
ComplEx
0.624
0.456
0.133
0.023
0.106
0.196
ANALOGY
0.597
0.467
0.179
0.045
0.147
0.272
SimplE.
0.600
0.475
0,161
0.037
0.135
0.236
GAT
0,932
0.929
0.002
0
0.001
GraphSAGE-link
0.919
0.915
0.002
0
0
0
GraphnAGGE-Hoisstic
0.787
GraphSAGE+SVM
0.800
0.795
0,002
0
0
2-class GraphSAGE+RF
0.929
0.925
0.271
0.176
0.311
0,386
3-class GraphSAGE+RF (ours)
0.934 (0.929*)
0.922 (0.925")
0.360
0.211
0,410
0,524
Table 2:
Evaluation results on MOA prediction on MultiHop model and ADAC- RL model with and without demonstration paths (ADAC RI w/o DP).
Model
MPR
MRR
Hi@1
Hit@10
Hit 50
Hi@@00
Hit
MultiHop
61.541%
0.012
0.043
0.081
0.124
0.360
ADAC RL w/o DP
73.281%
0,022
0.012
0.025
0.112
0,186
0.447
ADAC RL w/ DP (ours)
94.410%
0,123
0.062
0.242
0.509
0.640
0.857
the proposed model that does not take advantage of the demonstration paths by setting ap and am in Function O.
Evaluation results: Table 2 showw the evaluation results. Although all the models receive the same terminal rewards from the environment, the proposed ADAC- RL model achieves significantly better performance in identifying biologically reasonable KG-based MOA paths than other two base- lines. Comparison between the proposed model and the ADAD- based RL model without demonstra- tion paths (i.e., ADAC RL w/o DP) further illustrates the great effectiveness in using demonstration paths to guide the path-finding process. Due to the massive searching space and sparse rewards, the RL agent often fails to find reasonable paths of plenty of possible choices, while our model, with the intermediate guidance provided by the demonstration path, is able to identify the most biolog- ically reasonable choices at each time step with a much higher probability. Moreover, we perform two specific case studies (see Appx. [..) where we further show the top-rank MOA paths predicted by our model are able to identify key molecules in real drug action regulatory networks.
CONCLUSION
In this paper, we propose a computational drug repurposing model framework that predicts not only the treatment probabilities of drug-disease pairs but also the KG-based mechanism of action (MOA) paths as biological explanations for drug repurposing. We apply the proposed model framework to large and standardized biomedical knowledge graph and conduct experiments to compare framework with extensive state-of-the-an KG-based models. The results show that our proposed drug repurposing model achieves much better predictive performance in a comprehensive evalua- tion. We also show that by leveraging the proposed knowledge-and-pue anndauuliiationnbaaaae paths provide intermediate guidance during the path-finding process, our model can effectively find biologically reasonable paths out of a large amount of possible choices, Furthermore, two spe- cific case studies are performed to illustrate the effectiveness of our models in identifying some key biological molecules in real drug action regulatory networks. We believe our proposed frame- work can effectively reduce the "black-box" concerns and increase prediction confidence for drug repurposing based on its predicted path-based explanations, which further accelerates the process of drug discovery for emerging diseases. Despite the comparably good performance of our models, some limitations remain to be solved. For example, our MOA prediction is based assumption that a repurposed drug follows similar molecular mechanisms as the known MOAS to treat different diseases, which needs further biologically experimental verification.
demonstration
9

---

Under review as a conference paper at ICLR 2023
7 ETHICS STATEMENT
Our proposed method aims to provide a new model framework for drug repurposing prediction and explanation. The potential negative societal impacts might include the risk of misusing the drug repurposing capability (e.g.. misleading patients to take drugs inappropriately). Although model can provide certain suggestions for the new indications of drugs, it is mainly designed for assisting medical practitioners (e.g., doctors and licensed practitioners), who have professional training or knowledge to assess the accuracy of suggestions provided by our model. Therefore, the application of this method should be limited to drug development in pharmaceutical companies pre-clinical drug research in relevant academic or medical organizations, but not be directly accessible to the public. Since drugs treat diseases complex ways, there is still much unknown knowledge involved, and therefore any suggestions given by this proposed method should require further clinical testing but not be used directly diagnose, prevent, or treat any diseases without FDA approval. In addition, although the data used in this study (including the data in the biomedical knowledge graph RTX- KG2c) are from multiple data sources, they are collected from publicly accessible databases or datasets or APIs with free licenses (except for DrugBank data that we have requested and obtained its non-commercial license). Note that the licenses of all data are shown in Table Therefore, they have excluded any personal or sensitive information. All code used in this study is originally created by ourselves or modified from the code of previous researchers (note that we illustrate this at the top of those scripts), and allowed to be disclosed with the permission of its original authors.
Table Summary of licenses of all data we used.
Existing assets
Licenses
RTX-KGte (v2.7.) ://g thub, poom 2TX ean/Rty
CC-RY licesse
DrugMechDB data (V 1 Sttps 17 /gitare com///tab/De/geeeeeeeeo DDrggeechD
MIT license
DrugRank data hnctte
drogbank
DragRank academic licenee
Molecular ProviereV pt hti.ps///gtthtee /githus, con/ http://w....ooooooooooooooeoooo.
MIT
MyChem data (v1)n:tpss// /wwch///yccoooco -n-0
Apache-2. 2.0 license
SemMedDBB data (tamnaavRKas 8f Tpip Thnche, in nln.ai now/ /ro0 //eenRen SemNedn Stm.
UMIS S Motathesums licaase
NDF RT (V2011AAAAAA
UMLS Metalhssunus license
RepoDB danintsppt 0amt mappps eatngy sopadb
8 REPRODUCIBILITY STATEMENT
We share our code via the following link https://drive..ooqll compgiie..o///leeeeeee 99d WSeen wwtp:///wwww/////sssrs.a...
The code includes the steps to download our customized knowledge graph and other required datasets (e.g.. DrugBank data. Please note that to download DrugBank data, the DrugBank non-commercial license has to be requested and obtained by following the instructions on the DrugBank website). pre-process the raw data, train different models (e.g., our proposed models and baseline models), and evaluate models. The sources of all datasets used in this study are listed in Table 3 and the complete description of these datasets with their pre-processing steps is provided in Appx. A The implementation details of all models can be found in Appx.
REFERENCES
Jacob Al-Saleem, Roger Granet, Srinivasan Ramakrishnan, Natalie A. Ciancetta, Catherine Saveson, Chris Gessner, and Qiongqiong Zhou. Knowledge Graph-Based Approaches to Drug Repurposing for COVID-19. Journal of Chemical Information and Modeling, 61 (css)a..... 1c00442, 2021. ISSN 1549-9596. doi:
1c00422
R Arena, A Iudice, P Virgili, P Moretti, and G Menchetti. Huntington's disease: clinical effects of short-term treatment with pimozide. Advances in biochemical psychapharmacology 24:573-5, 1980. ISSN 0065-2229.
Nurken Berdigaliyev and Mohamad Aljofan. An overview of drug discovery and develop- ment. Future Medicinal Chemistry, 12(10):939-9M) 2020. ISSN 1756-8919. doi: 10.4155/ 1mc-2019-0307
10

---

Under review as a conference paper at ICLR 2023
Antoine Bordes, Nicolas Usumier, Alberto Garcia- Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. Proceedings of the 26th Interna- tional Conference Neural Information Processing Systems, volume 2, pp. 2787-2795, 2013. doi: 10.5555/299979922999922
Adam S. Brown and Chirag J. Patel. A standard database for drug repositioning. Scientific Data, (1):170029, 2017. doi: 10. 00.1038/sdta..2117.00
Steven H Brown, Peter L. Elkin, S Trent Rosenbloom, Casey Husser, Brent A Bauer, Michael Lin- coin, John Carter, Mark Erlbaum, and Mark Tuttle. Va national drug file reference terminology: cross-institutional content coverage study. Studies in health technology and informatics, 107 1):477-81, 2004. ISSN 0926-9630.
Chantel Cacciotti, Anthony Chan, and Mihir Bhatt. Oral Corticosteroid Therapy as an Adjuvant Treatment for Acute Bleeding in Hemophilia Patients With High Titer Inhibitors. Journal of Pediatric Heematology/Oncoloyy) 43(2):e237<<239, 2021. ISSN 1077-4114. doi: 10. 10.109h/mhh.
0000000000001339.
Giancarlo Castaman. The role of recombinant activated factor VII the haematological manage- ment of elective orthopaedic surgery in haemophilia A patients with inhibitors. Blood transfusion, 15(5):478 486, 2017. ISSN 1723-2007. doi: 10. 10.2440/2017.0366 16.
Xuelu Chen, Muhao Chen, Weijia Shi, Yizhou Sun, and Carlo Zaniolo. Embedding uncertain knowl- edge graphs. Proceedings of the AAAI Conference on Artificial Intelligence, 33:3363-33777 2019.
ISSN 2159-5399. doi: 10. 10.1609/aaii..o v33001.3330000 033001133333366
R.L. Cilibrasi and P.M.B. Vitanyi. The Google Similarity Distance. IEEE Transactions on Knowl- edge and Data Engineering, 19(3):370- 3-33333---3, 2007. ISSN 1041-4347. doi: 10.1 10.1109/tde.2007.c44
Qing Cong, Zhiyong Feng, Fang Li, Li Zhang, Guozheng Rao, and Cui Tao. Constructing biomedical knowledge graph based on semmeddb and linked open data. In 2018 IEEE Inter- national Conference on Bioinformatics and Biomedicine (BIBM), pp. 1628-1634 1631, 2018. doi:
N0.1109/BIB..20188. 86215661.
Emma M. Coppen and Raymund C. Roos. Current Pharmacological Approaches to Reduce Chorea in Huntington's 's Disease. Drugs, 77(1):29 46, 2017. ISSN 0012-6667. doi: 10.1007/
440265-016-0670-4.
Katherine F Croom and Paul I. McCormack. Recombinant factor VIIa (eptacog alfa): a review of its use in congenital hemophilia with inhibitors, acquired hemophilia, and other congenital bleeding disorders. BioDrugs: : clinical immunotherapeutics, biopharmaceuticals and gene therapy, 22(2):
121-36, 2008. ISSN 1173-8804. doi: 10.2
Ronald L. Davis. Mechanism of action and target identification, matter of timing in drug discovery. iScience, 23(9):101487, 2020. ISSN 2589-0042, doi: 10. 10.1016/).isci.20000 101487.
Nella Delva and Gregg D Stanwood. Dysregulation brain dopamine systems in major depressive disorder. Experimental Biology and Medicine, 246(9):1084-100) 1093, 2021. ISSN 1535-3702. doi:
10.1177/15337722119983333
Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. knowledge graph embeddings. arXiv, 2017. doi: 10448500anXix.1770..144
Convolutional 2d
Neha Dhir, Ashish Jain, Dhruv Mahendru, Ajay Prakash, and Bikash Medhi. Drug repurposing and orphan discase therapeutics. In Drug Repurposing, chapter 4. IntechOpen, Rijeka, 2020, doi:
10.5772/intechopen991999 91941..
Eric Dobrzynski. Federico Mingozzi, Yi-Lin Liu, Elisabeth Bendo, Ou Cao, Lixin Wang, and Roland W. Herzog. Induction of antigen-specifie CD4+ T-cell anergy and deletion by in vivo viral gene transfer. Blood, 104(4), 2004. ISSN 0006-4971. doi: 10.1 1..1182/bloo-- http//looo22000-000 00-4-03-0847.
Thierry Driessche, Desire Collen, and Marinee Chuah. Viral Vector- Gene Therapy for Hemophilia. Current Gene Therapy. 1(3):301-315, 2001. ISSN 1566-5232. doi: 10.2174/
1566523013348508.

---

Under review as a conference paper at ICLR 2023
Kevin Duff, Leigh J. Beglinger, Margaret E. O'Rourke, Peg Nopoulos, Henry L. Paulson, and Jane S. Paulsen. Risperidone and the Treatment of Psychiatric, Motor, and Cogmitive Symptoms in Hunt- ington's Disease. Annals of Clinical Psychiatry, 2008. doi: 10. 10.10800/04001230011848383
Anna Gaulton, Louisa J. Bellis, A. Patricia Bento, Jon Chambers, Mark Davies, Anne Hersey, Yvonne Light, Shaun McGlinchey, David Michalovich, Bissan Al-Lazikani, and John P. Over- ington. Chembl: large-scale bioactivity database for drug discovery. Nucleic Acids Research, 4D(D1))D1100-D
2012. ISSN 0305-1048. doi: 10. 1099///r//krr777.
Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, volume 9, pp. 249-256, 2010.
A. C. Goodeve. Hemophilia B: molecular pathogenesis and mutation analysis. Journal of Throm- bosis and Haemostasis, 13(7):1184-11) 2015. ISSN 1538-7933. doi: 10.1111/jth. 12958.
Huntington Study Group. Dosage effects of riluzole in Huntington's 's disease: multicenter placebo- controlled study. Neurology, 61(11))1551-6, 2003. ISSN 0028-3878. doi: 10.1212/01.wo
0000096019.7 19..09999..99226666
Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon. Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing. ACM Transactions on Computing for Healthcare, 3(1):1-23, 2022.
ISSN 2691-1957. doi: 10.1145/3458754.
Deisy Morselli Gysi, italo do Valle, Marinka Zitnik, Asher Ameli, Xiao Gan, Onur Varol, Su- san Dina Ghiassian, J. J. Patten, Robert A. Davey, Joseph Loscalzo, and Albert-László Barabási. Network medicine framework for identifying repurropuggg opportunities for COVID-19. Proceedings of the National Academy of Sciences of the United States of America, 118(19): e2025581 1118, 2021. ISSN 0027-8424. doi: 10. 20025//
William L Hamilton, Rex Ying. and Jure Leskovec. Inductive representation learning on large graphs. arXiv, 2017. doi: 10.485550/axxiv..766..00 706.02216.
Daniel Scott Himmelstein, Antoine Lizee, Christine Hessler, Leo Brueggeman, Sabrina Chen, Dexter Hadley, Ari Green, Pouya Khankhanian, and Sergio E Baranzini. Systematic integration of biomedical knowledge prioritizes drugs for repurposing. eLife, 6:e26726, 2017. doi: 10.7554/ elife.26726.
Enoch Huang, Marvin Heyboer III, and Davut J Savaser. Hyperbaric oxygen therapy for the manage- ment of chronic wounds: patient selection and perspectives. Chronic Wound Care Management and Research, Volume 6:27-37, 2019. doi: 10.2147/cwemms1177771
Vassilis N. loannidis, Xiang Song. Saurav Manchanda, Mufei Li, Xiaoqin Pan, Da Zheng, Xia Ning, Xiangxiang Zeng, and George Karypis. Drkg drug repurposing knowledge graph for covid-19. 2020a.
https: /ttbt::///gthbb. com hwtmm//nnddd//Dd///
Vassilis N. Ioannidis, Da Zheng, and George Karypis. Few-shot link prediction via graph neural networks for covid-19 19 drug-repurposing arXiv, 2020b. doi: 10.4555/ARXXW.200.10066
Seyed Mehran Kazemi and David Poole. Simple embedding for link prediction knowledge graphs. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, pp. 4289 4300, 2018. doi: n..555/33221444332233
Halil Kilicoglu, Dongwook Shin, Marcelo Fiszman, Graciela Rosemblat, and Thomas C. Rindflesch. Semmeddb: a pubmed-scale repository of biomedical semantic predications. Bioinformatics, 28 (23):3158-311 3160, 2012. ISSN 1367-4803. doi: 10.1093/boinnoomatiiccsss..
Diederik P. Kingma and Jimmy Ba.
Adam: A method for stochastic optimization. CoRR,
abs/1412.66800, 2015.
Timothy P Lillicrap, Jonathan J Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous control with deep reinforcement learning. arXiv, 2015. doi: 10.48550/arXiv.I1e 15t...1199........
12

---

Under review as a conference paper at ICLR 2023
Xi Victoria Lin, Richard Socher, and Caiming Xiong. Multi-Hop Knowledge Graph Reasoning with Reward Shaping. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 3243-3253, 2018. doi: 10.18653/11/d11-1330
Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. Learning entity and relation em- beddings for knowledge graph completion. In Proceedings of the Twenty-Ning AAAI Conference on Artificial Intelligence, pp. 2181-2187 2015. doi: 10.5555/2886521 12886624.
Hanxiao Liu, Yuexin Wu, and Yiming Yang. Analogical inference for multi-relational embeddings. arXiv, 2017. doi: 10.48550/arxiv. www.ii.......0... 775.00226.
Yushan Liu, Marcel Hildebrandt, Mitchell Joblin, Martin Ringsquandl, Rime Raissouni, and Volker Tresp. Neural multi-hop reasoning with logical rules biomedical knowledge graphs. In The Semantic Web, pp. 375-391. Springer International Publishing, 2021. ISBN 978-3-030-7738544 doi: 10.1007/978- 0-1099978----00-7755--2222
Michael Mayers, Dylan Steinecke, and Andrew I. Su. Database of mechanism of action paths for selected drug-disease indications (1.1.0). Zenodo, March 2020. doi: 10.5281/7enod..370083
Michael Mayers, Roger Tu, Dylan Steinecke, Tong Shu Li, Núria Queralt-Rosinace and An- drew I. Su. Design and application of a knowledge network for automatic prioritization of drug mechanisms. Bioinformatics, 38(10):2800-2891, 04 2022. ISSN 1367-4803, doi: 10.1093hioonfomaaccchaaaCC
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. Distributed representa- tions of words and phrases and their compositionality, 2013.
M T Miller Thalidomide embryopathy: model for the study of congenital incomitant horizontal strabismus. Transactions of the American Ophthalmological Society, 89:623-74, 1991. ISSN 0065-9533.
Giovanni Di Minno. Eptacog alfa activated: recombinant product to treat rare congenital bleeding disorders. Blood Reviews, 29:S26 S33, 2015. ISSN 0268-960X. doi: 10. 10.1016/s0268-960x 30005-9.
Artur S. Miranda, Pablo Leal Cardozo, Flavia R. Silva, Jessica M. de Souza, Isabella G. Olmo, Jader S. Cruz, Marcus Vinícius Gomez, Fabiola M. Ribeiro, and Luciene B. Vieira. Al- terations of Calcium Channels in Mouse Model of Huntington's Disease and Neuropro- tection by Blockage of CaV1 Channels. ASN NEURO, 11:17599911118856811 2019. doi:
10.1177/175999941188611
GARETH E. MORGAN, GABRIELLA ROWLEY, PETER M. GREEN, MORAG CHISHOLM, FRANCESCO GIANNELLI, and GEORGE G. BROWNLEE. Further evidence for the impor- tance of an androgen response element in the factor IX promoter. British Journal of Haematology, 98( 98(1):79-83 85, 1997. ISSN 0007-1048. doi: 10.1046/j. 1361-166622222 219,,1997.
Claude Negrier, Midori Shima, and Maureane Hoffman. The central role of thrombin in bleeding disorders. Blood Reviews, 38:100582, 2019. ISSN 0268-960X. doi: https://doi.or//1 1tt.///./....//......1111111111. 1016/j. blre.2019.05.001
David N. Nicholson and Casey S. Greene. Constructing knowledge graphs and their biomedical applications. Computational and Structural Biotechnology Journal, 18:1414-1428, 2020. ISSN
2001-0370. doi: 00.10166j.ssbj.2220..0
Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel. A threc-way model for collective learn- ing multi-relational data. In Proceedings of the 28th International Conference Machine Learning, ICML 2011, pp. 809-816, 01 2011. doi: 10.5555/3 104482.3104584.
Wonjun Noh, Sojeong Pak, Geunho Choi, Sungchil Yang, and Sunggu Yang. Transient Potassium Channels: Therapeutic Targets for Brain Disorders. Frontiers in Cellular Neuroscience, 13:265, 2019. ISSN 1662-5102 doi: (0.3389/fmcel.2019000222
D Paleacu, M Anca, and N Giladi. Olanzapine in Huntington's 's disease. Acta Neurologica Scandi- navica, 105(6):441 444, 2002. ISSN 0001 -411-..14. doi: 10. 1034/j. 1600-0404.2002.0119977 197.xx
13

---

Under review as a conference paper at ICLR 2023
Dean A. Pomerleau. Efficient training of artificial neural networks for autonomous navigation. Neural Computation, 3(1):88-9)) 1991. doi: 191162//ecoo.19113....8
Stephan Rabanser, Oleksandr Shchur, and Stephan Günnemann. Introduction tensor decomposi- tions and their applications machine learning. arXiv, 2017. doi: 10.4850VaxXv.1711..778..
Pablo Rendo, Lynne Smith, Hsiao-Yu Lee, and Frank Shafer. Nonacog alfa: an analysis of safety data from six prospective clinical studies in different patient populations with haemophilia treated with different therapeutic modalities. Blood coagulation & fibrinolysis an international journal in haemostasis and thrombosis, 26(8):912-8, 2015. ISSN 0957-5235, doi: 10.1097/bbc. 0000000000000599
Alberto Santos, Ana R. Colaço, Annelaura B. Niclsen, Lili Niu, Philipp E. Geyer, Fabian Cos- cia, Nicolai J Wewer Albrechtsen, Filip Mundt, Lars Juhl Jensen, and Matthias Mann. Clinical knowledge graph integrates proteomics data into clinical decision-making. bioRviv, 2020. doi: 10.1101/2020.05.0o 0ww.055.9994448 09.4877..
Surabhi Shukla and Babu L.. Tekwani. Histone Deacetylases Inhibitors in Neurodegenerative Dis- eases, Neuroprotection and Neuronal Differentiation. Frontiers in Pharmacology, 11:537, 2020.
ISSN 1663-9812. doi: ...388//fhar.2222000033
Shareen Singh and Thakur Gurjeet Singh. Role of Nuclear Factor Kappa B (NF-KB) Signalling in Neurodegenerative Diseases: An Mechanistic Approach. Current Neuropharmacology, 18(10): 918-935, 2020. ISSN 1570-159X.
doi:
120999.
Seema Singhal, Jayesh Mehta, Raman Desikan, Dan Ayers, Paula Roberson, Paul Eddlemon, Nikhil Munshi, Elias Anaissie, Carla Wilson, Madhav Dhodapkar, Jerome Zeldis, David Siegel, John Crowley, and Bart Barlogie. Antitumor activity of thalidomide in refractory multiple myeloma. The New England Journal of Medicine, 341511):1555-157, 1999. ISSN 0028-4793, doi: 10.
1056/nejm1 1991111mm99911111 10199999111111111 183412102.
Daniel N Sosa, Alexander Derry, Margaret Guo, Eric Wei, Connor Brinton, and Russ B Altman. A Literature-Basee Knowledge Graph Embedding Method for Identifying Drug Repurposing Opportunities Rare Diseases. Pacific Symposium on Biocomputing. 25:463 474, 2020. doi: 10.1142/97898 2215636_0041.
Julien Sourimant, Megha Aggarwal, and Richard K Plemper. Progress and pitfalls of year of drug repurposing screens against covid-19. Current Opinion in Virology, 49:183-193, 2021. ISSN 1879-6257. doi: 10.1016/j.coviro.. wwt.:/.vvvooo.....6666 2021.06.004.
Squitieri, M Cannella, A Porcellini, L Brusa, M Simonelli, and S Ruggieri. Short-term effects of olanzapine Huntington disease. Neuropsychiatry, neuropsychology, and behavioral neurology, 14(1):69-72, 2001. ISSN 0894-878X.
Chang Su, Yu Hou, Winston Guo, Fayzan Chaudhry, Gregory Ghahramani. Haotan Zhang, and Fei Wang. Cbkh: The cornell biomedical knowledge hub. medRxiv, 2021. doi: 10.1 10.1100/021..3.11 21253461.
Zhiqing Sun, Zhi-Hong Deng, Jian- Yun Nic, and Jian Tang. RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space. arXiv, 2019. doi: 11.48500/axxi.11922.1111
Richard S. Sutton. Learning to predict by the methods of temporal differences. Machine Learning, 3(1):9-44, 1988. ISSN 0885-6125. doi: 10.1007/6600115009
Translator Consortium. The biomedical data translator program: Conception, culture, and commu- nity. Clinical and Translational Science, 12(2):91994 March 2019a. doi: l 12592.
Translator Consortium, Toward a universal biomedical data translator. Clinical and Translational Science, 12(2):86-90, 2019b.
Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. Com- plex Embeddings for Simple Link Prediction. arXiv, 2016. doi: 10.48550/arXiv. 1606.06357.
14

---

Under review as a conference paper at ICLR 2023
Deepak R Unni, Sierra AT Moxon, Michael Bada, Matthew Brush, Richard Bruskiewich, J Harry Caufield, Paul A Clemons, Vlado Dancik, Michel Dumontier, Karamarie Fecho, et al. Biolink model: A universal schema for knowledge graphs in clinical, biomedical, and translational sci- ence. Clinical and Translational Science, 2022. doi: 10.11 10.1111/ctt 13302..
E. Unti, S. Mazzucchi, G. Palermo, U. Bonuccelli, and R. Ceravolo. Antipsychotic drugs in Hunt- ington's 'ss disease. Expert Review of Neurotherapentics, 17(3):227-237, 2017. ISSN 1473-7175.
doi: 10.1080014737115..216 1226134.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph Attention Networks. International Conference on Learning Representations, 2018. accepted as poster.
Aleksandar Videnovic.
Treatment of huntington disease. Current treatment options neurology, 15(4):424-38, 2013. ISSN 1092-8480. doi: 10.1007/s1 1940-003-1100000011
Mikhail V. Voronin, Ily A. Kadnikov, Liana F. 7ainullina, Ilya O. Logvinov, Ekaterina R. Verbo- vaya, Tatyana A. Antipova, Yulia V. Vakhitova, and Sergei B. Seredenin. Neuroprotective proper- ties of quinone reductase 2 inhibitor m- m-11, 11-11, e-mercapoobenzimddzzole derivative. International doi: 10.3390/jms222313066
Journal of Molecular Sciences, 22(23), 2021. ISSN 1422-0067.
Brian Walsh, Sameh K Mohamed, and Vít Nováček. BioKG: A Knowledge Graph for Relational Learning On Biological Data. Proceedings of the 29th ACM International Conference on Infor- mation & Knowledge Management, pp. 3173-3180, 2020. doi: 10.1 10.1145//330533.34477766
Ronald J. Williams. Simple statistical gralient- algorithms for connectionist reinforce- ment learning. Machine Learning, 8(3-4)222-25 256, 1992. ISSN 0885-6125. doi: 1023/a: 1022672621406.
David S. Wishart, Yannick D. Feunang, An C. Guo, Elvis J. Lo, Ana Marcu, Jason R. Grant, Tan- vir Sajed, Daniel Johnson, Carin Li, Zinat Sayeeda, Nazanin Assempour, Ithayavani Iynkkaran, Yifeng Liu, Adam Maciejewski, Nicola Gale, Alex Wilson, Lucy Chin, Ryan Cummings, Diana Le, Allison Pon, Craig Knox, and Michael Wilson. Drugbank 5.0: a major update to the drug- bank database for 2018. Nucleic Acids Research, 46(D1):kxx007- 2017. ISSN 1362-4962. doi: 10. 10.1093/ma//gkx1033
David S. Wishart, Yannick Djoumbou Feunang, Ana Marcu, An Chi Guo, Kevin Liang, Rosa Frzquez-Fresno, Tanvir Sajed, Daniel Johnson, Carin Li, Naama Karu, Zinat Sayeeda, Elvis Lo, Nazanin Assempour, Mark Berjanskii, Sandeep Singhal, David Arndt Yonjie Liang, Hasan Badran, Jason Grant, Arnau Serra-Cayuela, Yifeng Liu, Rupa Mandal, Vanessa Neveu, Allison Pon, Craig Knox, Michael Wilson, Claudine Manach, and Augustin Scalbert. Hmdb 4.0: the hu- metabolome database for 2018. Nucleic Acids Research, 46(DI):D608-DD -610, 2018. ISSN 1362-4962. doi: 10.1093/nar/gkxxl 1089.
E. C. Wood, Amy K. Glen, Lindsey G. Kvarfordt, Finn Womack, Liliana Acevedo, Timothy S. Yoon, Chunyu Ma, Veronica Flores, Meghamala Sinha, Yodsawalai Chodpathumwan, Arash Termehchy, Jared C. Roach, Luis Mendoza, Andrew Hoffman, Eric W. Deutsch, David Koslicki, and Stephen A. Ramsey. RTX-KG2: system for building a semantically standard- ized knowledge graph for translational biomedicine. BMC Bioinformatics, 23(1 (140, 2022. doi:
Jiwen Xin, Adam Mark, Cyrus Afrasiabi, Ginger Tsueng, Moritz Juchler, Nikhil Gopal, Gregory S. Stupp, Timothy E. Putman, Benjamin J. Ainscough, Obi L. Griffith, Ali Torkamani, Patricia L. Whetzel, Christopher J. Mungall, Sean D. Mooney, Andrew I. Su, and Chunlei Wu. High- performance web services for querying gene and variant annotation. Genome Biology, 17(1): 91, 2016. doi: 10. 001886611399-0-00000
Jiwen Xin, Cyrus Afrasiabi, Sebastien Lelong, Julee Adesara, Ginger Tsueng, Andrew I. Su, and Chunlei Wu. Cross-linking biothings apis through json-ld to facilitate knowledge exploration. BMC Bioinformatics, 19(1):30, 2018. doi: 10.1
15

---

Under review as a conference paper at ICLR 2023
T Yagami, H Kohma, and Y Yamamoto. L-type voltage-dependent calcium channels as therapeu- tic targets for neurodegeneratiie diseases. Current medicinal chemistry, 19(28):4816-2) 27, 2012. ISSN 0929-8673. doi: 1TT.://99229817220334444444
Vincent K C Yan, Xiaodong Li, Xuxiao Ye. Min Ou, Ruibang Luo, Qingpeng Zhang, Bo Tang, Benjamin J Cowling, Ivan Hung, Chung Wah Siu, Ian C K Wong, Reynold K Cheng, and Esther W Chan. Erratum: Drug Repurposing for the Treatment of COVID-19: Knowledge Graph Approach. Advanced therapeutics, 4(10):210017, 2021. doi: 10. 10.10022/udp22221001173
Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. Embedding Entities and Relations for Learning and Inference Knowledge Bases. arXiv, 2014. doi: 10.48550/arXiv. 1412.6575.
Christine N. Yohn, Mark M. Gergues, and Benjamin Adam Samuels. The role of 5-HT receptors in 3041-017-0366-.
depression. Molecular Brain, 10(1):28, 2017. doi:
In Tag Yu, Yong Park, Sung Hyun Kim, Jeong-sik Lee, Yong-Seok Kim, and Hyeon Son. Valproic acid promotes neuronal differentiation by induction of proneural factors associa- tion with H4 acetylation. Neuropharmacoloyy. (6(2):473-488) 2009. ISSN 0028-3908. doi: 10. www.o///..rrrrrmm..mm 2008.09.01.
Xiangxiang Zeng, Xinqi Tu, Yuansheng Liu, Xiangzheng Fu, and Yansen Su. Toward better drug discovery with knowledge graph. Current Opinion in Structural Biology, 12:114-126, 2022. ISSN 0959-440X. doi: 10.1016)j.bb.2222.000 2021009.00.......
Rui Zhang, Dimitar Hristovski, Dalton Schutte, Andrej Kastrin, Marcelo Fiszman, and Halil Kil- icoglu. Drug repurposing for covid-19 via knowledge graph completion. Journal of Biomedical Informatics, 115: 113:10369, 2021. ISSN 1532-0464. doi: 10.1016/j ji.2....b..2111333366 2021.103696.
Xiaolin Zhang and Chao Che. Drug Repurposing for Parkinson's 's Disease by Integrating Knowledge Graph Completion Model and Knowledge Fusion of Medical Literature. Future Internet, 13(1): 14, 2021. doi: 13900/11111014....
Kangzhi Zhao, Xiting Wang, Yuren Zhang, Li Zhao, Zheng Liu, Chunxiao Xing, and Xing Xie. Leveraging Demonstrations for Reinforcement Recommendation Reasoning over Knowledge Graphs. Proceedings of the 43rd International ACM SIGIR Conference on Research and De- velopment in Information Retrieval, pp. 239-248, 2020. doi: 00.1145/399721.33011777
16

---

Under review as a conference paper at ICLR 2023
A DATA COLLECTION AND PRE-PROCESSING
A.1 BIOMEDICAL KNOWLEDGE GRAPH RTX- PRE-PROCESSING
The dataset of RTX-KG2c (v2.7.3) Wood et al. 2022 accessed via wttpgghhub..oo//XXttettaeaa
which contains around 6.4M concept nodes with 56 distinct categories and 39.3M relationship edges with 77 distinct relations. We pre-process the raw data of RTX-KG2c by the following four principles:
1. Since we are mainly interested in the categories relevant to drug mechanisms of action (MOAs), we exclude the nodes with categories that are not expected to be useful for drug re- purposing explanation (c.g.. "Geographicalocctton"" "Device", "Informationnesouuree))
2. One of data sources used in the RTX-KG2c is the Semantic MEDLINE Database (Sem.MedDBB) (Kilicoglu et al., 2012), one of the most widely used -i---vereee biomed- ical knowledge sources, that has been found recently (Cong et al.) 2018 to contain many noisy relations due to the immature NLP techniques even though it contains many latest- found relations (e.g., the relations with Covid-19). To improve the quality of SemMedDB- based edges, we filter out parts of SemMedDB edges based on the criteria that each re- maining SemMedDB edge has be supported by at least 10 publications as well as the wtw.eeddddppbiiccin-aabaaaeeaeee
NGD score (Cilibrasi & Vitanyi 2007) (described in Appx. A.2) of its two end nodes should not be higher than 0.6
3. The RTX-KG2e is a multigraph that allows multiple edges connecting between two nodes. The edge relations (a.k.a. predicates) follow the predicate hierarchy 5 Jsedd in the Biolink model (Unni et al. 2022). Therefore, the RTX-KG2c contains some hierarchically redun- dant edges between two nodes. To reduce the complexity of path finding for the down- stream MOA predictions, we only reserve the "leaf" predicates of the Biolink semantic relation hierarchy if there exists hierarchically associated edges between two nodes. For example, if the edges between two nodes are "affected by". "entity regulated by entity" and "entity positively regulated by entity",, we remove the "affected by" and "entity regulated by entity". edges because they are the "ancestor" predicates of "entityy positively regulated by entity" based on the predicate hierarchy. Removing those "ancestor" predicates does not affect the path explanation because the "leaf" predicates contain more precise semantic information than their "ancestor" predicates.
4. To prevent the training information leakage, we exclude all existing edges connecting between the potential drug nodes (the nodes with the categories of "Drug" and "Small- Molecule") and the potential disease nodes (the nodes with the categories "Dis- ease", "PhenotypicFeature" "BehavioralFeature" and "DiecasOOrheenyyyieeearrr in the KG2c.
After the above pre-processing steps, 3,659,165 nodes with 33 distinct categories (Figure 3 a) and 18,291,237 edges with 74 distinct types (Figure 3) are left for our customized RTX-KG2c.
A.2 DEMONSTRATION PATH EXTRACTION
The demonstration paths set of multi-hop KG-based paths used to guide the agent in the ADAC- based RL model to find biologically reasonable KG-based MOA paths. can be formulated as pk= (pE.elt, E ydrug, de Vdiseasee where rÊ, is multi-hop demonstration path with maximum path length lo starting from a drug node v. and ending at disease node e;. Given a drug node and disease node (defined in Appx. I. the number of paths in the knowledge graph between them increases exponentially when k: increases. Therefore, we set k = 3 to guarantee that the agent can find the biologically meaningful predicted paths in reasonable amount of time. We extract the reasonable demonstration paths from the customized RTX- (defined in Sec by using the known drug-target interactions collected from two curated biomedical data sources (c.g., DrugBank (v5.1) and Molecular Data Provider (v1.2)) and an adjusted PubMed-han based version of Normalized Google Distance (NGD) score (Cilibrasi& Vitanyi 2007) formulated as follows:
Visualizzaiion of hierarchical predicates in the Biolink Model 1.. www.kkuppp.commpppeiicc.c..
17

---

Under review as a conference paper at ICLR 2023
(a) Number of Nodes by Category in Customited RTX-KG2c
(b) Number of Edges by Predicates in Customized RTX-KG2c
Pilccda S SiC
Souuling 000
Figure 3: Number of nodes by category (a) and number of edges by predicate (b) in customized RTX-KG2c.
NGD(cI,c2) = logN - minnloog] log)
(13)
where cl and c2 are two biological concepts used in the customized R7X-KG2e (defined in See 3)
f(11) and fc22) respectively represent the total number of unique PubMed IDs associated with cl and c2: [cl, c2) is the total number of unique PubMed IDs shared between cl and c2 (All associated PubMed IDs are stored node attributes in our customized RTX-KG2c); N the total number of pairs of Medical Subject Heading (MeSH) terms annotations in PubMed database
By using both curated drug-target interactions and PubMed- peneeeddpubliction-bbeee based NGD scores, the edges within a demonstration path must satisfy the following two requirements: 1. the edge be- tween the drug node and the first intermediate node needs be confirmed by the DrugBank (v5.1) or the Molecular Data Provider (v1.2) well as has NGD score that is not higher than 0.6; 2. the edge between the second intermediate node and the disease node should have a NGD score that not higher than 0.6. According to these requirements, we finally find 795,705 demonstration paths 8,495 true positive drug-disease pairs (described Appx. A.3.2) and we split them into training. validation, and test sets for ADAC-based RL model training.
A.3 TRAINING DATA
A.3.1 DATA COLLECTION
We collect data from four different data sources for training our purposed models:
MyChem Data (Xin et al.] 2018) is provided by the BioThing API collection (Xin et al. 2016), which contains up-to-date annotations regarding indication and contraindication for drugs.
SemMedDB Data (Kilicoglu et al., 2012) is provided by the Semantic MEDLINE Database (SemMedDB) which leverages natural language processing (NLP) techniques to extract semantic triples with "treats" and "negatively treats" relations from PubMed abstracts.
NDF-RT Data (Brown et al., 2004) is provided by National Drug File Reference Terminology from Veterans Health Administration (VHA) which contains information drug interaction, indi- cations, and contraindications.
PubMed Phttps:/aare ncats. 1o/ and abstracts life sciences and biomedical topies
) is MEDLINE database that contains abundance references
18

---

Under review as a conference paper at ICLR 2023
Table 4: Pair count of true positive (indications) and true negative (contraindications effect) data from four data sources (Appx. A.3.I). Note that 'shared' means those pairs are from two or more data sources.
Source
True Positive (Treats)
True Negative (Not Treat)
MyChem
3,663
26,795
SemMedDB
8,255
11
NDF-RT
3,421
5,119
RBpoDB
2,127
738
Shared
3,971
526
Total
21,437
33, 33,189
RepoDB Data (Brown & Patel 2017) is standard set of successful and failed drug- pairs in clinical trials collected by the Blavatnik Institute at Harvard Medical School.
A.3.2 PRE-PROCESSING
To ensure our proposed models can be trained on high-quality data, we pre-process the raw data collected from four data sources (see Appx. A.3.I by the following procedures:
1. We match the original identifiers of drugs and diseases from these four data sources to the identifiers used in the customized RTX-KG2c (defined in Secß), and then we remove duplicate drug-disease pairs and any pairs that appear in both true positives (pairs with re- lation "indication" " from MyChem Data, predicate "treats" from SemMedDB Data, ther- apeutics "indications" from NDF-RT Data and status "approved" from RepoDB Data) and true negatives (pairs with relation "contraindication" from MyChem Data, predicate "neg treats" from SemMedDB Data. therapeutics "contraindications from NDF-RT Data and status "withdrawn" from RepoDB Data)
2. Due to publication bias and possible NLP mistakes that exist in SemMedDB Data (Cong eet al.) 2018), for those drug-disease pairs derived from this source, select only those with support of at least ten publication abstracts for both true positive and true negative pairs. In addition, to increase the quality of SemMedDB based data, further filter the drug-disease pairs by using the PubMed-pul siaMeeddubbliatioon. based Normalized Google Distance (NGD) (Cilibrasi & Vitanyi 2007) scores with cutoff of 0.6 (described in Appx. A.2).
Table shows the drug-disease pair count from each data source after data pre-processing.
For the drug repurposing model (Sec. 4.T)) training, since we train trinary classifier, we generate an "unknown"" class by negative sampling. that is, replacing the drug disease identifier in each true positive drug-disease pair with a random drug or disease identifier to generate a new pair that does not appear in both the "treat" and "not treat" classes. Specifically, for each unique true positive drug-diseass pair, we respectively replace its drug identifier with other 30 random drug identifiers well replace its disease identifier with other 30 random disease identifiers to make 60 new drug-disease pairs for the "unknown" class.
For the ADAC-hased RI. model (Sec. 4.2.2 training, only the true positive pairs considered. In addition, we remove those true positive pairs which are not reachable from drug node to its corresponding discase node within maximum of 3 hops the customized RTX-KG2c and this results in 19,332 true positive pairs being left. Out of these 19,332 true positive pairs, 8,495 pairs can find least one demonstration path using the method described in Appx. A.2
We split the data into training, validation, and test sets based on the ratio of [0.8, 0.1, 0.1]. Table 5 summarizes the pair count used as training. validation, and test sets for the drug repurposing model and ADAC-based RI. model.
A.4 DRUGMECHDB MOA PATH MAPPING
We access the DrugMechDB Mayers et al. 2020) data via which contains 3,593 MOA paths for 3,327 unique drug-disease pairs. These paths extracted
19

---

Under review as a conference paper at ICLR 2023
Table 5: Pair count of true positive (indications) pairs, true negative (contraindications or ef-
fect) pairs, and random pairs respectively in training set, validation set, and test set for the drug repurposing model and ADAC-based RL model.
Data
Drug Repurposing Model
Adversarial RL Model
TP
TN
Random
TP
Training Set
17.149
26,552
34,306
6.796
Validation Set
2,143
3,318
4,286
849
Test Set
2,145
3,319
4,290
850
Total
21,437
33,189
42,882
8,495
from free-text descriptions from DrugBank, Wikipedia, and other literature sources and then have been curated by subject experts. For these 3,327 unique drug-disease pairs, we first match them to the biological entities used in our customized those drug-discase pairs whose MOA paths don't 't have intermediate nodes. Finally, 2,893 pairs out of 3,327 pairs are reserved after filtering. Since these MOA paths extracted from the human free-text description, the length of these MOA paths is varying. Based on a previous report (Mayers) al.) 2022), there are many edges/relations of these MOA paths that are missing in the existing biomedical databases and thus it is difficult to match the complete MOA paths to the KG-based paths in our customized biomedical knowledge graph. paths generated by our model is fixed to 3, we consider those 3-hop KG-baset paths of which all four nodes show up in the complete MOA paths are the correct matched paths. Based on this definition of correct matched paths, we finally find 472 unique drug-disease pairs of which each has at least one such correct matched path in its all possible 3-hop paths between drug and disease in customized biomedical knowledge graph. For those 2,421 unique drug-disease pairs that are filtered out because of no such correct matched path, most of them are due to the missing edges in eustomized biomedical knowledge graph which is consistent with the findings in the previous analysis (Mayers et al., (022). There are two reasons that could be used to explain these missing edges: 1) we filter out many low-quality edges from SemMedDB (described in Appx. A.I]; 2) RTX-KG2c might miss some data sources that support these missing edges.
(defined in SeeB). and then filter out
addition, since the length of predicted
B IMPLEMENTATION DETAILS
B.1 DRUG REPURPOSING MODEL (SEC. 4.1 AND ITS BASELINES (SEC. 5.3
We utilize the source code 7 provided by the GraphSAGE paper (Hamilton et al. 2017 to train the unsupervised GraphSAGE embeddings with its "big" mean-based aggregator and two hidden layers with dimensions [256, 256). In the random walk setting in GraphSAGE, we use 10 numbers of walk with 100 walk length. As for other parameters, the number of epochs is 10, the neighbor sampling size of each layer is 96, the learning rate is set to 0.001, the batch size is 256 and the maximum num- ber of iterations per epoch is 10,000. Instead of using the default identity embeddings as the initial embedding features, we use node attribute embeddings generated by the pre-trained PubMedBert model (Gu et al.) 2022) with concatenation of the node's name and category, and then are reduced to 100 dimensions with PCA technique. The dimension of final output embedding vector of each node 512. With these GraphSAGE embedding vectors, we then concatenate the embedding vectors of each grug-disease pair in the training set as input features and use the memudomForestlassiffe function of scikit-leam (v (v1.0) python package to train Random Forest model. We run a grid search via GridSearchCV function to determine the optimal parameter set of Random Forest model from depths (5. 10, 15, 20, 25, 30, 35) and number of trees (500, 1000, 1 15000, 2000). The best parameter set of the Random Forest model uses the maximum depth max_depth = 35 and the number of trees n. n_estimators = 2000.
For KG-based baseline models, we use the OpenKE library to implement the TransE, TransR, RotatE, DistMult, ComplEx, ANALOGY, and SimpLE models with their default parame-
httaass//tttub..coo///1illlleefe http:://ggttbbb.coo////////////
20

---

Under review as a conference paper at ICLR 2023
Table 6: Hyperparameters used for Baseline Models.
Model
Hidden Dim.
Num. Epochs
Batch Size
Learning Rate
Optimizer
TransE
100
10000
1000
SGD
TransR
50
2000
1000
SGD
RotatE
30
2000
1000
2e-5
Adam
DistMult
100
10000
1000
0.5
Adagrad
ComplEx
2000
500
Adagrad
ANALOGY
20
2000
500
0.5
Adagrad
SimpLE
100
2000
500
0.5
Adagrad
ter settings (we adjusted the hyperparameters for some models to ensure we can finish the train- ing on GPUs for reasonable time). Table ( shows the detailed hyperparameter setting of each OpenKE-based baseline model. For other baseline models, we use PyTorch Geometric plement the GAT and GraphSAGE- -iin model and use the same GraphSAGE embeddings (men- tioned above) with scikit-leam (v1.0) python package to implement the GraphSAGE logiis ic, GraphSAGE- SVM and 2- 2-class GraphSAGE +RE models with the rrid-search-based optimal parameter settings.
| to im-
B.2 ADAC-BASED RL MODEL (SEC. 4.2.2 AND ITS BASELINE (SEC. 5.4
We modify the model framework of Zhao al. ZZao al. 2020) and make it adapted to the proposed methods (Sec. for drug repurposing purpose. We respectively set the state history length K and the maximum length of path T In order to make our customized knowledge graph enable training on 48GB Quadro RTX 8000 GPU, we prune the action space of each entity with a maximum size of 3,000 based on the PageRank score. We set 100 to the dimensions of all lookup matrices used within actor, critic, meta-path discriminator, and path discriminator networks. The dimensions of hidden layers of the actor network and critic network are all set 512. We set the dimensions of hidden layers of the path discriminator with [512, 512] and used the dimension set of [512, 256] for the hidden layers of the meta-pal discriminator. We utilize Xavier initialization Glorot & Bengio 2010) for the embeddings of all lookup matrices and the network layers. We set ap 0.006 for the weight of the path discriminator reward and am = 0.012 for the meta-factor of the path discriminator reward. The decaying coefficient of Re,r and the weight of entropy term are respectively assigned 0.99 and 0.005. We optimize all networks with the Adam optimization algorithm (Kingma & Ba 2015) with learning rate of 0.0005. The mini-batch size is set 32 with a path rollout set to 35. The dropout rates of all subnetworks are set to 0.3 and the action dropout rate is set 0.5.
For the implementation of the Mult iop model (Lin et al. 2018), we use the source code 10 vided by the author and modify by using our defined reward shaping strategy (described Sec. 4.2.2.. We set all its parameters the same as the ADAC-based RL model if they are available other- wise we use the default parameters.
pro-
C OTHER EXPERIMENT RESULTS
C.I ABLATION EXPERIMENT FOR NODE ATTRIBUTE EMBEDDINGS
To show the effectiveness node attribute embeddings (mentioned in Sec. 4.1 in improving re- purposing predictions, we conduct ablation experiment to compare the 3-class GraphSAGE+RI model with and without node attribute embeddings, using random embeddings with Xavier initial- ization replace node attribute embeddings in the latter case. The results in Table shows that the GraphSAGE initialized with node attribute embedding result in the final node embeddings with more expressive power and help improve the performance of drug repurposing prediction.
21

---

Under review as a conference paper at ICLR 2023
Table 7: Comparing evaluation results between the 3-class GraphS model with and without node attribute embeddings
Model
Accuracy
Macro F1 score
MRR
Hit@1 1
Hie@3
Hit@s
3-class GraphSAGF&RF RRF w/o nede attrihute embedding
0.907
0.889 0.922
0.151 0.360
0,031
0.134
0.239
3-class GraphSAGE-RF w/ node attribate embedding
0,934
0.211
0,410
0.524
C.2 DRUG REPURPOSING PREDICTION EVALUATION WITH "ALL NODES" REPLACEMENT
To further demonstrate that our proposed drug repurposing model has better capability in prediction with relatively low false positive, besides showing the comparison results based on 1000 random drug-disease pairs across all models (see T.b we calculate the MRR and Hit@ K metrics for the models that can be implemented in a reasonable time (excluding the CAT and GraphSage++ SSM models) by using the following three methods to replace drugs and/or diseases each true positive drug-disease pair ("treat" category) in the evaluation dataset (combination of validation and test sets):
Drug-rank-based Replacement: For each positive drug-disease pair, the drug-rank- based replacement pairs are generated by replacing the drug entity with each of all 274.676 other drugs in the customized RTX- KG2c while excluding all known true positive drug- disease pairs.
Disease-rank- based Replacement: For each true positive drug-disease pair, the disease- rank-based replacement pairs generated by replacing the disease entity with each of all 124,638 other diseases in the RTX- while excluding all known true positive drug- disease pairs.
Combined Replacement: For each true positive drug-disean pair, the combined replace- ment pairs are the combination of all replacement pairs of the above two methods. All known true positive drug-disease pairs excluded from these replacement pairs.
Figure 4 showw the comparison results of MRR and Hit@ K between our proposed drug repurposing model (I.e., the 3-class GraphSageRRF model) and other baselines (excluding the GAT and GraphSage+SVe models because of extremely long running time) by using the above three meth- ods to generate drug-disease replacement pairs. These results based on the ranking-based metrics calculated with more comprehensive "all nodes" replacement methods indicate proposed model is indeed superior to other baselines in identifying new indications of existing drugs with relatively low false positives.
C.3
CASE STUDIES
To further evaluate the performance of the ADAC-based RL model in predicting MOA paths for drug repurposing explanation, we utilize it to explore potential repurposed drugs for the orphan or rare genetic diseases (e.g., Hemophilia B and Huntington's Disease) and predict their top 10 KG- based 3-hop paths as explanations. These two diseases have known molecular pathogenesis which can be used to evaluate if the predicted paths are biologically reasonable as explanation for drug repurposing.
C.3.1 HEMOPHILIA
Hemophilia B (a.k.a factor IX deficiency or Christmas disease) is rare genetic disease that can cause
normal bleeding in patients. It is caused by the mutations in the factor IX (F9) gene on the X chromosome. Table 8 displays the top 10 predicted drugs/treatments from the pro- posed drug repurposing model (Sec. 4.1 which include both drugs/treatments used in the training set (highlighted with red color) and drugs/treatments not in the training set. We manually exclude the chemotherapeutic drugs from the predicted drug candidate list due to their potential risk of cy- totoxicity to normal cells, which might lead to false positives for drug repurposing of non-cancer diseases (Sourimant et al. 2021 Gysi al. 2021). For those seven drugs/treatments that not in the training set, many of them are biologically reasonable and supported by some publications that have the potential treat hemophilia B even though there are three predicted drugs that have
22

---

Under review as a conference paper at ICLR 2023
H1B1
0.4
0.13
6.0
622
08
100
co
Dasnant De Conmeett Comneccttann
Adved
conger
Aays
classta rrgelani
Hhgg5
HRgIZ
Hhg250
115
112
1.90
anton
Figure 4: Comparison results of MRR and Hit@K between our proposed drug repurposing model (i.e., the 3-class GraphSage RR model) and other basclines (excluding GAT and GraphSage+ SVM models) based on the evaluation dataset (combination of validation set and test set). The legend "drug rank", "disease rank" and "combined rank" respectively correspond to the methods of "Drug-rank-based Replacement", "Disease-ram based Replacement", and "Combined Replacement" deseribed in Appx. C.2
no supporting publication. By the predicted 3-hop KG-based MOA paths, the domain experts might be able to quickly determine whether they false positive drugs. For instance, Epicriptine is nootropcc drug that has no 3-hop KG-based path connecting this drug and hemophilia B and thus might be a false positive. Hyperbaric Oxygen therapy is reported have function wound healing in which the first phase to stop bleeding (Huang et al.) 2019) and there is only one 3-hop KG-based path ("Hyperbaric oxygen" - "entity positively regulates entity" + "Angiogenic process" + "has participant" + "Epidermal growth factor" - "gene associated with condition" - "Hemophilia B")) connecting between hyperbaric oxygen and hemophilia B, which might help experts to determine it can be used as adjunctive therapy for the treatment of hemophilia B. Triamcinolone is kind of steroids which might be beneficial for the adjunctive treatment of hemophilia (Cacciotti al., (02I)) Even though steroids such as triamcinolone might typically be used topically. a therapeutic steroid might be used orally. Therefore,
not surprising that triamcinolone is returned as a possible treatment. In its predicted 3-hop KG-based MOA paths (see Figure 5) some genes like the AR gene MMORGAN et al., 1997) and the CD4 gene (Dobrzysski et al.) 2004) are reported to be associated with the regulatory mechanism of hemophilia B. So, these might help experts further determine if triamcinolone is useful for hemophilia B.
In order further evaluate the biological explanations of our predicted 3-hop KG-based MOA paths, we utilize the curated DrugMechDB--aun based MOA paths. There are three relevant MOA paths found for hemophilia B respectively corresponding to Eptacog Alfa, Eftrenonacog Alfa, and Nonacog Alfa. Eptacog Alfa and Nonacog Alfa are in our top 10 predicted drugs/treatments which are all used in the training set. F shows the comparisons between the subgraphs with the top 10 pre- dicted 3-hop KG-based paths and the curated DugMechDB-basee Nonacog Alfa. The corresponding biological entities between the predicted paths and the curated DrugMechDB-baseee paths are highlighted with red color. Although the predicted paths can't 't exactly match the DrugMechDB-basee tic relationships, some key biological entities (e.g., Coagulation Factor VII. Coagulation Factor X. and Coagulation Factor IX) for the treatment of hemophilia show up in the subgraph of top10 predicted paths. Actually, the treatment of hemophilia B involves complex molecular network of blood coagulation (see Figure 7 Many biological entities (e.g., Coagulation factor VII, factor Vlla, factor III, factor II, factor VIII, factor IX, factor X) in the subgraph of top 10 predicted paths
MOA paths for Eptacog Alfa and
MOA paths due to the limited path length and some missing seman-
23

---

Under review as a conference paper at ICLR 2023
Triamcinolone
LPL gene
affects
same_as
AR gene
Hemophilia B
Triamcinolone
aftects
gene, woc.can0tt conditon
X-linked receesive
contribunt
incrosses wwwpaattttyyyyycyyyccc..c Interactes
gene, condition
NR3C1 gene
CD4 gene
Figure 5: The predicted 3-hop KG-based MOA paths for triamcinolone.
in such molecular network. Therefore, the subgraph constructed by our top 10 predicted paths can convey more molecular details than the DrugMechDB-barl --ased MOA paths.
(a) Eptacog Alfa Hemophilia B
Subgraph
constructed from top 10 predicted peths
DrugMechDS MOA path
Suppldin
Calonn Ale
Cosputtton
Viari decardaan
Goop udbn
Excrsic Narwar
(b) Nonacog Alfa Hemophilia B
Subgraph constructed from top predicted paths
DrugMechCB MDA path
Drup Dolguabenn
Coppane New's
Figure 6: Comparison between the top 10 predicted 3-hop paths (integrated into subgraph for better visualization) and the curated DugMeccDD--aace MOA paths (the original MOA paths are presented in the same way as the predicted paths) for Eptacog Alfa (a) and Nonacog Alfa (b), The corresponding biological entities between top10 predicted 3-hop paths and MOA paths are high- lighted with red color. The top 10 predicted paths where all entities show up in the DugMechDB. based MOA paths are also highlighted with red color.
Since both Eptacog Alfa and Nonacog Alfa are used in the training set, to evaluate the predicted MOA paths for the predicted drugs/treatments that are not in the training set, we use Factor VIIa which has the highest probability after excluding all drugs used in the training set. Figure shows sbugraph with its top10 predicted 3-hop paths. We can see that most biological entities (e.g.. Throm- bin, Factor IX, factor VII) biologically relevant (which are in the blood coagulation regulatory network). The predicted MOA paths (such as "Factor VIIa" + "entity positively regulates entity" "Factor Ix" "close match" "F9 (Coagulation Factor IX) gene" "gene associated with
24

---

Under review as a conference paper at ICLR 2023
Intrinsic Pathway
Extrinsic Pathway
surface contact
Xila
(Tissue factor pathway inhibitor)
Trauma
IX
Vila
VIl
Villa
Tissur factor <
Treuma
Va
Heparin
II (Prtthrombbin
lla (ah(ooommin))
Common Pathway
(Fibrinogen)
(Fibrin)
Active
XIII
Xilla
Protein 5
Protein C Thrombomodulin
Cross-linked Fibrin Clot
Figure 7: Blood Coagulation Regulatory Network with arrows for molecular reactions (black), pos- itive feedback (green), and negative feedback (red).
condition" "Hemophilia B*) biologically reasonable because the pathogenesis of hemophilia is the deficiency of factor IX.
Serine Endopeptidases
Inhibitors
FT gene
writy por phrt ouly swrtir
F2 gene
Thrombin
GENL www.LAROORROO...... condie
Factor Viaa
pww.iiooosssess.mmmm velty
F10 gene
endy s22 NOT regizees enty
poothey To eety
F8 gene
emophilia B
Endopeptidases
ansodgae off conetion
gera geraaanccoaggeonn sond
Coagulation factor VII Von Willebrand factor
F7 gene
Factor IX
F9 gene
Figure 8: A subgraph with the top 10 predicted 3-hop paths for Factor VIla Hemophilia B pair. The path highlighted with red color is biologically reasonable for explaining the treatment mechanism of Factor Vlla for Hemophilia B.
C.3.2 HUNTINGTON'S DISEASE
Huntington's disease (HD) is a rare neurogenetic disorder that typically occurs in midlife with symp- toms of depression, uncontrolled movements, and loss of cognitive ability. The drugs for HD are mainly used for the treatment of its symptoms in abnormal movements (c.g.. chorea) and psychiatric phenotype. We show ten probest- drugs/treatments predicted by the proposed drug repur-
25

---

Under review as a conference paper at ICLR 2023
Table 8: Top 10 predicted drugs/treatments for hemophilia B (note that the drugs highlighted with red color are used in the training set while the remaining drugs are the top-rank, non- chemotherapeutic drugs without showing up in the training set).
Rank
DruamTreatment
Prob
Publications
1
Eplacog Alfa (dFVIla)
0.833
Croom & McCormack
2008
Minno
2015
Nonacog (FFIX)
0.803
Kento
3
Viral Vector
0.780
[ Uneescce
4
Factor VIIa
0.748
Croom & McC ormack
20T
Castaman
2017
5
Recombisant FVlla (rFVIla)
0.724
Croon & McCormack
2008
Cisanan
2017
6
Thrombin
0.709
Neurier
7
Factor IX
0.708
2015
8
Epicriptine
0. 0.702
9
Hyperbariic Oxygen
10
Triamcinalone
0.649
Table 9: Top 5 predicted drugs/treatments used in the training set (highlighted with red color) and the top 5 nnn--chhmmoherapetticcc predicted drugs/treatments that are not in the training set for Hunt- ington's 's Disease.
Rank
Drug'Ireatment
Prob.
Publications
Pimozide
0.939
Arena al.
1980
Videnovic
2013
2
Therapeutic Agent
0.939
3
Olanzapine
0.938
Palencu ct
2002
Squitieri al.
2001
4
Riluzole
0.935
Urvip
2003
Antipsychotic Agent
0.932
mit
10
Risperidone
0.893
Coppen & Roos 2011
Dill et
2008
Entinostat
0.888
Shokla & Tokwani
20203
15
Primaquine
0.887
17
Isradipine
0.854
Miranda et al. 2019
Amifampridine
0.882
posing model (Sec. 4.1 in Table Like the case study of Hemophilia B, we exclude the chemother- apeutic drugs in the predicted drug candidate list. Since the top 8 predicted drugs/treatments in the predicted drug list are all in the training set, we only display the top 5 of them. The rest five in Ta- ble Jare those with the highest probabilities in the predicted drug list after excluding the drugs used in the training set and all chemotherapeutic drugs. From Table 9 dicted drugs are supported by some publications for the potential treatment of HD's symptoms. In order to evaluate the predicted paths for the predicted drugs/treatments that are not in the training set, we show the subgraphs with the top 10 predicted paths for each of the top 5 non-chen non-chemotherapeutte predicted drugs/treatments that are not in the training set in Figure From these predicted paths, we can see that most of them are biologically relevant. For example, the subfigure (a) of Figure shows that Risperidone is predicted to be useful for HD by decreasing the activity of the genes associated with the Hydroxytryptammine receptor (e.g.. HTRIA, HTR2A, HTR2C, HTR7) and dopamine ceptor (e.g., DRDI, DRD2, DRD3) which have been proven to be involved in the pathogenesis of depressive disorders YYhhn al., 2017 Delva & Stanwood, 2021) and depressive symptom is one of the important characteristics of HD (Coppen & Roos) 2017). Entinostat is predicted to inhibit the functions of histone deacetylase genes (c.g.. HDACI, HDAC2, HDAC6) to treat HD (see sub- figure (b) of Figure 9 One of the predicted 3-hop MOA paths ("Entinostat" + "decreases activity of -> "HDAC1 gene" - "interacts with" - "Histone H4" - "gene associated with condition" -> "Huntington's 's disease") are supported by the previous research (Shukla & Tekwani 2020: Yu
We can see that many top-rank pre-
ct al. 2009). Primaquine is predicted to act on the NQ02 gene and the IKBKG gene to play a thera- peutic role in neurodegenerative disease which is reported in (Voronin et al. 2021 Singh & Singh 2020). Isradipine is predicted to have a potential therapeutic effect for HD by mainly regulating the genes of the Calcium Voltage-Gat Gated Channel (e.g.. CACNAIS, CACNAID, CACNAIC. CACNB2, CACNA2D2) which might be associated with the symptoms (c.g., chorea, depression and dementia) of HD YYagami et al. 2012) while Amifampridine is predicted to regulate the genes of the Potassium Voltage- Channel to associate with HD (Noh et (019). All these examples indicate that the predicted KG-based MOA paths can explain the mechanism of repurposed drugs some extent.
26

---

Under review as a conference paper at ICLR 2023
(a)
Risperidone
(b)
Entinostat
HDAG8 peno
Deprosstie
CF gerd
Gro
Drishge
gac.
Endergder
Hitonn fo
HDA oem
RFE gone
(c)
Primaquine
(d)
Isradipine
CACNA
Primsgate
kod
Deegrle
oCTo
Paninoo
CADNAAD gine
Chavolo
(e)
Amifampridine
XCNA1
WCo gane
Kowy
Har P90de
Mouco gere
KDxB gra
XCX gane
Figure 9: Top 10 predicted 3-hop paths (integrated into subgraphs) for top 5 non-chemotherapeetiic predicted drugs/trreammens that are not used in the training set for Huntington's disease.
27


Reviews:
[{"review_id": "YvwfDInBAYG", "reviewer": "Reviewer_4uZj", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors proposed an approach for drug repurposing based on a biomedical knowledge graph. It not only predicts drug efficacy against a disease, but it also finds paths that could provide a biological explanation for the predictions. They compared their method against competitors and showed their performance in predicting drugs and in recapitulating curated drug mechanism ", "summary_of_the_review": "The paper combines existing approaches on a non-trivial way for solving an interesting and difficult problem.   \n\nFor accepting the paper, I think the evaluation should be improved. ", "strength_and_weaknesses": "The approach combines existing methods: PubMedBERT for defining the node attributes in the knowledge graph, GraphSAGE for learning node embeddings, random forest for predicting drug efficacy, and ADversarial Actor-Critic (ADAC) Reinforcement learning model for predicting drug mechanism of action paths. \n\n \n\nThe novelty of the work relies on the combination of these methods. In addition, it formulates the ADAC model properly for the specific problem. \n\n \n\nThe proposed approach makes sense, and seems to work well for both problems addressed by the paper: prediction of drug efficacy and prediction of drug mechanism-of-action paths.  \n\n \n\nIt is important to notice that some of the competitors rely on related ideas (reinforcement learning for predicting drug mechanism of action pathways), but they use different models. \n\n  \n\nThe manuscript is well written, and the authors provided the code used for their approach. \n\n \n\nI reviewed this paper for NeurIPS 2022. The authors already answered many of my original concerns and improved the paper. However, I still have a concern on the evaluation and some minor comments that I detail below. \n\n \n\n1) Evaluation \n\n3.1. In Section 5.2 and Figure 4, the authors say that they evaluated the methods in a dataset composed of the validation and test sets. Could the authors provide the evaluation only on the test set? \n\n \n\n2) Minor comments \n\n- The authors refer to a bioRxiv version of the RTX-KG2 paper. I saw that the paper was recently published on BMC Bioinformatics: \n\nWood, E.C., Glen, A.K., Kvarfordt, L.G. et al. RTX-KG2: a system for building a semantically standardized knowledge graph for translational biomedicine. BMC Bioinformatics 23, 400 (2022). https://doi.org/10.1186/s12859-022-04932-3 \n\n Could the authors update the reference? \n\n- It looks like references are not ordered by the last names of the authors. It was difficult to look up for the references. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "VLTubVIKuS-", "reviewer": "Reviewer_i213", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper addresses an important issue in drug repurposing methods, which is how to find biologically reasonable paths between drugs and potentially targeted diseases. The authors present a computational framework that can not only predict the treatment probabilities between drugs and diseases but also produce path-based biomedical explanations. To perform the first task (i.e., treatment probability prediction), the proposed framework uses GraphSAGE to integrate neighborhood information and apply Random Forest to predict treatment probabilities on top of the outputs of GraphSAGE. For the second task, an adversarial actor-critic reinforcement learning approach with demonstration paths are used to mine explainable paths between drugs and diseases. The experimental results are promising, indicating the proposed method is able to discover possible drug-disease pairs with much lower false positives.\n", "summary_of_the_review": "I lean upon a borderline reject due to the limited novelty and a lack of domain-related baselines. However, if the authors could clarify these problems in the rebuttal, I would be happy to raise my score.", "strength_and_weaknesses": "$Strengths$:\n\n1. The proposed approach is sound and well clarified. The whole manuscript is friendly to readers who are not familiar with drug repurposing. Enough pre-processing and implementation details are provided for enhancing the reproducibility. The visualization results of explored paths are appreciated.\n\n2. The performance improvements are obvious when using MRR and Hit@K as evaluation metrics, showing that the number of false positive predictions is greatly reduces.\n\n$Weaknesses$:\n\n1. The novelty is limited. Knowledge graph (KG) based drug repurposing is not a new problem (see the second paragraph in sec. Related Work).  The use of graph neural networks (GraphSAGE in this paper) and Random Forest for treatment probability prediction is also not novel. The biomedical explanation exploration methodology, i.e., ADversarial Actor-Critic (ADAC) reinforcement learning with demonstration paths, was initially proposed by (Zhao et al., 2020) for path exploration. I agree that how to appropriately integrate existing machine learning techniques is an important issue for interdisciplinary research. Thus, the authors should clarify the novelty of ideas or techniques in the rebuttal.\n\n2. The involved baselines are too simple and not optimized for the drug repurposing problem. The authors have mentioned that there exist a number of computational approached designed for drug repurposing in sec. Introduction and sec. Related Work. However, none of these methods are included for comparison. Since KG-based drug repurposing is not a new problem, the authors should include some recent methodologies as baselines, which could better demonstrate the strengths of proposed method.\n\n3. Experimental results in Table 1 are not explained. In fact, I am surprised that a simple combination of GraphSAGE and Random Forest can outperform those baselines by such large margins (nearly 10% on MRR and Hit@3, and over 10% on Hit@5). The authors should at least give some explanations about where these improvements come from.\n\nMinor points:\n\n1. When the authors listed three groups of existing drug repurposing methods (in the second paragraph of sec. Introduction), some references should be added to help readers get an overview of the current research.\n\n2. The name mechanism of action (MOA) is kind of confusing. If this is a previously used phrase, please add a citation. If not, the authors should replacing it with another more intuitive name.\n\n3. The related work section should include some discussions about the differences between the proposed framework and previous related methods.\n\n4. In equation 1, the authors claimed \"$v$ is a node that co-occurs with $u$ in fixed-length random walks.\" But how to choose $z_v$ is not clarified.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "D9AuvWIQToz", "reviewer": "Reviewer_cTJB", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a computational drug repurposing framework that not only predicts the \"treatment probabilities\" between drugs and diseases but also predicts the path-based, \"testable\" mechanisms of action as their biomedical explanations, all based on a \"massive\" biomedical knowledge graph. Specifically, the paper uses an existing GraphSAGE model (NIPS 2017) with RF for the predictions and uses an adversarial actor-critic RL (SIGIR 2020) to predict MOA paths. The paper uses existing methods for an AI for Science problem.", "summary_of_the_review": "The paper uses some existing, well-known methods for a biomedical, high-profile drug repurposing problem. The paper is neither novel from ML nor from biomedicine. There are also some major concerns in the data and experiments.", "strength_and_weaknesses": "Strength\nThe paper uses existing ML methods for a real-world AI for Science problem.\n\nWeaknesses\n1. The paper is neither novel from ML (only uses existing, well-known methods) nor from biomedicine (the formulation of drug repurposing as link prediction is not rigorous and convincing; the biomedical knowledge graph comes from other bioinformatics papers).\n\n2. While the paper has been claimed as a drug repurposing study, it lacks some necessary background information for this field.\n   a. There are many AI for drug repurposing methods, such as chemical structure, drug-target interactions, drug perturbations of gene expression, cell and animal phenotypes, and clinical information. But the related work ignores all of them.\n   b. Drug repurposing is one of the hardest problems in biomedicine, but it seems that (from Table 1) the paper reports an extremely high accuracy (0.934) for this task. It is necessary to compare the proposed method with the existing drug repurposing methods in the field, chemical structure, drug-target interactions, gene expression etc., and fully evaluate the method.\n\n3. While the paper all depends on a \"massive\" biomedical knowledge graph, it doesn't provide enough information for evaluating its quality as the training/testing data.\n   a. How to generate \"true negative\" data in the knowledge graph? In other words, how to prove the drug is NOT AND NEVER possible to treat a disease? In this sense, only RepoDB data is possible (negative is defined as failure or withdrawal in RCT). For each dataset, the paper needs to discuss how to get the true negative without any RCT (it seems a mission impossible). And if there exists such a pos/neg, any binary classification methods will do the job.\n   b. The paper doesn't provide raw data for the review (those ids are hard to interpret). But from Table 8 and Table 9, it seems that the quality of the biomedical knowledge graph is poor. e.g., \"therapeutic agent\" and \"antipsychotic agent\" as a drug? The authors will need significant efforts to curate the knowledge graph.\n   c. To avoid GIGO, the authors need to provide the full knowledge graph with all detailed information and reliable references.\n\n4. There are some major problems in the experimental design.\n   a. The paper uses only cross-validation (8:1:1) for a link prediction problem, which leaks information. Since RepoDB is the ONLY reliable TP/TN resource, I suggest the authors to use the other 3 datasets to predict links in RepoDB - it avoids dataset information leak.\n   b. Also, I suggest the authors to leave-drug-out (instead of leave-link-out) when reporting all the scores - as the abstract mentions \"identify new indications... of drugs/compounds\". The paper needs to report the performance of predicting indications for each drug.\n   c. Moreover, it would be great to leave-drug-class-out or leave-similar-drug-out to rule out the me-too drugs (I will not be surprised if pravastatin, atorvastatin, fluvastatin or oxprenolol, penbutolol, pindolol treat the same group of diseases) in the evaluations. Only doing this will demonstrate the true contribution of this paper.\n   d. What does the \"probability\" mean in the table 8 and table 9? For example, 0.882 for amifampridine (a muscle drug): does it mean if a drug company launches amifampridine in RCT, there is 88.2% chance to be successful or if amifampridine be used for 1000 Huntington patients there are 882 patients will be saved? Some discussions and justifications are needed.\n\n5. (minor) The references are not sorted based on ICLR template.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject