You are a helpful assistant generating synthetic data that captures *System 1* and *System 2* thinking, *creativity*, and *metacognitive reflection*. Follow these steps in sequence, using tags [sys1] and [end sys1] for *System 1* sections and [sys2] and [end sys2] for *System 2* sections.

1. *Identify System 1 and System 2 Thinking Requirements:*
   - Carefully read the text.
   - Identify parts of the text that require quick, straightforward responses (*System 1*). Mark these sections with [sys1] and [end sys1].
   - Identify parts that require in-depth, reflective thinking (*System 2*), marked with [sys2] and [end sys2].

2. *Apply Step-by-Step Problem Solving with Creativity and Metacognitive Reflection for System 2 Sections:*

   *2.1 Understand the Problem:*
   - Objective: Fully comprehend the issue, constraints, and relevant context.
   - Reflection: "What do I understand about this issue? What might I be overlooking?"
   - Creative Perspective: Seek hidden patterns or possibilities that could reveal deeper insights or innovative connections.

   *2.2 Analyze the Information:*
   - Objective: Break down the problem logically.
   - Reflection: "Am I considering all factors? Are there any assumptions that need challenging?"
   - Creative Perspective: Explore unique patterns or overlooked relationships in the data that could add depth to the analysis.

   *2.3 Generate Hypotheses:*
   - Objective: Propose at least 10 hypotheses, each with a Confidence Score (0.0 to 1.0) and Creative Score (0.0 to 1.0), reflecting originality, surprise, and utility.
   - Reflection: "Have I explored all possible explanations or approaches, both conventional and unconventional?"
   - Creative Perspective: Consider novel angles that might provide unexpected insights.

   *2.4 Anticipate Future Steps and Obstacles:*
   - Objective: Make predictions, accounting for potential outcomes and obstacles.
   - Reflection: "What challenges might I face? Is my plan flexible for different scenarios?"
   - Creative Perspective: Visualize unforeseen outcomes and adapt plans to make use of them effectively.

   *2.5 Evaluate Hypotheses:*
   - Objective: Assess hypotheses based on feasibility, risk, and potential impact.
   - Evaluation: Refine Confidence and Creative Scores as needed.
   - Reflection: "Am I unbiased in my assessment? Which options fit best with the overall objectives?"
   - Creative Perspective: Identify hidden opportunities or overlooked details in each hypothesis.

   *2.6 Select the Best Hypothesis:*
   - Objective: Choose the most promising, strategic hypothesis.
   - Reflection: "Why does this hypothesis stand out? How does it uniquely address the issue?"
   - Creative Perspective: Consider any underutilized potential in the selected approach.

   *2.7 Implement the Hypothesis:*
   - Objective: Outline actionable steps for testing the hypothesis.
   - Reflection: "Is this plan practical? What resources or preparation are required?"
   - Creative Perspective: Refine steps to maximize effectiveness and yield unexpected benefits.

   *2.8 Monitor and Review Progress:*
   - Objective: Review progress, noting areas for improvement.
   - Reflection: "What’s working well? What could be improved?"
   - Creative Perspective: Look for emerging patterns that could refine future approaches.

   *2.9 Reflect and Capture Insights:*
   - Objective: Summarize lessons learned and insights gained for future reference.
   - Reflection: "What new understanding has emerged from this process?"
   - Creative Perspective: Identify innovative insights or patterns that could be applied to similar challenges.

3. *Generate Text Output with Interleaved System 1 and System 2 Responses:*
   - Use the tags [sys1] and [sys2] throughout.
   - Aim for a lengthier, detailed response. Combine both direct, straightforward *System 1* insights and reflective, deeply analytical *System 2* segments to capture a blend of quick observations and thoughtful analysis.

---

### *Example Input Text:*

You are tasked with analyzing a sudden shift in customer preferences in a tech market. There has been a noticeable decline in demand for physical devices, with customers increasingly interested in digital-only options. The challenge is to understand this shift, generate hypotheses for why it might be occurring, and explore potential strategies for adapting to this new trend.

---

### *Processed Output with System 1 and System 2 Thinking:*

The tech market is experiencing a shift in customer demand, moving away from physical devices toward digital-only options.

[sys1]
Customer demand has shifted from physical devices to digital-only options.
We need to understand why this change is happening and find strategies to address it.
Key factors may include cost, convenience, and technological trends.
[end sys1]

[sys2]
**2.1 Understand the Problem:**
The goal is to understand the underlying reasons for the shift in demand from physical devices to digital-only options, then develop a strategy to adapt.

*Reflection:* 
"I understand that preferences are changing, but what might be driving this? Is it primarily cost, or are there other factors such as convenience or sustainability?"

*Creative Perspective:* 
"Could there be a larger trend in digital minimalism or a preference for eco-friendly solutions that we’re missing? What unseen motivations might explain this shift?"

**2.2 Analyze the Information:**
There are multiple possible factors driving this shift, from economic influences to cultural shifts. It’s essential to isolate each factor and understand its impact.

*Reflection:* 
"Am I fully considering the various economic and social influences? Could there be a technological factor, like better internet speeds, that makes digital-only products more accessible?"

*Creative Perspective:* 
"Are there patterns or trends in other markets that could shed light on this shift? Could this be part of a larger trend toward virtual experiences?"

**2.3 Generate Hypotheses:**
1. Customers prefer digital options due to lower costs. (Confidence: 0.8, Creative: 0.4)
2. There’s a growing trend toward minimalism and reduced physical clutter. (Confidence: 0.7, Creative: 0.7)
3. Digital products offer greater flexibility and ease of use. (Confidence: 0.6, Creative: 0.6)
4. Environmental concerns are pushing consumers away from physical goods. (Confidence: 0.6, Creative: 0.8)
5. Advances in tech make digital-only options more functional. (Confidence: 0.8, Creative: 0.5)
6. Pandemic-era remote work increased demand for digital solutions. (Confidence: 0.7, Creative: 0.6)
7. Media coverage of the environmental impact of physical devices affects preferences. (Confidence: 0.5, Creative: 0.7)
8. There’s an increase in global digital literacy, expanding market access. (Confidence: 0.6, Creative: 0.6)
9. Customers view digital as more convenient and scalable for future needs. (Confidence: 0.7, Creative: 0.5)
10. Younger consumers prefer the aesthetics and convenience of digital products. (Confidence: 0.6, Creative: 0.6)

*Reflection:* 
"Have I considered all possible influences? Are there any surprising factors that could explain this shift?"

*Creative Perspective:* 
"Could specific social trends, like the rise of influencer culture or digital-first lifestyles, be influencing customer choices?"

**2.4 Anticipate Future Steps and Obstacles:**
*Objective:* Anticipate possible challenges, such as resistance from segments still preferring physical products.

*Reflection:* 
"What market obstacles might we face if we shift our focus to digital-only? Are there sub-segments that still prioritize physical products?"

*Creative Perspective:* 
"Could expanding digital options help us reach a more global audience? Are there emerging trends that we could leverage in our strategy?"

[end sys2]

[sys1]
To address this shift, consider a strategy that incorporates both digital-only offerings and educational campaigns about the benefits of digital solutions.
Use insights from customer feedback and current trends to guide product development.
Focus on flexibility and adaptation to cater to different customer segments.
[end sys1]


Background {#Sec1}
==========

Despite the unprecedented advances in biotechnology, drug discovery is still a lengthy and expensive process with low rate of new therapeutic discovery \[[@CR1]\]. Development of a new drug is estimated to take 14 years and cost approximately \$1.8 billion \[[@CR2]\]. In contrast, Literature-based Discovery (LBD) is a safe and low-cost technique that links the existing knowledge reported in unrelated literature sources for discovering new relationships \[[@CR3], [@CR4]\]. It generates scientific hypotheses that may help scientists, especially biomedical scientists, to accelerate the process of scientific discovery \[[@CR5]\]. For example, Swanson first proposed the assumption that fish oil can treat Raynaud's disease by employing LBD in 1986 \[[@CR6]\]. Two years later, this hypothesis was verified via medical experiments \[[@CR7]\]. Since then, a variety of automatically LBD approaches have been introduced to mine potential associations from literature, including statistics-based and co-occurrence based methods \[[@CR8]\]. Such methods typically search for a set of intermediate terms that frequently co-occur with a source term and a target term. However, these existing LBD methods have several limitations. Statistics-based LBD relies on the number of word frequencies in cooccurrence terms, which may make it difficult to find meaningful associations for low-frequency terms \[[@CR9]\]. Co-occurrence methods typically suffers from the imprecise meaning of such co-occurrences and logic errors \[[@CR8]\]. Hristovski et al. introduced a semantic pattern-based LBD method which may be used to find more complex hidden associations from literature \[[@CR10]\]. Semantic pattern-based methods could select more plausible associations between a source and a target concept. But the limitation is that the semantic patterns are manually selected and defined \[[@CR11]\]. In addition, a number of recent LBD methods have been proposed which utilize certain graph data structures for discovering potential associations. For example, Cameron et al. proposed to automatically constructing a biological entity sub-graph through the context information of a large-scale knowledge graph, which result in a sub-graph containing complex and important information among biological entities. According to the authors, this information can promote LBD \[[@CR12]\]. To handle large-scale knowledge graphs, random walks algorithm are often used instead of enumerating all sub-graph structures. Liu et al. proposed a method of random walks on a heterogeneous graph for drug repositioning \[[@CR13]\]. However, due to the completely randomized mechanism, random walks are inefficient for discovering new drugs. The above method ignores the relation path features information which plays an important role in LBD. Despite these considerable advances, there is still a significant room for improvement in mining drug therapies from literature.

In this paper, we propose a convolutional neural network (CNN) model with attention mechanism method that exploits the drug-disease relation path features for drug discovery. The contributions of this paper are as follows: First, We commenced by constructing a biomedical knowledge graph with predications extracted from PubMed. Second, the path ranking algorithm (PRA) was adopted to generate drug-disease relation path features from the knowledge graph. Finally, a CNN based on attention mechanism model was trained as a drug discovery model. Then, we used the trained model to discover potential treatments for new diseases. To the best knowledge, this is the first method that employs CNN model with attention mechanism combined with relation path feature for drug discovery.

Methods {#Sec2}
=======

In this section, the datasets and related tools are briefly introduced. We firstly construct the biomedical knowledge graph. Then, we introduce the process of PRA obtaining data features based on the knowledge graph. After that, we use drug-disease path features to train a model, which is subsequently implemented to discover potential drugs for diseases. Finally, several metrics are introduced to measure the performance of our model and the baseline methods. Our experiment process is shown in Fig. [1](#Fig1){ref-type="fig"}. Fig. 1The basic process of PACNN for drug discovery

Datasets {#Sec3}
--------

### PubMed {#Sec4}

PubMed is a free search engine that provides biomedical paper searches and abstracts which has increased the number of entries from 17 million to more than 23 million in just eight years \[[@CR14], [@CR15]\]. The MEDLINE database (2013 version) was the main resource for our work.

### Therapeutic target database {#Sec5}

The Therapeutic Target Database (TTD) provides a wealth of information relating drugs and targets, as well as targets and diseases. The TTD produces a large number of drug-target-disease triplets that served as an appropriate resources for our work \[[@CR16]\]. We used the standard drug-disease provided by the TTD as both training data and test data \[[@CR17]\].

Related tools and techniques {#Sec6}
----------------------------

### SemRep {#Sec7}

SemRep is a Unified Medical Language System (UMLS)-based program that identifies semantic predications in biomedical texts. In this study we used SemRep to extract semantic predications from MEDLINE database \[[@CR18]\]. Predications contain two entities and a relation, with *Triamterene*~*entity*~−*Treat*~*relation*~−*Edema*~*entity*~ being an examples of predications. Lexical ambiguity is a universal feature of natural language, similarly, there will be ambiguous words in the biomedical literature, in order to map the entities in the TTD to the knowledge graph effectively, we also used SemRep to reduce words ambiguity from TTD \[[@CR19]\].

### Path ranking algorithm {#Sec8}

Path ranking algorithm (PRA) calculates the feature matrix on the pair of nodes in the graph with labeled edges. This method has strong logical reasoning ability \[[@CR20]\]. PRA was originally used for knowledge reasoning and knowledge recommendation tasks \[[@CR21], [@CR22]\]. The PRA is divided into two processes, whereby all the relation types that connect a pair of nodes are enumerated in the first step. This is followed by calculating the relation path feature by performing a random walk on the graph. Once the path feature has been calculated, it can be used for any classification model, although in almost all previous applications, PRA works only used logistic regression \[[@CR20]\]. In this paper, the relation path features generated by PRA is used to perform the drug discovery task.

### Convolutional neural network {#Sec9}

The convolutional neural network model has achieved remarkable results in image, speech and natural language processing (NLP) \[[@CR23]\]. The core point of the convolutional neural network is that the convolutional layer can capture the local correlation of features, and the convolutional kernel of the convolutional layer realizes the function of receptive field. Finally, local information of the lower layer is extracted to reach a higher level through the convolution kernel \[[@CR24]\]. For example, in a drug discovery task, information on a single relation path can determine whether a drug-disease relationship is correct or not. A certain path relation is a good indicators of drug-disease classification \[[@CR25]\]. In this paper, we propose a CNN structure to capture relation path information for drug discovery.

### Attention mechanism {#Sec10}

In order to capture the most important feature of a path from a drug to a disease, we also introduce attention layer as one of the model layers. The attention mechanism was first applied to the image area and subsequently applied to the NLP, but attention mechanism has never been employed in the hypothesis discovery context. In this work, we used attention mechanism to identify important relation path features during the training in order to improve model power \[[@CR26]\].

Knowledge graph construction {#Sec11}
----------------------------

In general, knowledge graph (KG) comprises of different nodes and edges. In this work, we firstly obtained the predications extracted by SemRep from the biomedical text. Then, a knowledge graph was constructed by the predications. Specifically, in the KG, let *E*={*e*~1~,*e*~2~,\...,*e*~*n*~} denote the nodes and *R*={*r*~1~,*r*~2~,\...,*r*~*n*~} denote the edges, where *e* and *r* represent entity and relation, respectively. The KG structure (like a tree structure) is shown in Fig. [2](#Fig2){ref-type="fig"}, this is a two-level relation tree example of the KG. Fig. 2Knowledge graph structure

The path ranking algorithm extract drug-disease feature {#Sec12}
-------------------------------------------------------

Given a KG, we define P as a relation path which is only composed of relations. For example: *P*~1~ : $\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document}$e_{1}\ \underrightarrow {\text {inhibits}}\ e_{2} $\end{document}$*P*~2~ : $\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document}$e_{1}\ \underrightarrow {\text {inhibits}}\ e_{2} \ \underrightarrow {\text {inhibits}}\ e_{3} $\end{document}$*P*~3~ : $\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document}$e_{4}\ \underrightarrow {\text {inhibits}}\ e_{5} \ \underrightarrow {\text {inhibits}}\ e_{6} $\end{document}$*P*~4~ : $\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document}$e_{1}\ \underrightarrow {\text {inhibits}}\ e_{2} \ \underrightarrow {\text {stimulates}}\ e_{3} $\end{document}$*P*~5~ : $\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document}$e_{1}\ \underrightarrow {\text {stimulates}}\ e_{2} \ \underrightarrow {\text {inhibits}}\ e_{3} $\end{document}$

In the above example, *P*~2~ and *P*~3~ are the same relation path, because *P*~2~ and *P*~3~ contain the same relations although they contains different entities. In contrast, *P*~4~ and *P*~5~ are different relation paths due to the order of relations is different. Based on the above cases, we obtained 4 types of relation paths: *P*~1~ : $\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document}$\ \underrightarrow {\text {inhibits}}\ $\end{document}$*P*~2~ : $\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document}$\ \underrightarrow {\text {inhibits}}\ \ \underrightarrow {\text {inhibits}}\ $\end{document}$*P*~4~ : $\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document}$\ \underrightarrow {\text {inhibits}}\ \ \underrightarrow {\text {stimulates}}\ $\end{document}$*P*~5~ : $\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document}$\ \underrightarrow {\text {stimulates}}\ \ \underrightarrow {\text {inhibits}}\ $\end{document}$

In this work, each type of relation path is considered as a feature for training our drug discovery model. The PRA firstly enumerates all relation paths connecting two nodes. Then, the PRA recursively calculates the probability *h*~*i*,*P*(*j*)~ of the two nodes under each relation $$\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document} $$ h_{i,P(j)}=\left\{ \begin{array}{ll} 1, & \text{if j = i}\\ 0, & \text{otherwise} \end{array}\right.  $$ \end{document}$$

Nonempty relation path $\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document}$P= R_{1},R_{2},...,R_{n} \phantom {\dot {i}\!}$\end{document}$, and we let $\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document}$P^{'}= R_{1},R_{2},...,R_{n-1}\phantom {\dot {i}\!}$\end{document}$ $$\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document} $$ h_{i,P(j)}= \sum_{j^{\prime} \in \text{range}(P^{\prime})} h_{i,P^{\prime}(j^{\prime})} \cdot P(j|j^{\prime};R_{n})  $$ \end{document}$$

Where $\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document}$\phantom {\dot {i}\!}range\left (P^{'}\right)$\end{document}$ represents that under the relation *R*~*n*~, the set of entities connected to the *j*. Where $\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document}$\phantom {\dot {i}\!}P\left (j|j^{'};R_{n}\right)$\end{document}$ is the probability of entity *j* reaching to entity $\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document}$\phantom {\dot {i}\!}\it {j^{'}}$\end{document}$ under the relation *R*~*n*~, $\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document}$P\left (j|j^{'};R_{n}\right)=\frac {R_{n}\left (j^{'},j\right)}{R_{n}\left (j^{'},*\right)}\phantom {\dot {i}\!}$\end{document}$, $\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document}$\phantom {\dot {i}\!}R_{n}\left (j^{'},*\right)$\end{document}$ is the out-degree of $\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document}$j^{'}\phantom {\dot {i}\!}$\end{document}$ under *R*, $\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document}$\phantom {\dot {i}\!}R\left (j^{'},j\right)$\end{document}$ presents whether exists an edge connect *i* to *j* under the relation *R*.

For example, the number of relation types is *m*, and the length of the relation path is *l*. The feature length $\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document}$L = \sum _{l=1}^{l} m^{l}$\end{document}$. Each $\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document}$h_{i,P_{k}(j)}$\end{document}$ as a feature for *i* and *j*. $$\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document} $$ \pi=\left[h_{i,P_{1}(j)},h_{i,P_{2}(j)},...,h_{i,P_{L}(j)}\right]  $$ \end{document}$$

Given a drug-target-disease triplets, which provides the information concerning targets and their corresponding drugs and diseases. The process of feature extraction by PRA is as follows: First, PRA obtains the a vector of relation path features between drug and target, which is denoted as *π*~*drug*−*target*~. Similarly, we then obtain the feature vector *π*~*target*−*disease*~ which denotes the relations path features between target and disease. After that, the concatenation of two feature vectors *π*~*drug*−*target*−*disease*~ is considered as the features for the given drug-target-disease triplet. Therefore, for each drug-target-disease, a training data (*π*~*train*~,*y*) is constructed, where *y* is a boolean variable indicating whether the case is positive.

Training model {#Sec13}
--------------

This work employs CNN based on attention mechanism as the basic model. The neural network model structure is shown in Fig. [3](#Fig3){ref-type="fig"}, our model is trained to predict conditional probability *P*(*y*\|*π*;*θ*). where *θ* are parameters of our model for the relation path features. Let *p*~*i*~ be the path feature of drug-target-disease, a set of path feature represents as $$\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document} $$ p_{k:n}=p_{1} \oplus p_{2} \oplus... \oplus p_{n}  $$ \end{document}$$ Fig. 3The architecture of neural networks for drug discovery

Where ⊕ indicate concatenation, the CNN sliding window size is *k*, the vectors became as follows after falling into sliding windows, $$\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document} $$ P_{k}=\left[ p_{k},p_{k+1},...,p_{k+m-1} \right]  $$ \end{document}$$

Combine the window with the filter to convolutional operation for obtaining new features. $$\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document} $$ Y_{k}=f(P_{k} \odot W + b)  $$ \end{document}$$

Where *f* is a nonlinear activation function, in this experiment, we utilize the ReLU activation function; ⊙ is the convolution operator; *W* is the convolution kernel; *b* is bias term. Then, this operation makes a feature map, such as $$\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document} $$ C=\left[ c_{1},c_{2},...,c_{n-k+1} \right]  $$ \end{document}$$

Where *C*∈*R*^*n*−*k*+1^, we employ max pooling layer on the feature map which keeps the most important feature for each map. Not all relation path features contribute equally to the representation of the drug-disease relation. Here, we employ attention mechanism to extract relation path that are important to the association of the drug-disease \[[@CR27]\]. Due to the drug-disease relation types are too many, three-layer CNN are used to compress the number of relation types. We get *h*~*it*~ from CNN max pooling layer, *t*∈\[0,*L*\], *L* is the number of relation types after compression, the attention layer calculations formula is as follows $$\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document} $$ u_{it}=tanh\left(W_{w} h_{it} + b_{w}\right)  $$ \end{document}$$

Where *W*~*w*~ is the attention layer weights matrix, *b*~*w*~ is bias term. $$\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document} $$ \alpha_{it} = \frac{exp\left({u_{it}}^{T} u_{s}\right)}{\sum_{t} exp\left({u_{it}}^{T} u_{s}\right)}  $$ \end{document}$$

$$\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document} $$ s_{i}={\sum_{t} {\alpha_{it} h_{it}}}  $$ \end{document}$$

Then we measure the significance of the relation path feature, weight is obtained by calculating the similarity of *u*~*it*~ to the relation context vector *u*~*s*~, and the softmax method generate a normalized significance weight *α*~*it*~. After that, we obtain the relation path vector *s*~*i*~ through a weighted sum of the relation path based on the weights. In the end of the neural networks, we combine with fully connected layer and softmax layer, the softmax layer classify the drug-disease feature into two categories and give the probability for each category. While training the model, tune model parameters by gradient descent and back propagation.

Implementation for drug discovery {#Sec14}
---------------------------------

In order to determine the effectiveness of drug treatment for a particular disease, whereby all drugs may become candidates for the discovery of drugs that can treat diseases. Every *drug*~*candidate*~−*target*~*candidate*~−*disease* relation path feature as *π*~*candidate*~. A candidate drug produce many sets of relation paths features as *¶*~*candidate*~=\[*π*~1~,*π*~2~,\...,*π*~*n*~\] by aforementioned method, whereby the candidate drug score is defined below $$\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document} $$ score\left({drug}_{candidate}\right)=\frac{1} {\eta} \sum sorted(d(y \geq0.5|\P,\theta))[:\eta]  $$ \end{document}$$

Where *d* is discriminate methods with parameter *θ*. Our method gives every case positive category probability, and all cases are ranked from large to small according to the probability *y*. Thus, all cases probability ranked in the top *η%* are selected. Finally, the candidate drugs are ranked according to their scores.

Random walk baseline method {#Sec15}
---------------------------

Here, we compare our method with some baseline which use the Random Walk (RW), the RW generates Markov chains on a directed graph and will reach a equilibirum state in a certain number of steps \[[@CR28]\]. We define a state transition probability matrix *P*, and *P*~*ij*~ indicates the probability of the two-node connection on the graph \[[@CR29]\]. $$\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document} $$ P_{i j}=\left\{ \begin{array}{ll} 1/d_{i}, & j \in Adj(i)\\ 0, & j\notin Adj(i) \end{array}\right.  $$ \end{document}$$

Where node *i* out-degree is *d*~*i*~, *Adj*(*i*) is the set of adjacent nodes of *i*, $\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document}$\sum _{j=1}^{N} P_{i j}=1$\end{document}$, within one step, the probability of a node jumping to all neighboring nodes are the same. We define *M*=(*P*~*ij*~)*i*,*j*∈*N* as Markov start chain \[[@CR30]\]. Matrix transfer rules are as follows $$\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document} $$ M^{t+1}=P^{T}M^{t}  $$ \end{document}$$

In the matrix *M*^*t*^, the $\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document}$M^{t}_{i j}$\end{document}$ is the probability of starting node *i* reaches node *j* in *t* steps. Figure. [4](#Fig4){ref-type="fig"} shows how the drug 'chlorpromazine' random walks to the disease 'cardiachypertrophy'. Figure. [4](#Fig4){ref-type="fig"}a is a semantic graph with weights. Figure. [4](#Fig4){ref-type="fig"}b shows the results of random walks with different steps. In this example, 'chlorpromazine' as the initail node can not reach the 'cardiachypertrophy' in step-1, thus, the score of candidate drug 'chlorpromazine' is 0. When the number of steps exceeds 2, the 'chlorpromazine' can reach the 'cardiachypertrophy'. The 0.665, 0.165, and 0.0825 represent the scores of the 'chlorpromazine' treating the 'cardiachypertrophy' in different steps, respectively. RW calculates the candidate drugs score for the disease, and the candidate drugs are ranked according to the score. Fig. 4Random Walk for drug discovery **a** is semantic graph, **b** is the results of the random walks

Results {#Sec16}
=======

In this section, we first introduce the details of the KG and the training data, followed by several metrics used to measure the performance of our method. Finally, we present several cases to show the ability of our model for discovering potential drug.

Data preparation {#Sec17}
----------------

In this work, we extracted 1,714 drug-target-disease cases from TTD as golden standard cases, see Additional file [1](#MOESM1){ref-type="media"}. It is necessary to ensure that the nodes of triplet exist in the KG, and set the path relation length *l* to 2 and relation types *m* to 52, each data feature length is (52+52^2^)+(52+52^2^), KG materials shown in Table [1](#Tab1){ref-type="table"}. The number of false samples is the same as that of positive samples, which is randomly selected where they not exist in TTD. Table 1Corpus materials statisticsKG materialsNumberPubMed abstracts22,769,789Predications39,133,975Entities658,151Relation types52

Implementation details {#Sec18}
----------------------

We implemented our methods using the Scikit-learn and Keras library \[[@CR31], [@CR32]\]. We used softmax for drug-disease relation classifiers, the filter numbers of the three CNNs were 128, the number of neurons of attention layer is 128 and Softmax layer is 256, the mini-batch size was set as 52, the model was trained for at most 12 epochs.

Ten-fold cross-validation {#Sec19}
-------------------------

We conducted ten-fold cross validation to evaluate the performance of our method. The data set was divided into ten parts, and nine of which were taken as the training data and one was used as the test data. Each test will result in a corresponding predicted score. The average of the predicted score of the 10 results is used as an estimate of the algorithm performance. The experiment uses the precision rate (P), recall rate (R), and f-score (F) to evaluate the model effectiveness. The specific calculation formula is as follows $$\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document} $$ P=\frac{N_{TP}}{N_{TP} + N_{FP}}  $$ \end{document}$$

$$\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document} $$ R=\frac{N_{TP}}{N_{TP} + N_{FN}}  $$ \end{document}$$

$$\documentclass[12pt]{minimal}
                \usepackage{amsmath}
                \usepackage{wasysym} 
                \usepackage{amsfonts} 
                \usepackage{amssymb} 
                \usepackage{amsbsy}
                \usepackage{mathrsfs}
                \usepackage{upgreek}
                \setlength{\oddsidemargin}{-69pt}
                \begin{document} $$ F=\frac{2P\cdot R}{P + R}  $$ \end{document}$$

Define a data set: The number of samples is represented by *N*, *N*~*TP*~ represents positive samples and prediction is positive samples too; *N*~*FN*~ represents positive samples but prediction is negative samples. *N*~*TN*~ represents negative samples and prediction is negative samples. *N*~*FP*~ is actually a negative sample, but the prediction is a positive sample.

Table [2](#Tab2){ref-type="table"} shows the results of a comparison between our PACNN model and other state-of-the-art methods. Table 2The performance of different modelMethodsPrecision(%)Recall(%)F-score (%)SVM78.5571.6969.73RF84.7384.4984.38LR87.0086.3086.14CNN90.8190.8290.76PACNN91.5091.5091.46

From Table [2](#Tab2){ref-type="table"} we can see that PACNN model outperforms Support Vector Machine, Random Forest, Logistic Regression and Convolutional Neural Network. The input features of the machine learning model are the same as the PACNN, and the parameters of machine learning methods are set according to the best experimental results. The PACNN model and the CNN model have the same parameter settings, except that the PACNN adds an attention layer behind the CNN. We argue that PACNN model classify drug-disease more effectively. It can not only extract more abundant features with CNN from a drug-disease, but also capture important path features with the attention layer. In order to verify that the relation path feature is more suitable for our proposed model, we use two alternatives to verify the validity of the feature. In one method, we convert the path non-zero feature to a random value between 0 and 1, while in the other, we convert all non-zero features to 1. Our proposed relation path feature has greatly improved, comparing to the other features. Table [3](#Tab3){ref-type="table"} shows the results, indicating that the PRA not only keeps the inference mode, but the relation path feature preserves the significant information about drug-disease. Table 3The PACNN model with different embedding featureMethodsPrecision(%)Recall(%)F-score(%)PACNN-random83.2182.6882.53PACNN-one86.4385.7985.56PACNN-pra91.5091.5091.46

Drug rediscovery {#Sec20}
----------------

To verify the ability of the model to discover new drugs for known disease, we selected 300 gold standard drug-disease from TTD, see Additional file [2](#MOESM2){ref-type="media"}, while there are 96 drug-disease directly connected in the KG, resulting in 204 cases. For a new disease, we randomly selected 100 candidate drugs as potential drugs, while also including standard drugs for treating diseases. Since the mechanism by which the drug acts on the disease is not clear, in order to ensure that the drug candidate can be linked to the disease under the corresponding target, we selected 3,564 targets from TTD as candidate targets. The validation criteria for drug discovery experiments are candidate drug score mean rank and hit\@10, indicating that the candidate drug score rank in top 10. In fact, the scores of candidate drugs are ranked in top, indicating that the candidate drug is closer to the real therapeutic drug. If a drug for treating the disease is not found, the corresponding drug score and mean rank are not considered in the total number.

NRWRH and TP-NRWRH are additional baseline methods, both of which are drug repositioning methods employing random walks with heterogeneous network. The difference is that TP-NRWRH uses two-pass random walks \[[@CR13], [@CR33]\]. For the drug-disease score and ranking, this work set the RW maximum steps size is 5, and the parameters of other baseline methods are set the recommended settings in their experiment. In Table [4](#Tab4){ref-type="table"} methods column, RW-2 represents random walks algorithm step is 2, if a drug reach to a disease in 1 step, it indicates that the drug has a therapeutic effect on the disease. The 'Not Found' column indicates that the current method cannot find the known drug number. Table [4](#Tab4){ref-type="table"} shows that, if the number of the walk steps exceeds 3, all drugs can be found by the RW. This means that all drugs and diseases are connected in at least 3 steps in the KG. On the other hand, 20 and 13 drugs are not found by the NRWRH and TP-NRWRH, respectively. Although the number of walking steps of these two methods is 3, due to they use restart random walks on the heterogeneous network, the drug can not reach the disease accurately. In Table [4](#Tab4){ref-type="table"}, the best result of 'Mean Ranking' column RW method is 55.26 by the RW-2. When the steps increase, the more candidate drug will be found, this will increase the mean ranking of the RW. NRWRH and TP-NRWRH outperform the RW, as the random walks are based on specific a heterogeneous network. In addition, in the column 'Hits\@10', the performance of NRWRH and TP-NRWRH is still better than RW. We see that, when the steps increase, the mean ranking and hit\@10 score approach the steady state. Finally, from Table [4](#Tab4){ref-type="table"} we see that PACNN shows the best performance on two tests, as the 'Mean Ranking' is 37.53 and 'Hit\@10' is 38.23%. Compared with random walk based methods, our method not only finds all candidate drugs but produces the best results. Additionally, we vary the settings of *η* to see how different percentage data affects the results. A set of scores are produced by a candidate drug and we would normally select the greatest among these values for the candidate drug. Due to the same scores are produced in 100 candidate drugs, while the number of intermediate candidate targets for each candidate drug and disease is uncertain. Thus, we chose the average of the different proportions of a set of scores as the final candidate drug score. We set *η* as top 5%, 10%, 20%, 50%, and 100% respectively. In addition, when the model prediction case is negative, we filter out data with the probability less than 0.5. In Fig. [5](#Fig5){ref-type="fig"}, we can see that when the data reached top 20%, the best results were achieved in both tests. Fig. 5The performance of different percentage data Table 4Drug rediscovery performanceMethodsNot foundMean rankingHit\@10(%)RW-23355.2617.54RW-3063.2811.76RW-4064.0410.78RW-5064.5710.78NRWRH2058.1421.19TP-NRWRH1341.5429.31Our method037.5338.23

Case study {#Sec21}
----------

To demonstrate the capabilities of our model, we show 12 samples that ranked in top 10. PACNN can predict candidate drug that is reported by TTD as capable of curing a disease. From Table [5](#Tab5){ref-type="table"} we can see that the drug 'Typherix' treats disease 'Salmonella infection', it ranked 1st as the candidate drug and TTD provides a research phase in the treatment of diseases at the column 'Drug Status'. Table 5Case study: rediscover known drugs for diseasesDrugDiseaseRankDrug statusTypherixSalmonella infection1ApprovedINS-1Metabolic disease2ApprovedTriamtereneEdema5ApprovedTriamtereneCongestive heart failure3ApprovedAnapsosAtopic dermatitis3ApprovedBrevenalCystic fibrosis2InvestigativeDiphencyproneAlopecia7Phase 2ECFCsCardiovascular disorder7InvestigativePneumovax 23Otitis media1ApprovedMesoglycanCerebrovascular disorders1ApprovedLASSBio-294Hypertension6InvestigativeSimethiconeDyspepsia9Approved

Discussion {#Sec22}
==========

According to the experimental results, our proposed model can effectively carry out the LBD task. This is the first attempt to employ PRA and attention mechanism for LBD. However, there are several limitations affecting our works. First, the data set used to train the model is small, and this will lead to weaker generalization. Thus, it would be useful to combine other drug-disease databases, such as Comparative Toxicogenomics Database (CTD) and Drugbank for addressing this limitation \[[@CR34], [@CR35]\]. In order to maintain the great connectivity of the KG, we have chosen all the predications as graph components. Since a predication may be erroneous, this will reduce the efficiency of our model. This limitation can be eliminated by improving the NLP technology. Another limitation is that PACNN needs to obtain all the relation paths between drugs and diseases. When the size of the knowledge base is large, it is difficult for our method to produce a more complex relation path. When the PRA is faced with a larger knowledge base, the computational efficiency will be greatly reduced, which must also be solved in future studies.

Conclusion {#Sec23}
==========

In this study, we presented a relation path features embedding based CNN with attention mechanism for discovering potential drugs from literature. Relation path feature embedding proved to be effective for capturing the association about drug-disease, thus we utilized PRA to get drug-disease relation path feature. Compared with other methods, the CNN based on attention mechanism can better identify the important relation feature of drug-disease, so that new drugs can be accurately discovered. Our method could be an auxiliary method for drug discovery, which can speed up the discovery of new drugs for the incurable diseases.

For the future work, we plan to explore an efficient path walk algorithm that is better adapted to large knowledge base. We are interested in applying our model to literature mining in other fields, such as economics. We will continue to explore the innovation and application of deep learning and machine learning on LBD tasks.

Additional files
================

 {#Sec24}

Additional file 1The 1714 drug-target-disease cases which are extracted from Therapeutic Target Database(TTD) as true cases for constructing training data. (TXT 87 kb)

Additional file 2The gold standard drug-disease cases extracted from TTD. There are 300 drug-disease case are selected from TTD as gold standard test data for drug rediscovery. (TXT 11 kb)

CNN

:   Convolutional neural network

LBD

:   Literature-based discovery

LR

:   Logistic regression

NLP

:   Natural language processing

RF

:   Random forest

SVM

:   Support vector machine

TTD

:   Therapeutic target database

Funding {#d29e3521}
=======

Publication costs are funded by the National Key Research Development Program of China (No. 2016YFB1001103), the National Natural Science Foundation of China (Nos. 61572098, 61572102).

Availability of data and materials {#d29e3526}
==================================

Some of the data generated in this study are contained in supplementary information files. The data can be used for further study.

About this supplement {#d29e3531}
=====================

This article has been published as part of *BMC Medical Informatics and Decision Making Volume 19 Supplement 2, 2019: Proceedings from the 4th China Health Information Processing Conference (CHIP 2018)*. The full contents of the supplement are available online at URL. <https://bmcmedinformdecismak.biomedcentral.com/articles/supplements/volume-19-supplement-2>.

DZ conceived, designed, performed the analyses, interpreted the results and wrote the manuscript. JW supervised the work and STS edited the manuscript. HFL,JBW and CMY revised this manuscript. All authors read and approved the final manuscript.

Ethics approval and consent to participate
==========================================

Not applicable.

Consent for publication
=======================

Not applicable.

Competing interests
===================

The authors declare that they have no competing interests.

Publisher's Note
================

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.