AUTOMATED PREDICTION OF CLINICAL TRIAL OUTCOME

A system for prediction of clinical trial outcome. The system includes: a processor of a trial prediction (TP) node connected to at least one cloud server node over a network configured to host a machine learning (ML) module; a memory on which are stored machine-readable instructions that when executed by the processor, cause the processor to: receive a clinical trial (CT) data, parse the CT data to derive drug molecules data, disease information data, and trial protocols data, encode the drug molecules data, the disease information data, and the trial protocols data into corresponding embeddings, generate knowledge pre-trained embeddings using external knowledge data, and provide the knowledge pre-trained embeddings to the ML module for prediction of the CT outcome.

FIELD OF DISCLOSURE

The present disclosure generally relates to clinical trials, particularly to an intelligent AI-based automated system for the prediction of a clinical trial outcome.

BACKGROUND

Clinical trials are research studies involving trial participants to evaluate the safety and efficacy of medical devices and drugs that have been newly developed to treat diseases and health conditions. Clinical trials are typically conducted after the medical device or drug has been tested on animals. Clinical trials typically produce the evidence upon which governmental regulatory agencies rely when approving a medical device or drug for human use.

Clinical trials follow strict scientific standards in order to produce reliable results. Clinical trials are crucial for drug development but are time-consuming, expensive, and often burdensome on patients. More importantly, clinical trials face uncertain outcomes due to issues with efficacy, safety, or problems with patient recruitment. Clinical trials are an indispensable step towards developing a new drug, where human participants are tested in responding to a treatment (e.g., a drug or drug combinations) for treating target diseases. The costs of conducting clinical trials are extremely expensive (up to hundreds of millions of dollars) and the time required to run a clinical trial is very long with low success probability. However, many factors such as the inefficacy of the drug, drug safety issues, and poor trial protocol design can cause the failure of a clinical trial. If a reliable prediction of a clinical trial success probability could be made, the trials that would inevitably fail with their current design can be modified or redesigned prior to implementation of the clinical trial.

Recently limited attempts of trying to predict individual components in clinical trials have been made. Specifically, leverage gradient-boosted decision trees have been used to predict the effect of antidepressant treatment in improving depressive symptoms based on electroencephalographic (EEG) measures. Some conventional solutions use an ensemble classifier based on weighted least squares support vector regression to predict the clinical trial outcome based on drug-property and target-property features. Also, the recurrent neural network has been used to predict the phase-3 trial results based on phase-2 results in terms of longitudinal data. Some solutions expand beyond optimizing individual components to predict trial outcomes for 15 disease groups based on disease-only features using linear models, including random forest and logistic regression.

Despite these initial efforts, several following limitations impede the utility of existing trial outcome prediction models. The first limitation of the conventional solutions is a limited task scope. Existing solutions either focus on predicting individual components of trials such as patient-trial matching or only covering disease groups of which the disease-specific features are available. Although these solutions are potentially helpful for a limited part of the trial design, they do not answer the fundamental problem—will this trial be approved, or what probability of the trial approval may be expected. The second limitation of the conventional solutions is the limited features used for prediction. The existing solutions only leverage restricted disease-specific features, which cannot be generalized to other diseases. These solutions largely ignore the multi-faceted risks, including drug safety, treatment efficiency, and trial recruitment, where abundant information is publicly available for assessing those different risks. For example, the biomedical knowledge base provides explicit biochemical structures of the drug molecules, and previous trial history for similar diseases can be useful for trial outcome prediction.

Yet another limitation of the existing solutions is handling complex relations among trial components and outcomes. The existing works are often over-simplified, which significantly impedes them from modeling the complicated relations among various trial components.

Accordingly, a system and method for an intelligent AI-based automated prediction of a clinical trial outcome is desired.

BRIEF OVERVIEW

This brief overview is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This brief overview is not intended to identify key features or essential features of the claimed subject matter. Nor is this brief overview intended to be used to limit the claimed subject matter's scope.

One embodiment of the present disclosure provides a system for automated prediction of a clinical trial outcome. The system includes: a processor of a trial prediction (TP) node connected to at least one cloud server node over a network configured to host a machine learning (ML) module; a memory on which are stored machine-readable instructions that when executed by the processor, cause the processor to: receive a clinical trial (CT) data, parse the CT data to derive drug molecules data, disease information data, and trial protocols data, encode the drug molecules data, the disease information data, and the trial protocols data into corresponding embeddings, generate knowledge pre-trained embeddings using external knowledge data, and provide the knowledge pre-trained embeddings to the ML module for prediction of the CT outcome.

Another embodiment of the present disclosure provides a method that includes one or more of: receiving a clinical trial (CT) data; parsing the CT data to derive drug molecules data, disease information data, and trial protocols data; encoding the drug molecules data, the disease information data, and the trial protocols data into corresponding embeddings; generating knowledge pre-trained embeddings using external knowledge data; and providing the knowledge pre-trained embeddings to an ML module for prediction of the CT outcome.

Another embodiment of the present disclosure provides a computer-readable medium including instructions for receiving a clinical trial (CT) data; parsing the CT data to derive drug molecules data, disease information data, and trial protocols data; encoding the drug molecules data, the disease information data, and the trial protocols data into corresponding embeddings; generating knowledge pre-trained embeddings using external knowledge data; and providing the knowledge pre-trained embeddings to an ML module for prediction of the CT outcome.

DETAILED DESCRIPTION

The present disclosure includes many aspects and features. Moreover, while many aspects and features relate to, and are described in, the context of processing job applicants, embodiments of the present disclosure are not limited to use only in this context.

The present disclosure provides a system, method and computer-readable medium for an intelligent AI-based automated prediction of a clinical trial outcome.

Historical clinical trial data, and massive knowledge base about approved and failed drugs may bring a new opportunity for using machine learning AI-based models to tackle the key question—can a clinical trial success probability be accurately predicted. The disclosed embodiments may address these challenges by formulating an all-cause and an all-phase clinical trial outcome prediction task, which has practical utility for a clinical trial design. In one embodiment, a Hierarchical Interaction NeTwork (HINT) that explicitly simulates each clinical trial component and the complicated relations among them is provided. The HINT uses a graph neural network-based approach to model the interplay among various trial risks, including drug safety, treatment efficiency, and trial recruitment. It also covers a wide range of drugs and indications (e.g., diseases to be treated). A disclosed model framework can generalize over new trials given the easily accessible drug, disease and protocol information.

According to the disclosed embodiments, problem formulation may be implemented as follows. A clinical trial aims to validate the safety and efficacy of a treatment set towards a target disease set on a patient group.

Definition 1 (Treatment Set). In a trial, the treatment set includes one or multiple drug candidates, denoted by M={m1, . . . , mNm}. The system of the disclosed embodiment is restricted to trials that aim at discovering new indications of drug candidates. Other trials that involve surgeries or devices are not considered. Drug molecule is used.

Definition 2 (Target Disease Set). Each trial targets one or more diseases, denoted D={d1, . . . , dNd}, direpresents the disease code of the i-th disease. An ICD-10 code is used based on the 10th revision of the International Statistical Classification of Diseases402.

Definition 3 (Trial Protocol). Each clinical trial protocol consists of inclusion and exclusion criteria in a text format, describing the characteristics of desired patients for the trial, such as age, gender, medical history, target disease conditions, and current health status. It is denoted as C=[c1, . . . , cNp].

Problem 1 (Clinical Trial Outcome Prediction). The trial outcome is a binary label y, y=1 indicates trial success (i.e., the trial received approval status) and 0 indicates failure (i.e., all the other status). In this trial outcome prediction task, the system trains a deep neural network y=fθ(M,D,C) to predict the outcome, based the model parameters θ.

In one embodiment, an input embedding module encodes multi-modal data, including drug molecules, disease information, and trial protocols to embeddings. Next, these embeddings are fed into a knowledge embedding module to generate knowledge embeddings pre-trained using external knowledge including drug pharmaco-kinetics and disease risk data. Then, the interaction graph module connects all the embeddings via domain knowledge to fully capture various trial components and their complex relations as well as their influences on trial outcomes. Based on that, the HINT learns (i.e., trains) dynamic attentive graph neural network to predict trial outcome.

An input embedding module may learn embedding from drug molecules; disease information; trial protocols.

The drug molecules are represented by molecule graphs. The disclosed embodiments may use message passing network (MPN) to encode molecular graphs and average over embeddings of all the molecules in the treatment set as the treatment embedding:

Disease information also affects the outcome. For example, drugs in oncology have much lower approval rates than ones in infectious diseases. The disease information comes from its description and its corresponding ontology, such as disease hierarchies like International Classification of Diseases (ICD). D={d1, . . . , dNd}(Definition 2), the diseases in the trial are represented as:

where GRAM (di) represents embedding of diusing GRAM (graph-based attention model), which leverages the hierarchical information inherent to medical ontologies.

The trial protocol describes eligibility criteria, i.e., patient recruitment requirements. Each inclusion or exclusion criteria corresponds to a sentence. Clinical-BERT may be applied to get the sentence representation and average over all the sentences:

One special challenge associated with clinical trial data is that there can be missing data on molecular information due to proprietary information. This poses a problem since representation of many nodes depends on the molecular information. It has been observed that there exists high correlation between the drug molecule, disease and protocol features.

One special challenge associated with clinical trial data is that there can be missing data on molecular information due to proprietary information. This poses a problem since representation of many nodes depends on the molecular information. It has been observed that there exists high correlation between the drug molecule, disease and protocol features. Thus, the disclosed embodiments provide a missing data imputation module based on learning embeddings that capture inter-modal correlations and intra-modal distribution. In particular, the imputation module IMP(⋅) uses disease and protocol embedding (hd, hp) to recover molecular embedding hm:

Here, the system adopts MSE (Mean Square Error) loss as the learning objective to minimize the distance between the ground truth molecule embedding and predicted one.

In one embodiment, the HINT utilizes external knowledge to pretrain knowledge nodes and further enhance those input embeddings. In case of pharmaco-kinetics knowledge, the embeddings are pre-trained using the pharmaco-kinetics (PK) knowledge about how the body reacts to the intake of a drug. The success of a clinical trial highly depends on factors including drugs' pharmaco-kinetics properties and disease risk. Specifically, the HINT system may leverage various public PK experimental scores. Based on this information, the system may pre-train on prediction models for Absorption, Distribution, Metabolism, Excretion, Toxicity (ADMET) properties, which are used to observe how a drug interacts with the body. Absorption describes how drugs travels from administration site to the site of action. Distribution measures the drug's ability to move through the bloodstream. Metabolism measures the duration of a drug's efficacy. Excretion measures how much of drug's toxic components can be removed from the body. Toxicity model measures damage to the body. For each pharmaco-kinetics property, binary labels may be used to indicate whether the drug is desirable in this property. Concretely, for each *∈{A,D,M,E,T}, the system pre-trains a knowledge node based on the label y*∈{0,1} as follows:

where X*is a two-layer highway network. FC is a fully-connected layer. Binary cross-entropy is a loss criteria. After training, the drug in each trial can be fed to the pre-trained model to obtain absorption/distribution/metabolism/excretion/toxicity embeddings.

With respect to a disease risk, the disclosed embodiments consider the knowledge distilled from historical trials of the target diseases. The historical trial success rate may be used for the disease. As detailed statistics for trial success rate of each disease at different trial phases are widely available, the system considers that as the supervision signal to train the disease risk prediction model:

where hdis the disease embedding defined above, hRis disease risk embedding, yR∈{0,1} is the risk label. Cross-entropy loss is minimized. After training, the disease information in the trial is fed to the pretrained model to obtain disease risk embedding.

In one embodiment, a hierarchical interaction graph G may be constructed to connect input data sources and important factors affecting clinical trial outcomes. The graph reflects the real-world trial development process and consists of four tiers of nodes that are connected between tiers:

Input nodes include drugs, diseases and protocols with node features of input embedding hm,hd,hp∈Rd.

External knowledge nodes include ADMET embeddings hA,hD,hM,hE,hT∈Rdand disease risk embedding hR∈Rd. These representations are pretrained on external knowledge.

(c) augmented interaction node hVthat augments interaction node hIusing disease risk node hR.

According to the disclosed embodiments, the aggregation nodes are implemented as follows. The PK (Pharmaco-Kinetics) node is to gather information about ADMET properties. PK (Pharmaco-Kinetics) embedding is obtained by:

where fK(⋅) a one-layer fully-connected layer followed by a two-layer highway network.

Then, the system models the interaction among drug molecule, disease and protocol, the interaction node embedding is:

fI(⋅) has the same architecture as fK(⋅).

The Augmented interaction node combines (i) trial risk of the target disease hRand (ii) the interaction node hI:

where fVuses the same architecture as fK(⋅), fI(⋅).

The Prediction node summarizes pharmaco-kinetics and augmented interaction

where fpuses the same architecture as fI, fv.

The Dynamic Attentive Graph Neural Network is implemented as follows. The trial embeddings provide initial representations of different trial components and their interactions via a graph. To further enhance interaction, a dynamic attentive graph neural network is designed. Mathematically, in interaction graph G, nodes are trial components, edges are the relations among them.

A∈{0,1}′ denotes the adjacency matrix of G, K=13 is the node number. Node embeddings H(0)are initialized by stacking representation of all the components:

The disclosed embodiment, advantageously, enhances the interaction between nodes using graph convolutional network (GCN). The updating rule for the l-th layer is:

where L is GCN depth. In the l-th layer, H(l)∈RK×dis node embeddings, B(l), W(l)are the bias/weight parameter, ⊙ represents element-wise multiplication. Different from conventional GCN, the system employs a dynamic layer-independent attentive matrix V E R+K×K, Vi,jwhich measures the importance of the edge between i-th and j-th nodes. It is parameterized by a two-layer fully-connected neural network with ReLU and Sigmoid activation function in the hidden and output layer:

V is the reasoning matrix where each entry corresponds to the reasoning score among each trial component. It is then element-wisely multiplied to adjacency matrix A so that it assigns weights to edges in interaction graph G.

The prediction and training. After GNN message-passing, updated representations for trial components are obtained. To generate the trial success prediction9, the last-layer (L-th) representation of the prediction node is fed into a one-layer fully-connected network with sigmoid function, and leverage binary cross-entropy loss to guide training:

where y∈{0,1} is the ground truth. The HINT is trained in an end-to-end manner.

FIG.1illustrates a network diagram of a system for an intelligent AI-based automated determination of clinical trial outcome consistent with the present disclosure.

Referring toFIG.1, the example network100includes the trial prediction (TP) server node102connected to a cloud server node(s)105and servers113over a network. The cloud server node(s)105is configured to host an AI/ML module107. The TP server node102may receive clinical trial parameters (e.g., drug molecules, disease information, and trial protocols data) from a user device111that may be a smartphone, a tablet, a laptop/PC, etc. The TP server node102may acquire pharmaco-kinetics data from the servers113. As discussed above, the TP server node102may query available pharmacological databases (not shown) based on the clinical trial parameters and may store the drug molecules data, pharmaco-kinetics data and disease information data in a local database103. The TP server102may predict a trial outcome by ingesting feature vectors data derived from the drug molecules data, disease information, and trial protocols data into an AI/ML module107.

The AI/ML module107may generate a predictive model(s)108to predict patient having the disease the drug based on data collected and stored in the local database103. Once the trial outcome is positively predicted a list of the potential participants is established based on the trial protocols data, these participants may be contacted and invited to the clinical trial.

FIG.2illustrates a network diagram of a system including detailed features of a trial prediction (TP) server node102consistent with the present disclosure.

Referring toFIG.2, the example network200includes the TP server node102connected to a cloud server node(s)105over a network. The cloud server node(s)105is configured to host an AI/ML module107. As discussed above with reference toFIG.1, the TP server node102may receive clinical trial data201including drug molecules, disease information, and trial protocols data.

The AI/ML module107may generate a predictive model(s)108based on historical trial data provided by the TP server102from a local data storage. As discussed above, the AI/ML module107may provide predictive outputs data that indicate a trial outcome and a probability of success based on the trial parameters and historical data acquired form the database103. Note that the AI/ML module107may be implemented on the TP server node102. The TP server node102may process the predictive outputs data received from the AI/ML module107to generate a trial outcome report.

While this example describes in detail only one TP server node102, multiple such nodes may be connected to the network. It should be understood that the TP server node102may include additional components and that some of the components described herein may be removed and/or modified without departing from a scope of the TP server node102disclosed herein. The TP server node102may be a computing device or a server computer, or the like, and may include a processor204, which may be a semiconductor-based microprocessor, a central processing unit (CPU), graphics processing unit (GPU), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), and/or another hardware device. Although a single processor204is depicted, it should be understood that the TP server node102may include multiple processors, multiple cores, or the like, without departing from the scope of the TP server node102system.

The TP server node102may also include a non-transitory computer readable medium212that may have stored thereon machine-readable instructions executable by the processor205. Examples of the machine-readable instructions are shown as214-222and are further discussed below. Examples of the non-transitory computer readable medium212may include an electronic, magnetic, optical, or other physical storage device that contains or stores executable instructions. For example, the non-transitory computer readable medium212may be a Random-Access memory (RAM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), a hard disk, a solid-state drive, an optical disc, or other type of storage device.

The processor204may fetch, decode, and execute the machine-readable instructions214to receive a clinical trial (CT) data. The processor204may fetch, decode, and execute the machine-readable instructions216to parse the CT data to derive drug molecules data, disease information data, and trial protocols data. The processor204may fetch, decode, and execute the machine-readable instructions218to encode the drug molecules data, the disease information data, and the trial protocols data into corresponding embeddings. The processor204may fetch, decode, and execute the machine-readable instructions220to generate knowledge pre-trained embeddings using external knowledge data. The processor204may fetch, decode, and execute the machine-readable instructions222to provide the knowledge pre-trained embeddings to the ML module for prediction of the CT outcome.

FIG.3Aillustrates a flowchart of a method for an intelligent AI-based prediction of a clinical trial outcome consistent with the present disclosure.

Referring toFIG.3A, the method300may include one or more of the steps described below.FIG.3Aillustrates a flow chart of an example method executed by the TP server102(seeFIG.2). It should be understood that method300depicted inFIG.3Amay include additional operations and that some of the operations described therein may be removed and/or modified without departing from the scope of the method300. The description of the method300is also made with reference to the features depicted inFIG.2for purposes of illustration. Particularly, the processor204of the TP server102may execute some or all of the operations included in the method300.

With reference toFIG.3A, at block302, the processor204may receive a clinical trial (CT) data. At block304, the processor204may parse the CT data to derive drug molecules data, disease information data, and trial protocols data. At block306, the processor204may encode the drug molecules data, the disease information data, and the trial protocols data into corresponding embeddings. At block308, the processor204may generate knowledge pre-trained embeddings using external knowledge data. At block310, the processor204may provide the knowledge pre-trained embeddings to the ML module107for prediction of the CT outcome. As discussed above, the ML module107may generated the predictive models108.

FIG.3Billustrates a further flowchart of a method for an automated AI-based prediction of a clinical trial outcome consistent with the present disclosure. Referring toFIG.3B, the method310may include one or more of the steps described below.FIG.3Billustrates a flow chart of an example method executed by the TP server102(seeFIG.2). It should be understood that method310depicted inFIG.3Bmay include additional operations and that some of the operations described therein may be removed and/or modified without departing from the scope of the method310. The description of the method310is also made with reference to the features depicted inFIG.2for purposes of illustration. Particularly, the processor204of the TP server102may execute some or all of the operations included in the method300.

With reference toFIG.3B, at block314, the processor204may query a database103(seeFIG.2) for drug pharmaco-kinetics data and disease risk data. At block316, the processor204may generate the knowledge pre-trained embeddings based on the drug pharmaco-kinetics data and the disease risk data. At block318, the processor204may generate a disease risk embedding to pre-train prediction models for the disease risk. At block320, the processor204may train a dynamic attentive graph neural network to predict the CT outcome. At block322, the processor204may train a deep neural network y=fθ(M,D,C) to predict the CT outcome based on model parameters θ. At block324, the processor204may use a message passing network to encode molecular graphs representing the drug molecules and to average over embeddings of multiple drugs molecules. At block326, the processor204may pre-train prediction models for absorption, distribution, metabolism, excretion, and toxicity based on the drug molecules data.

In one disclosed embodiment, the clinical trial outcome prediction model may be generated by an AI/ML module107that may use training data sets to improve accuracy of the prediction of trial success/failure. The parameters used in training data sets may be stored in a centralized database103(FIG.1A). In one embodiment, a neural network may be used for trial participation modeling and prediction.

FIG.4illustrates a diagram of a clinical trial outcome prediction process consistent with the present disclosure.

According to the disclosed embodiments, problem formulation may be implemented as follows. A clinical trial aims to validate the safety and efficacy of a treatment set towards a target disease set on a patient group.

Definition 1 (Treatment Set). In a trial, the treatment set includes one or multiple drug candidates, denoted by M={m1, . . . , mNm}. The system ofFIG.1restricts to trials that aim at discovering new indications of drug candidates. Other trials that involve surgeries or devices are not considered. Drug molecule401is used.

Definition 2 (Target Disease Set). Each trial targets one or more diseases402, denoted D={d1, . . . , dNd}, direpresents the disease code of the i-th disease. An ICD-10 code is used based on the 10th revision of the International Statistical Classification of Diseases402.

Definition 3 (Trial Protocol). Each clinical trial protocol403consists of inclusion and exclusion criteria in a text format, describing the characteristics of desired patients for the trial, such as age, gender, medical history, target disease conditions, and current health status. It is denoted as C=[c1, . . . , cNp].

Problem 1 (Clinical Trial Outcome Prediction). The trial outcome is a binary label y, y=1 indicates trial success (i.e., the trial received approval status) and 0 indicates failure (i.e., all the other status). In this trial outcome prediction task, the system trains a deep neural network y=fθ(M,D,C) to predict the outcome based on the model parameters θ.

As discussed above, an input embedding module encodes multi-modal data, including drug molecules, disease information, and trial protocols to embeddings. Next, these embeddings are fed into a knowledge embedding module to generate knowledge embeddings pre-trained using external knowledge including drug pharmaco-kinetics and disease risk data. Then, the interaction graph module connects all the embeddings via domain knowledge to fully capture various trial components and their complex relations as well as their influences on trial outcomes. Based on that, the HINT learns (i.e., trains) dynamic attentive graph neural network to predict trial outcome.

An input embedding module may learn embedding from drug molecules; disease information; trial protocols.

The drug molecules are represented by molecule graphs. The disclosed embodiments may use message passing network (MPN)404to encode molecular graphs and average over embeddings of all the molecules in the treatment set as the treatment embedding:

Disease information also affects the outcome. For example, drugs in oncology have much lower approval rates than ones in infectious diseases. The disease information comes from its description and its corresponding ontology, such as disease hierarchies like International Classification of Diseases (ICD). D={d1, . . . , dNd}(Definition 2), the diseases402in the trial are represented as:

where GRAM(di) represents embedding of diusing GRAM (graph-based attention model405), which leverages the hierarchical information inherent to medical ontologies.

The trial protocol describes eligibility criteria, i.e., patient recruitment requirements. Each inclusion or exclusion criteria corresponds to a sentence. Clinical-BERT406may be applied to get the sentence representation and average over all the sentences:

One special challenge associated with clinical trial data is that there can be missing data on molecular information due to proprietary information. This poses a problem since representation of many nodes depends on the molecular information. It has been observed that there exists high correlation between the drug molecule, disease and protocol features. Corresponding embeddings414,415and416are generated.

FIG.5illustrates a diagram of drug embedding for clinical trial outcome prediction consistent with the present disclosure.

The disclosed embodiments provide a missing data to an imputation neural network module507based on learning embeddings that capture inter-modal correlations and intra-modal distribution.

If a trial has a molecular structure at block501, a message passing neural network502is used to generate drug embedding503. Otherwise, trial embedding504and disease embedding505are fed to the imputation neural network module507to generate the drug embedding509.

In particular, the imputation module IMP(⋅) uses disease and protocol embedding (hd,hp) to recover molecular embedding hm:

Here, the system adopts MSE (Mean Square Error) loss as the learning objective to minimize the distance between the ground truth molecule embedding and predicted molecule embedding.

FIG.6illustrates a further diagram of drug embedding consistent with the present disclosure.

In case of pharmaco-kinetics knowledge, the embeddings are pre-trained using the pharmaco-kinetics (PK) knowledge about how the body reacts to the intake of a drug. The success of a clinical trial highly depends on factors including drugs' pharmaco-kinetics properties and disease risk. Specifically, the HINT system may leverage various public PK experimental scores. Based on this information, the system may pre-train on prediction models for Absorption, Distribution, Metabolism, Excretion, Toxicity (ADMET) properties608, which are used to observe how a drug interacts with the body. Absorption describes how drugs travels from administration site to the site of action. Distribution measures the drug's ability to move through the bloodstream. Metabolism measures the duration of a drug's efficacy. Excretion measures how much of drug's toxic components can be removed from the body. Toxicity model measures damage to the body. For each pharmaco-kinetics property608, binary labels may be used to indicate whether the drug is desirable in this property. Concretely, for each *∈{A,D,M,E,T}, the system pre-trains a knowledge node based on the label y*∈{0,1} as follows:

where X is a two-layer highway network609. FC is a fully-connected layer. Binary cross-entropy is a loss criteria. After training, the drug in each trial can be fed to the pre-trained model to obtain absorption/distribution/metabolism/excretion/toxicity embeddings610.

FIG.7illustrates a diagram of disease embedding for clinical trial outcome prediction consistent with the present disclosure.

With respect to a disease risk, the disclosed embodiments consider the knowledge distilled from historical trials of the target diseases. The historical trial success rate411may be used for the disease. As detailed statistics for trial success rate of each disease at different trial phases are widely available, the system considers that as the supervision signal to train the disease risk prediction model based on multi-layer highway neural network712:

where hdis the disease embedding714defined above, hRis disease risk embedding, yR∈{0,1} is the risk label. Cross-entropy loss is minimized. After training, the disease information in the trial is fed to the pretrained model712to obtain disease risk embedding715.

FIG.8illustrates a diagram of connection of different nodes used for node embedding for the trial outcome prediction interaction graph consistent with the present disclosure.

In one embodiment, a hierarchical interaction graph G may be constructed to connect input data sources and important factors affecting clinical trial outcomes. The graph818reflects the real-world trial development process and consists of four tiers of nodes that are connected between tiers:

Input nodes include drugs, diseases and protocols814with node features of input embedding hm,hd,hp∈Rd;

External knowledge nodes include ADMET embeddings815hA,hD,hm,hE,hT∈Rdand disease risk embedding hR∈Rd. These representations are pretrained on external knowledge.

(c) augmented interaction node hVthat augments interaction node hIusing disease risk node hR.

According to the disclosed embodiments, the aggregation nodes are implemented as follows. The PK (Pharmaco-Kinetics) node816is to gather information about ADMET properties. PK (Pharmaco-Kinetics) embedding is obtained by:

where fK(⋅) a one-layer fully-connected layer followed by a two-layer highway network.

Then, the system models the interaction among drug molecule, disease and protocol, the interaction node embedding is:

fI(⋅) has the same architecture as fK(⋅).

The Augmented interaction node combines (i) trial risk of the target disease hRand (ii) the interaction node hI:

where fVuses the same architecture as fK(⋅), fI(⋅).

The Prediction node summarizes pharmaco-kinetics and augmented interaction:

where fpuses the same architecture as fI, fV.

The Dynamic Attentive Graph Neural Network is implemented as follows. The trial embeddings provide initial representations of different trial components and their interactions via a graph. To further enhance interaction, a dynamic attentive graph neural network is designed. Mathematically, in interaction graph G, nodes are trial components, edges are the relations among them.

A∈{0,1}K×Kdenotes the adjacency matrix818based on trial outcome817. The matrix818of G, K=13 is the node number. Node embeddings H(0)are initialized by stacking representation of all the components:

FIG.9illustrates a diagram for generation of a reasoning matrix associated with predicted trial outcome consistent with the present disclosure.

As discussed above, different from conventional GCN, the system employs a dynamic layer-independent attentive matrix V∈R+K×K, Vi,jwhich measures the importance of the edge between i-th and j-th nodes819. It is parameterized by a two-layer fully-connected neural network820with ReLU and Sigmoid activation function in the hidden and output layer:

V is the reasoning matrix821where each entry corresponds to the reasoning score among each trial component. It is then element-wisely multiplied to adjacency matrix A so that it assigns weights to edges in interaction graph G.

FIG.10illustrates a diagram for determining a trial outcome probability based on trial embedding consistent with the present disclosure;

The disclosed embodiment enhances the interaction between nodes using graph convolutional network (GCN)1022. The updating rule for the l-th layer is:

where L is GCN depth. In the l-th layer, H(l)∈RK×dis node embeddings, B(l), W(l)are the bias/weight parameter, ⊙ represents element-wise multiplication. Different from conventional GCN, the system employs a dynamic layer-independent attentive matrix V∈R+K×K, Vi,jwhich measures the importance of the edge between i-th and j-th nodes919(seeFIG.6).

The prediction and training are implemented as follows. After GNN message-passing, updated representations for trial components are obtained. To generate the trial success (probability) prediction1023ŷ, the last-layer (L-th) representation of the prediction node is fed into a one-layer fully-connected network with sigmoid function, and leverage binary cross-entropy loss to guide training.

where y∈{0,1} is the ground truth. The HINT is trained in an end-to-end manner.

The trial embedding1022is based on matrixes1018-1021.

According to the disclosed embodiments, Data Preparation is implemented as follows. As there is no public trial outcome prediction dataset available, data may be collected from several public data sources. For each trial registered in clinicaltrials.gov, the following is collected:

1) molecule information for tested drugs, available at DrugBank (www.drugbank.com);

2) ICD-10 codes, public API (clinicaltables.nlm.nih.gov) can convert the disease description into ICD-10 code;

4) trial outcome, a binary indicator of trial success (1) or failure (0), available at clinicaltrials.gov.

Trials that have statistical analysis results in terms of p-value for the primary outcome measure may be selected. The trial is labelled as success if p-value is less than 0.05 and negative if p-value is higher than 0.05. It is validated by an internal dataset.

Additionally, auxiliary the following data is provided:

Pharmaco-kinetics data, which consists of wet lab experiment results for five important PK tasks and available at moleculenet.ai;

Disease risk data, including the past disease trial history success rate, also available at clinicaltrials.gov.

Experimental setting may be implemented as follows. Evaluation settings may consider two realistic setups:

(1) Phase-level evaluation predicts the outcome of single-phase study. Since each phase has different goals (e.g., Phase I is for safety whereas Phase II and III are for efficacy), experiments for Phase I, II and III may be conducted individually. The test datasets are created using the FDA guideline on the success-failure ratio for each phase, specifically 70% success rate for Phase I, 33% for Phase II and 30% for Phase III.

(2) Indication-level evaluation predicts if the drug can pass all three phases for market approval. To imitate it, all phase studies assembled related to the drug and disease of the study and then use the latest phase protocol as the input to our model. Drugs that have Phase III success are labelled positive and other drugs that fail in any of the three phases are labelled negative. Data statistics are shown in Table 1 (see appendix ii).

Data split is implemented as follows. The dataset is split based on a registration date. The earlier trials are used for learning while the later trials are used for inference. For example, for Phase I dataset, the model is trained using the trials before Aug. 13, 2014 and inference is made using the trials after that date, as shown in Table 1 (see appendix ii).

Metrics are implemented as follows: apply (i) PR-AUC (Precision-Recall Area Under Curve); (ii) F1 score; (iii) ROC-AUC (Area Under the Receiver Operating Characteristic Curve) to measure the prediction accuracy. The hypothesis testing results measured by p-value are reported to showcase the statistical significance of our method over the best baseline method.

Baselines—HINT is compared with several baselines, including both conventional machine learning models and deep learning methods, including:

Exp 1. Phase-level prediction focuses on predicting the outcome for a particular phase. In general, phase I tests the toxicity and side effects of the drug; phase II explores the efficacy of the drug (i.e., if the drug works); phase III focuses on drug effectiveness (whether the drug is better than the standard practice/placebo for a well-defined population). For each phase, we train a separate model.

From the results in Table 2 (see appendix ii), it is easy to see deep learning-based approaches including FFNN, DeepEnroll, COMPOSE and HINT outperforms conventional machine learning approaches (LR, RF, XGBoost, AdaBoost) significantly in outcome prediction for all the three phases, thus validating the benefit of deep learning methods for clinical trial outcome prediction. Among all the deep learning methods, HINT performs best with 0.837 F1 for phase I, 0.638 for phase II and 0.683 for phase III. Compared with the strongest baseline (COMPOSE), which is also deep learning approach that uses all the features, HINT achieved 12.2%, 5.6%, 5.6% relatively improvement in terms of PR-AUC and 9.0%, 4.2%, 4.6% relative improvement in terms of F1 score.

The reason is that HINT incorporates insightful multimodal data embedding and finer-grained interaction between multimodal data and trial components (i.e., node in interaction graph). The full HINT performs better than the variant without pre-training model (HINT-Pretrain) and the one without GNN model (HINT-GNN) in both phase I and II scenarios. This observation confirmed the importance of all modules in HINT. When comparing the prediction performance across phase I-Ill, it is found that phase I achieves highest accuracy for almost all the methods while phase II is most challenging with lowest accuracy. This result is consistent with historical trials statistics and reported accuracy of machine learning models on these tasks.

Exp 2. Indication-level prediction focuses on whether the indication of a drug will be approved (pass all 3 phases). A separate model is built using a combined dataset where the successful trials are the ones passed phase III (plus the ones reached phase IV) while the failed trials are the ones that failed in any phase from I to III. The results are presented in Table 3 (see appendix ii). Similar trends have been observed. In particular, HINT performs the best with 0.670 PRAUC, 0.762 F1 and 0.783 ROC-AUC which achieves 6.5%, 5.8%, 7.0% relative improvements on PR-AUC, F1 and ROC-AUC over the strongest baseline (COMPOSE).

The above embodiments of the present disclosure may be implemented in hardware, in a computer-readable instructions executed by a processor, in firmware, or in a combination of the above. The computer computer-readable instructions may be embodied on a computer-readable medium, such as a storage medium. For example, the computer computer-readable instructions may reside in random access memory (“RAM”), flash memory, read-only memory (“ROM”), erasable programmable read-only memory (“EPROM”), electrically erasable programmable read-only memory (“EEPROM”), registers, hard disk, a removable disk, a compact disk read-only memory (“CD-ROM”), or any other form of storage medium known in the art.

An exemplary storage medium may be coupled to the processor such that the processor may read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an application specific integrated circuit (“ASIC”). In the alternative embodiment, the processor and the storage medium may reside as discrete components. For example,FIG.11illustrates an example computing device (e.g., a server node)500, which may represent or be integrated in any of the above-described components, etc.

FIG.11illustrates a block diagram of a system including computing device500. The computing device500may comprise, but not be limited to the following:

Mobile computing device, such as, but is not limited to, a laptop, a tablet, a smartphone, a drone, a wearable, an embedded device, a handheld device, an Arduino, an industrial device, or a remotely operable recording device;

A supercomputer, an exa-scale supercomputer, a mainframe, or a quantum computer;

A minicomputer, wherein the minicomputer computing device comprises, but is not limited to, an IBM AS500/iSeries/System I, A DEC VAX/PDP, a HP3000, a Honeywell-Bull DPS, a Texas Instruments TI-990, or a Wang Laboratories VS Series;

A microcomputer, wherein the microcomputer computing device comprises, but is not limited to, a server, wherein a server may be rack mounted, a workstation, an industrial device, a raspberry pi, a desktop, or an embedded device;

The trial prediction (TP) server node102(seeFIG.2) may be hosted on a centralized server or on a cloud computing service. Although method300has been described to be performed by the TP server node102implemented on a computing device500, it should be understood that, in some embodiments, different operations may be performed by a plurality of the computing devices500in operative communication at least one network.

Embodiments of the present disclosure may comprise a computing device having a central processing unit (CPU)520, a bus530, a memory unit550, a power supply unit (PSU)550, and one or more Input/Output (I/O) units. The CPU520coupled to the memory unit550and the plurality of I/O units560via the bus530, all of which are powered by the PSU550. It should be understood that, in some embodiments, each disclosed unit may actually be a plurality of such units for the purposes of redundancy, high availability, and/or performance. The combination of the presently disclosed units is configured to perform the stages any method disclosed herein.

Consistent with an embodiment of the disclosure, the aforementioned CPU520, the bus530, the memory unit550, a PSU550, and the plurality of I/O units560may be implemented in a computing device, such as computing device500. Any suitable combination of hardware, software, or firmware may be used to implement the aforementioned units. For example, the CPU520, the bus530, and the memory unit550may be implemented with computing device500or any of other computing devices500, in combination with computing device500. The aforementioned system, device, and components are examples and other systems, devices, and components may comprise the aforementioned CPU520, the bus530, the memory unit550, consistent with embodiments of the disclosure.

At least one computing device500may be embodied as any of the computing elements illustrated in all of the attached figures, including the design server node102(FIG.2). A computing device500does not need to be electronic, nor even have a CPU520, nor bus530, nor memory unit550. The definition of the computing device500to a person having ordinary skill in the art is “A device that computes, especially a programmable [usually] electronic machine that performs high-speed mathematical or logical operations or that assembles, stores, correlates, or otherwise processes information.” Any device which processes information qualifies as a computing device500, especially if the processing is purposeful.

With reference toFIG.11, a system consistent with an embodiment of the disclosure may include a computing device, such as computing device500. In a basic configuration, computing device500may include at least one clock module110, at least one CPU520, at least one bus530, and at least one memory unit550, at least one PSU550, and at least one I/O560module, wherein I/O module may be comprised of, but not limited to a non-volatile storage sub-module561, a communication sub-module562, a sensors sub-module563, and a peripherals sub-module565.

A system consistent with an embodiment of the disclosure the computing device500may include the clock module510may be known to a person having ordinary skill in the art as a clock generator, which produces clock signals. Clock signal is a particular type of signal that oscillates between a high and a low state and is used like a metronome to coordinate actions of digital circuits. Most integrated circuits (ICs) of sufficient complexity use a clock signal in order to synchronize different parts of the circuit, cycling at a rate slower than the worst-case internal propagation delays. The preeminent example of the aforementioned integrated circuit is the CPU520, the central component of modern computers, which relies on a clock. The only exceptions are asynchronous circuits such as asynchronous CPUs. The clock510can comprise a plurality of embodiments, such as, but not limited to, single-phase clock which transmits all clock signals on effectively 1 wire, two-phase clock which distributes clock signals on two wires, each with non-overlapping pulses, and four-phase clock which distributes clock signals on 5 wires.

Many computing devices500use a “clock multiplier” which multiplies a lower frequency external clock to the appropriate clock rate of the CPU520. This allows the CPU520to operate at a much higher frequency than the rest of the computer, which affords performance gains in situations where the CPU520does not need to wait on an external factor (like memory550or input/output560). Some embodiments of the clock510may include dynamic frequency change, where, the time between clock edges can vary widely from one edge to the next and back again.

A system consistent with an embodiment of the disclosure the computing device500may include the CPU unit520comprising at least one CPU Core521. A plurality of CPU cores521may comprise identical CPU cores521, such as, but not limited to, homogeneous multi-core systems. It is also possible for the plurality of CPU cores521to comprise different CPU cores521, such as, but not limited to, heterogeneous multi-core systems, big.LITTLE systems and some AMD accelerated processing units (APU). The CPU unit520reads and executes program instructions which may be used across many application domains, for example, but not limited to, general purpose computing, embedded computing, network computing, digital signal processing (DSP), and graphics processing (GPU). The CPU unit520may run multiple instructions on separate CPU cores521at the same time. The CPU unit520may be integrated into at least one of a single integrated circuit die and multiple dies in a single chip package. The single integrated circuit die and multiple dies in a single chip package may contain a plurality of other aspects of the computing device500, for example, but not limited to, the clock510, the CPU520, the bus530, the memory550, and I/O560.

The CPU unit520may contain cache522such as, but not limited to, a level 1 cache, level 2 cache, level 3 cache or combination thereof. The aforementioned cache522may or may not be shared amongst a plurality of CPU cores521. The cache522sharing comprises at least one of message passing and inter-core communication methods may be used for the at least one CPU Core521to communicate with the cache522. The inter-core communication methods may comprise, but not limited to, bus, ring, two-dimensional mesh, and crossbar. The aforementioned CPU unit520may employ symmetric multiprocessing (SMP) design.

The plurality of the aforementioned CPU cores521may comprise soft microprocessor cores on a single field programmable gate array (FPGA), such as semiconductor intellectual property cores (IP Core). The plurality of CPU cores521architecture may be based on at least one of, but not limited to, Complex instruction set computing (CISC), Zero instruction set computing (ZISC), and Reduced instruction set computing (RISC). At least one of the performance-enhancing methods may be employed by the plurality of the CPU cores521, for example, but not limited to Instruction-level parallelism (ILP) such as, but not limited to, superscalar pipelining, and Thread-level parallelism (TLP).

Consistent with the embodiments of the present disclosure, the aforementioned computing device500may employ a communication system that transfers data between components inside the aforementioned computing device500, and/or the plurality of computing devices500. The aforementioned communication system will be known to a person having ordinary skill in the art as a bus530. The bus530may embody internal and/or external plurality of hardware and software components, for example, but not limited to a wire, optical fiber, communication protocols, and any physical arrangement that provides the same logical function as a parallel electrical bus. The bus530may comprise at least one of, but not limited to a parallel bus, wherein the parallel bus carry data words in parallel on multiple wires, and a serial bus, wherein the serial bus carry data in bit-serial form. The bus530may embody a plurality of topologies, for example, but not limited to, a multidrop/electrical parallel topology, a daisy chain topology, and a connected by switched hubs, such as USB bus. The bus530may comprise a plurality of embodiments, for example, but not limited to:

Internal data bus (data bus)531/Memory bus

System Management Bus (SMBus)

External Bus Interface (EBI)Local bus

Expansion busLightning bus

Controller Area Network (CAN bus)

Advanced Technology management Attachment (ATA), including embodiments and derivatives such as, but not limited to, Integrated Drive Electronics (IDE)/Enhanced IDE (EIDE), ATA Packet Interface (ATAPI), Ultra-Direct Memory Access (UDMA), Ultra ATA (UATA)/Parallel ATA (PATA)/Serial ATA (SATA), CompactFlash (CF) interface, Consumer Electronics ATA (CE-ATA)/Fiber Attached Technology Adapted (FATA), Advanced Host Controller Interface (AHCI), SATA Express (SATAe)/External SATA (eSATA), including the powered embodiment eSATAp/Mini-SATA (mSATA), and Next Generation Form Factor (NGFF)/M.2.

Small Computer System Interface (SCSI)/Serial Attached SCSI (SAS)

Mobile Industry Processor Interface (MIPI)

Coherent Processor Interface (CAPI)

Peripheral Component Interconnect (PCI), including embodiments such as, but not limited to, Accelerated Graphics Port (AGP), Peripheral Component Interconnect eXtended (PCI-X), Peripheral Component Interconnect Express (PCI-e) (e.g., PCI Express Mini Card, PCI Express M.2 [Mini PCIe v2], PCI Express External Cabling [ePCIe], and PCI Express OCuLink [Optical Copper{Cu} Link]), Express Card, AdvancedTCA, AMC, Universal IO, Thunderbolt/Mini DisplayPort, Mobile PCIe (M-PCIe), U.2, and Non-Volatile Memory Express (NVMe)/Non-Volatile Memory Host Controller Interface Specification (NVMHCIS).

Industry Standard Architecture (ISA), including embodiments such as, but not limited to Extended ISA (EISA), PC/XT-bus/PC/AT-bus/PC/105 bus (e.g., PC/105-Plus, PCI/105-Express, PCI/105, and PCI-105), and Low Pin Count (LPC).

Music Instrument Digital Interface (MIDI)

Universal Serial Bus (USB), including embodiments such as, but not limited to, Media Transfer Protocol (MTP)/Mobile High-Definition Link (MHL), Device Firmware Upgrade (DFU), wireless USB, InterChip USB, IEEE 1395 Interface/Firewire, Thunderbolt, and eXtensible Host Controller Interface (xHCI).

Consistent with the embodiments of the present disclosure, the aforementioned computing device500may employ hardware integrated circuits that store information for immediate use in the computing device500, know to the person having ordinary skill in the art as primary storage or memory550. The memory550operates at high speed, distinguishing it from the non-volatile storage sub-module561, which may be referred to as secondary or tertiary storage, which provides slow-to-access information but offers higher capacities at lower cost. The contents contained in memory550, may be transferred to secondary storage via techniques such as, but not limited to, virtual memory and swap. The memory550may be associated with addressable semiconductor memory, such as integrated circuits consisting of silicon-based transistors, used for example as primary storage but also other purposes in the computing device500. The memory550may comprise a plurality of embodiments, such as, but not limited to volatile memory, non-volatile memory, and semi-volatile memory. It should be understood by a person having ordinary skill in the art that the ensuing are non-limiting examples of the aforementioned memory:

Volatile memory which requires power to maintain stored information, for example, but not limited to, Dynamic Random-Access Memory (DRAM)551, Static Random-Access Memory (SRAM)552, CPU Cache memory525, Advanced Random-Access Memory (A-RAM), and other types of primary storage such as Random-Access Memory (RAM).

Non-volatile memory which can retain stored information even after power is removed, for example, but not limited to, Read-Only Memory (ROM)553, Programmable ROM (PROM)555, Erasable PROM (EPROM)555, Electrically Erasable PROM (EEPROM)556(e.g., flash memory and Electrically Alterable PROM [EAPROM]), Mask ROM (MROM), One Time Programable (OTP) ROM/Write Once Read Many (WORM), Ferroelectric RAM (FeRAM), Parallel Random-Access Machine (PRAM), Split-Transfer Torque RAM (STT-RAM), Silicon Oxime Nitride Oxide Silicon (SONOS), Resistive RAM (RRAM), Nano RAM (NRAM), 3D XPoint, Domain-Wall Memory (DWM), and millipede memory.

Semi-volatile memory which may have some limited non-volatile duration after power is removed but loses data after said duration has passed. Semi-volatile memory provides high performance, durability, and other valuable characteristics typically associated with volatile memory, while providing some benefits of true non-volatile memory. The semi-volatile memory may comprise volatile and non-volatile memory and/or volatile memory with battery to provide power after power is removed. The semi-volatile memory may comprise, but not limited to spin-transfer torque RAM (STT-RAM).

Consistent with the embodiments of the present disclosure, the aforementioned computing device500may employ the communication system between an information processing system, such as the computing device500, and the outside world, for example, but not limited to, human, environment, and another computing device500. The aforementioned communication system will be known to a person having ordinary skill in the art as I/O560. The I/O module560regulates a plurality of inputs and outputs with regard to the computing device500, wherein the inputs are a plurality of signals and data received by the computing device500, and the outputs are the plurality of signals and data sent from the computing device500. The I/O module560interfaces a plurality of hardware, such as, but not limited to, non-volatile storage561, communication devices562, sensors563, and peripherals565. The plurality of hardware is used by the at least one of, but not limited to, human, environment, and another computing device500to communicate with the present computing device500. The I/O module560may comprise a plurality of forms, for example, but not limited to channel I/O, port mapped I/O, asynchronous I/O, and Direct Memory Access (DMA).

Consistent with the embodiments of the present disclosure, the aforementioned computing device500may employ the non-volatile storage sub-module561, which may be referred to by a person having ordinary skill in the art as one of secondary storage, external memory, tertiary storage, off-line storage, and auxiliary storage. The non-volatile storage sub-module561may not be accessed directly by the CPU520without using intermediate area in the memory550. The non-volatile storage sub-module561does not lose data when power is removed and may be two orders of magnitude less costly than storage used in memory module, at the expense of speed and latency. The non-volatile storage sub-module561may comprise a plurality of forms, such as, but not limited to, Direct Attached Storage (DAS), Network Attached Storage (NAS), Storage Area Network (SAN), nearline storage, Massive Array of Idle Disks (MAID), Redundant Array of Independent Disks (RAID), device mirroring, off-line storage, and robotic storage. The non-volatile storage sub-module (561) may comprise a plurality of embodiments, such as, but not limited to:

Magnetic storage such as, but not limited to, Hard Disk Drive (HDD), tape drive, carousel memory, and Card Random-Access Memory (CRAM).

Holographic data storage such as Holographic Versatile Disk (HVD).

Molecular Memory

Deoxyribonucleic Acid (DNA) digital data storage

Consistent with the embodiments of the present disclosure, the aforementioned computing device500may employ the communication sub-module562as a subset of the I/O560, which may be referred to by a person having ordinary skill in the art as at least one of, but not limited to, computer network, data network, and network. The network allows computing devices500to exchange data using connections, which may be known to a person having ordinary skill in the art as data links, between network nodes. The nodes comprise network computer devices500that originate, route, and terminate data. The nodes are identified by network addresses and can include a plurality of hosts consistent with the embodiments of a computing device500. The aforementioned embodiments include, but not limited to personal computers, phones, servers, drones, and networking devices such as, but not limited to, hubs, switches, routers, modems, and firewalls.

Two nodes can be said are networked together, when one computing device500is able to exchange information with the other computing device500, whether or not they have a direct connection with each other. The communication sub-module562supports a plurality of applications and services, such as, but not limited to World Wide Web (WWW), digital video and audio, shared use of application and storage computing devices500, printers/scanners/fax machines, email/online chat/instant messaging, remote control, distributed computing, etc. The network may comprise a plurality of transmission mediums, such as, but not limited to conductive wire, fiber optics, and wireless. The network may comprise a plurality of communications protocols to organize network traffic, wherein application-specific communications protocols are layered, may be known to a person having ordinary skill in the art as carried as payload, over other more general communications protocols. The plurality of communications protocols may comprise, but not limited to, IEEE 802, ethernet, Wireless LAN (WLAN/Wi-Fi), Internet Protocol (IP) suite (e.g., TCP/IP, UDP, Internet Protocol version 5 [IPv5], and Internet Protocol version 6 [IPv6]), Synchronous Optical Networking (SONET)/Synchronous Digital Hierarchy (SDH), Asynchronous Transfer Mode (ATM), and cellular standards (e.g., Global System for Mobile Communications [GSM], General Packet Radio Service [GPRS], Code-Division Multiple Access [CDMA], and Integrated Digital Enhanced Network [IDEN]).

The communication sub-module562may comprise a plurality of size, topology, traffic control mechanism and organizational intent. The communication sub-module562may comprise a plurality of embodiments, such as, but not limited to:

Wired communications, such as, but not limited to, coaxial cable, phone lines, twisted pair cables (ethernet), and InfiniBand.

Parallel communications, such as, but not limited to, LPT ports.

Serial communications, such as, but not limited to, RS-232 and USB.

Fiber Optic communications, such as, but not limited to, Single-mode optical fiber (SMF) and Multi-mode optical fiber (MMF).

Power Line and wireless communications

The aforementioned network may comprise a plurality of layouts, such as, but not limited to, bus network such as ethernet, star network such as Wi-Fi, ring network, mesh network, fully connected network, and tree network. The network can be characterized by its physical capacity or its organizational purpose. Use of the network, including user authorization and access rights, differ accordingly. The characterization may include, but not limited to nanoscale network, Personal Area Network (PAN), Local Area Network (LAN), Home Area Network (HAN), Storage Area Network (SAN), Campus Area Network (CAN), backbone network, Metropolitan Area Network (MAN), Wide Area Network (WAN), enterprise private network, Virtual Private Network (VPN), and Global Area Network (GAN).

Consistent with the embodiments of the present disclosure, the aforementioned computing device500may employ the sensors sub-module563as a subset of the I/O560. The sensors sub-module563comprises at least one of the devices, modules, and subsystems whose purpose is to detect events or changes in its environment and send the information to the computing device500. Sensors are sensitive to the measured property, are not sensitive to any property not measured, but may be encountered in its application, and do not significantly influence the measured property. The sensors sub-module563may comprise a plurality of digital devices and analog devices, wherein if an analog device is used, an Analog to Digital (A-to-D) converter must be employed to interface the said device with the computing device500. The sensors may be subject to a plurality of deviations that limit sensor accuracy. The sensors sub-module563may comprise a plurality of embodiments, such as, but not limited to, chemical sensors, automotive sensors, acoustic/sound/vibration sensors, electric current/electric potential/magnetic/radio sensors, environmental/weather/moisture/humidity sensors, flow/fluid velocity sensors, ionizing radiation/particle sensors, navigation sensors, position/angle/displacement/distance/speed/acceleration sensors, imaging/optical/light sensors, pressure sensors, force/density/level sensors, thermal/temperature sensors, and proximity/presence sensors. It should be understood by a person having ordinary skill in the art that the ensuing are non-limiting examples of the aforementioned sensors:

Flow and fluid velocity sensors, such as, but not limited to, air flow meter, anemometer, flow sensor, gas meter, mass flow sensor, and water meter.

Force, Density, and Level sensors, such as, but not limited to, bhangmeter, hydrometer, force gauge or force sensor, level sensor, load cell, magnetic level or nuclear density sensor or strain gauge, piezocapacitive pressure sensor, piezoelectric sensor, torque sensor, and viscometer.

Consistent with the embodiments of the present disclosure, the aforementioned computing device500may employ the peripherals sub-module562as a subset of the I/O560. The peripheral sub-module565comprises ancillary devices uses to put information into and get information out of the computing device500. There are 3 categories of devices comprising the peripheral sub-module565, which exist based on their relationship with the computing device500, input devices, output devices, and input/output devices. Input devices send at least one of data and instructions to the computing device500. Input devices can be categorized based on, but not limited to:

Modality of input, such as, but not limited to, mechanical motion, audio, visual, and tactile.

Whether the input is discrete, such as but not limited to, pressing a key, or continuous such as, but not limited to position of a mouse.

The number of degrees of freedom involved, such as, but not limited to, two-dimensional mice vs three-dimensional mice used for Computer-Aided Design (CAD) applications.

Output devices provide output from the computing device500. Output devices convert electronically generated information into a form that can be presented to humans. Input/output devices perform that perform both input and output functions. It should be understood by a person having ordinary skill in the art that the ensuing are non-limiting embodiments of the aforementioned peripheral sub-module565:

Input Devices

High degree of freedom devices, that require up to six degrees of freedom such as, but not limited to, camera gimbals, Cave Automatic Virtual Environment (CAVE), and virtual reality systems.

Video Input devices are used to digitize images or video from the outside world into the computing device500. The information can be stored in a multitude of formats depending on the user's requirement. Examples of types of video input devices include, but not limited to, digital camera, digital camcorder, portable media player, web cam, Microsoft Kinect, image scanner, fingerprint scanner, barcode reader, 3D scanner, laser rangefinder, eye gaze tracker, computed tomography, magnetic resonance imaging, positron emission tomography, medical ultrasonography, TV tuner, and iris scanner.

Audio input devices are used to capture sound. In some cases, an audio output device can be used as an input device, in order to capture produced sound. Audio input devices allow a user to send audio signals to the computing device500for at least one of processing, recording, and carrying out commands. Devices such as microphones allow users to speak to the computer in order to record a voice message or navigate software. Aside from recording, audio input devices are also used with speech recognition software. Examples of types of audio input devices include, but not limited to microphone, Musical Instrumental Digital Interface (MIDI) devices such as, but not limited to a keyboard, and headset.

Data Acquisition (DAQ) devices convert at least one of analog signals and physical parameters to digital values for processing by the computing device500. Examples of DAQ devices may include, but not limited to, Analog to Digital Converter (ADC), data logger, signal conditioning circuitry, multiplexer, and Time to Digital Converter (TDC).

Output Devices may further comprise, but not be limited to:

Display devices, which convert electrical information into visual form, such as, but not limited to, monitor, TV, projector, and Computer Output Microfilm (COM). Display devices can use a plurality of underlying technologies, such as, but not limited to, Cathode-Ray Tube (CRT), Thin-Film Transistor (TFT), Liquid Crystal Display (LCD), Organic Light-Emitting Diode (OLED), MicroLED, E Ink Display (ePaper) and Refreshable Braille Display (Braille Terminal).

Printers, such as, but not limited to, inkjet printers, laser printers, 3D printers, solid ink printers and plotters.

Audio and Video (AV) devices, such as, but not limited to, speakers, headphones, amplifiers and lights, which include lamps, strobes, DJ lighting, stage lighting, architectural lighting, special effect lighting, and lasers.

Other devices such as Digital to Analog Converter (DAC)

Input/Output Devices may further comprise, but not be limited to, touchscreens, networking device (e.g., devices disclosed in network562sub-module), data storage device (non-volatile storage561), facsimile (FAX), and graphics/sound cards.

Appendix ii

TABLE 1Data statistics. During training, we randomly select 15% training samplesfor validation. The ratio of learning/inference samples is controlledto be about 4:1. The trials before the split date are used for learningwhile the later trials are used for inference. “Success”, “Fail”are abbreviations for success and failure, respectively.TrainTestSplitSettingsuccessfailuresuccessfailDatePhase I702386199113Aug. 13, 2014Phase II9561655302487Mar. 20, 2014Phase III1,8202,493457684Apr. 7, 2014Indication1,8642,922473674May 21, 2014