SPECIFICATION FORMAT FOR PREDICTIVE MODEL

Provided are systems and methods for generating specification of a predictive model. In one example, the method may include receiving a predictive model developed via a test environment, generating a specification for the predictive model, the specification comprising a description of the predictive model in a format that is configured to be parsed and integrated into a predictive analytics application, and storing the generated specification in memory.

BACKGROUND

Predictive analytics can guide organizations in making informed decisions. According to predictive analytics, predictive models are “learned” based on large volumes of historical data and the models are then deployed in a production environment to predict future scenarios. The production environment may require the predictive model to be described in a particular format, such as Structured Query Language or the like.

It may be difficult to generate a learned predictive model in a format required by a production environment. Optimization of such a predictive model for the production environment presents further difficulties which may be best handled by developers of the production environment. Accordingly, what is needed is a system to efficiently describe predictive models in an agnostic and parseable manner.

DETAILED DESCRIPTION

The example embodiments are directed to a system and method for generating a specification which includes a description of a predictive formula (regression, classification, etc.) (i.e., a predictive model). The specification may be in JavaScript Object Notation (JSON) format and may include a definition of the predictive model along with transformations applied on raw data and influencers (variables of the model). The JSON format is not program code but is a description that can be parsed by a consumer system to extract the predictive formula therefrom and any other information needed, and integrated within applications written in multiple types of programming languages. By generating the specification, the example embodiments enable an application developer/consumer to integrate the predictive model in a manner that best suits the application.

The specification may be generated by a producing system in a test environment. In some embodiments, the specification may be exported to a consuming system in a production environment, which parses the specification to extract the predictive model and generates native code to implement the predictive model. The predictive model described by the specification may include equations (polynomials) having variables, data ranges, and the like. For example, the predictive model may include an encoding function to be applied on each variable of the model, and formulas to compute various predictive indicators based on the model.

A predictive model may be trained (e.g., through machine learning) using historical data as is known in the art and may be used to provide a prediction based on new/live data. Predictive models can be applied to various domains such as supply chain, weather, machine/equipment assets, maintenance, and the like. The predictive model may be trained based on patterns, trends, anomalies, and the like, identified within historical data. As a non-limiting example, a predictive model may include a sum of different variables with a coefficient. Examples of predictive model types include regression, classification, clustering, time-series, and the like.

The example embodiments include a specification that identifies transformations applied on variables, encoding information, and formulas. The specification may define all transformations steps applied on the variables until the value of the predictive indicator is reached.

FIG. 1illustrates a computing environment100for deploying a predictive model in accordance with an example embodiment. Referring toFIG. 1, the environment100may include multiple executing environments such as a testing environment110(also referred to as a development environment) and a production environment120(also referred to as a live environment). In this example, the testing environment110is operated by a testing platform101and the production environment120is operated by a host platform102. For example, each of the testing platform101and/or the host platform102may be a server, a cloud platform, a database, a combination of devices, and the like. Although not shown inFIG. 1, in some cases, the testing environment110and the productive environment120may be operated by the same computing system or they may be operated by different devices or groups of devices.

Within the testing environment110, users such as a data scientist may build (train) the predictive model114based on historical training data112. The users may look for bugs, design defects, and the like, while evaluating a performance of the predictive model114through an iterative process. Meanwhile, the production environment120is where the model114may be deployed and put into operation for its intended use. For example, the predictive model114may be deployed from the testing environment110into the productive environment120and integrated with application122.

In industrial use cases, the testing environment110where changes are originally made and the production environment120(what end users use) are separated through several stages in between. This structured release management process allows for phased deployment (rollout), testing, and rollback in case of problems. The phased deployment may include various stages which may include an initial hypothesis stage where a hypothesis is proposed, a load and transform data stage where data relevant to the hypothesis is collected and converted to fit a framework, a feature identification stage where data scientists can tailor a model before building it, a model building stage where one or more machine learning algorithms may be selected based on various factors (data, use case, available computational resources, etc.) and used to created predictive model114, an evaluation stage where the predictive model114is evaluated with test data, and a deployment stage where the fully trained predictive model114is launched or otherwise deployed into the live production environment120where it can generate and output predictions based on live data124.

According to various embodiments, when the predictive model114is deployed from the testing environment110into the production environment120, one or more of the testing platform101and the host platform102may generate a specification describing the predictive model in a generic format. The specification can be parsed and integrated into the application122in order to deploy the predictive model114.

In some embodiments, a user interface enables a user to select one or more predictive models which may be deployed and integrated with an application. For example, the user interface may display or otherwise output a list of predictive models available for integration within an application. In order to integrate a predictive model into the application, a process200shown inFIG. 2may be performed.

Referring toFIG. 2, a producing system220may generate a specification230based on a a predictive model210. The specification230may describe all elements of the predictive model230(classification, regression, etc.) For example, the specification230may include the transformation of raw data and one or more influencers of the predictive model including encoding data. In addition to the predictive model210, the specification230can also include formulas of various predictive indicators (probabilities, error bars, odds ratio, etc.) The specification230may be conform to a format such as a JSON format.

The format specification230describes the predictive model210in a format that can be parsed by a consuming system240to extract the model and freely integrate the model within an application250. All subsequent processing applied on variables within the formulas can be defined and may include internal transformations applied on input variables to derive additional variables, encoding of the variables, computation of predicted values from the encoded variables, and the like.

FIGS. 3A-3Dillustrate examples of various objects of data which may be included within a specification, according to various example embodiments.FIG. 3Aillustrates an example of an equation object310,FIG. 3Billustrates an example of a predictive indicator object320,FIG. 3Cillustrates an example of a transformation object330, andFIG. 3Dillustrates an example of an influencer object340. It should be appreciated that the objects shown inFIGS. 3A-3Dare merely for purposes of example and are not meant to limit the types and amount of objects that may be included within the specification. Listed below is an example of a specification. The specification is described in the examples ofFIGS. 3A-3D, but may also be understood to include additional and/or different data.

Referring toFIG. 3A, the equation object310includes an equation of a predictive model which may be a classification model, regression model, or the like. When applying the equation to data points with the same structure as the data points used to train the model, the user can compute a predictive target value for the data points. In addition to the formula(s) to compute one or more predicted target values, other predictive indicator formulas can also be provided. Depending on a type of the model, a set of available indicators may vary. For example, the indicators may include a score which sorts data points from most likely to least likely, a probability that the data point is a target, an error bar which is associated with the score, and an odds ratio. As another example, the indicator may include an estimation indicator which estimates a target value of a data point.

In the example ofFIG. 3A, the equation object310is a JSON equation object. The formula may be applied on data points to compute a predicted target value and other predictive indicators. The encoding of each variable may be defined by an encoding property of the related object. For each variable, a transformation may be applied on the variable and an encoding may be applied on the transformed variable. The encoding function of the variable may be defined through a set of functions (equations or mapping functions), and a specific function may be used for a specific range or set of values. For example, the function to use may be based on a value of an influencer variable as described in Table 1 below.

TABLE 1If the value of the influencer variable is ...... then... included in one of the specified range... we apply a formula on the value of thevaluesinfluencer variable... included in one of the specific set of values... we retrieve the specific encoded valueassociated to the set... missing... we retrieve the value of the missing valueproperty... not included in any of these cases... we retrieve the value of the default valueproperty

The influencers of the equation object may contain a lists of variables taken into account in the equation. For example, the influencers can be encoded variables which are defined in the influencers property, predictive indicators defined in the equation property, or the like. The equation input variable may be the sum of all the variables.

Referring toFIG. 3B, the predictive indicator object320is an indicator which relates to a target. It may be computed by a predictive model. The predictive indicator object320may contain information related to the predictive indicator, especially the way to compute it using influencer variables. The predictive indicator object may include different properties such as variables, names, output types, transformations, and influencers. The properties may have descriptions, types, and an indicator of whether the property is mandatory or not mandatory.

Referring toFIG. 3C, the transformation object330represents a formula to apply for a specific range of values. The transformation object may include various properties such as minimum, maximum, indicators of whether the lower bound and the upper bound are included in the range, a slope value to use, an intercept value to use, and a formula to apply. The properties may include descriptions, types, and an indication of whether the property is mandatory.

Referring toFIG. 3D, the influencer object340represents an influencer variable. The influencer object340includes a name of the encoded version of the influencer variable, a name of the original variable, a transformation to apply to the original variable to get the influencer variable, a storage type of the variable (integer, number, string, data, datetime, etc.), a value type of the variable (nominal, ordinal, continuous, etc.), and a definition of the encoding of the influencer variable. The properties may include descriptions, types, and an indication of whether the property is mandatory.

FIG. 4illustrates a method400for generating a specification for a predictive model in accordance with an example embodiment. As an example, the method400may be performed by a database node included within a distributed database system. As another example, the method400may be performed by a computing device such as a server, a cloud platform, a computer, a user device, and the like. In some examples, the method400may be performed by a plurality of devices in combination. Referring toFIG. 4, in410, the method may include receiving a predictive model developed via a test environment. The predictive model may include one or more formulas therein to be used by predictive analytics (e.g., classification, regression, etc.).

In420, the method may include generating a specification for the predictive model, the specification comprising a description of a predictive formula of the predictive model in a format that is configured to be parsed and integrated into a predictive analytic. For example, the specification may be in a JSON format that is capable of being parsed and exported or otherwise integrated into a predictive analytic application regardless of a programming language used to develop the predictive analytic application. The formula may be a trained formula that performs a type of prediction such as a classification or a regression.

In some embodiments, the format specification may further include information describing internal transformations that are applied on input variables of the predictive formula to generate additional variables of the predictive formula. In some embodiments, the format specification may include a description of a plurality of predictive formulas corresponding to a plurality of steps of the predictive model. In some embodiments, the format specification may include encoding information of variables of the predictive formula. In some embodiments, the generating at420may include exporting the predictive formula from the predictive model to the format of the specification.

In430, the method may include storing the generated specification in memory. The specification may be stored in association with the predictive model, or in place of the predictive model and may be exported into a live environment where it can be parsed and the predictive model can be integrated into a predictive analytics application. In some embodiments, the method may further include the parsing and the integrating of the specification.

FIG. 5illustrates a computing system500in accordance with an example embodiment. For example, the computing system500may be a database node, a server, a cloud platform, a user device, or the like. In some embodiments, the computing system500may be distributed across multiple devices. Referring toFIG. 5, the computing system500includes a network interface510, a processor520, an output530, and a storage device540such as an in-memory storage (e.g., RAM, etc.) Although not shown inFIG. 5, the computing system500may also include or be electronically connected to other components such as a display, an input unit, a receiver, a transmitter, a persistent disk, and the like. The processor520may control the other components of the computing system500.

The network interface510may transmit and receive data over a network such as the Internet, a private network, a public network, an enterprise network, and the like. The network interface510may be a wireless interface, a wired interface, or a combination thereof. The processor520may include one or more processing devices each including one or more processing cores. In some examples, the processor520is a multicore processor or a plurality of multicore processors. Also, the processor520may be fixed or it may be reconfigurable. The output530may output data to an embedded display of the computing system500, an externally connected display, a display connected to the cloud, another device, and the like. For example, the output530may include a port, an interface, a cable, a wire, a board, and/or the like, with input/output capabilities. The network interface510, the output530, or a combination thereof, may interact with applications executing on other devices. The storage device540is not limited to a particular storage device and may include any known memory device such as RAM, NRAM, ROM, hard disk, and the like, and may or may not be included within the cloud environment. The storage540may store software modules or other instructions which can be executed by the processor520to perform the method400shown inFIG. 4.

According to various embodiments, the processor520may receive a predictive model developed via a test environment, and generate a specification describing the predictive model. According to various embodiments, the specification may include a description of the predictive model in a generic format that is configured to be parsed and integrated into a predictive analytics application. Furthermore, the storage device540(e.g., memory, etc.) may store the generated specification. For example, the predictive model may include one of a regression formula and a classification formula which is described in a generic format within the specification. In some embodiments, the specification may conform to JSON format.

In some embodiments, the specification may further include information about internal transformations that are applied on variables of the predictive model. In some embodiments, the specification may include a plurality of formulas corresponding to the predictive model. In some embodiments, the specification may further include encoding information of variables of the predictive model. In some embodiments, the processor520may export the predictive model to the format of the specification. In some embodiments, the processor520may further parse the specification to integrate the predictive model within a predictive analytics application.