SYSTEM AND METHOD FOR CONFIGURING AN ARTIFICIAL INTELLIGENCE PIPELINE

A system and a method for configuring an AI pipeline. The system may receive raw information from a database. Further, one or more performance attributes are extracted from the raw information. A meta dataset is generated from the raw information to train a meta learning model. The meta learning model is trained based on the meta dataset, simulation data, and one or more machine learning algorithms. Subsequently, a configuration dataset is predicted based on the meta learning model. Further, a configuration file is generated based on the configuration dataset, the meta learning model, and an error minimization strategy.

PRIORITY INFORMATION

The present application claims a priority from Indian Patent Application No. 202321066946 dated Oct. 5, 2023.

TECHNICAL FIELD

The present subject matter described herein, in general, relates to a system and a method for configuring an Artificial Intelligence (AI) pipeline, and in particular, for automatically generating a configuration file for the AI pipeline.

BACKGROUND

Artificial Intelligence (AI) pipelines have become an integral part of numerous applications, ranging from image recognition and natural language processing to predictive analytics and autonomous systems. The AI pipelines typically include various stages, such as data preprocessing, feature engineering, model training, and deployment. The performance and effectiveness of the AI pipelines heavily depend on the configuration settings chosen for each stage and component.

Configuring the AI pipelines involves selecting appropriate algorithms, setting hyperparameters, defining preprocessing techniques, and establishing data processing workflows. However, determining the optimal configuration for the AI pipelines is a complex task, often requiring substantial expertise and significant trial-and-error iterations. The vast space of available configuration options makes it challenging for users to identify the most suitable settings.

SUMMARY

Before the present system(s) and method(s), are described, it is to be understood that this application is not limited to the particular system(s), and methodologies described, as there can be multiple possible embodiments which are not expressly illustrated in the present disclosures. It is also to be understood that the terminology used in the description is for the purpose of describing the particular implementations or versions or embodiments only and is not intended to limit the scope of the present application. This summary is provided to introduce aspects related to a system and a method for configuring an AI pipeline. This summary is not intended to identify essential features of the claimed subject matter nor is it intended for use in determining or limiting the scope of the claimed subject matter.

In one implementation, a method for configuring an AI pipeline is disclosed. The method may comprise receiving raw information from at least one of a database, an external flat file, and a user. Further, one or more performance attributes may be extracted from the raw information. Subsequently, a meta dataset may be generated for a meta learning model based on the one or more performance attributes. Further, the meta learning model may be trained based on the meta dataset, simulation data, and one or more machine learning algorithms. Further, a configuration dataset may be predicted using the meta learning model. The configuration dataset may comprise at least one of a set of features, a training model, and a parameter search space. Further, a configuration file may be generated based on the configuration dataset, the meta learning model, and an error minimization strategy.

In another implementation, a system for configuring an AI pipeline is disclosed. The system may receive raw information from at least one of a database, an external flat file, and a user. Further, the system may extract one or more performance attributes from the raw information. Subsequently, a meta dataset may be generated for a meta learning model based on the one or more performance attributes. Furthermore, the system may train a meta learning model based on the meta dataset, simulation data, and one or more machine learning algorithms. Further, the system may predict a configuration dataset comprising at least one of a set of features, a training model, and a parameter search space. Further, the system generates a configuration file based on the configuration dataset, the meta learning model, and an error minimization strategy.

In another implementation, a non-transitory computer program product having embodied thereon a computer program for configuring an AI pipeline is disclosed. The program may comprise a program code for extracting one or more performance attributes from raw information received from a user. Further, the program may comprise a program code for generating a meta dataset for a meta learning model based on the performance parameters. Further, the program may comprise a program code for training the meta learning model based on the meta dataset, simulation data, and one or more machine learning algorithms. Furthermore, the program may comprise a program code for predicting a configuration dataset comprising at least one of a set of features, a training model, and a parameter search space based on the meta learning model. Finally, the program may comprise a program code for generating a configuration file based on the configuration dataset, the meta learning model, and an error minimization strategy.

DETAILED DESCRIPTION

Some embodiments of this disclosure, illustrating all its features, will now be discussed in detail. The words “extracting”, “generating,” “predicting,” and other forms thereof, are intended to be open ended in that an item or items following any one of these words is not meant to be an exhaustive listing of such item or items or meant to be limited to only the listed item or items. It must also be noted that as used herein and in the appended claims, the singular forms “a,” “an,” and “the” include plural references unless the context clearly dictates otherwise. Although any system and methods similar or equivalent to those described herein can be used in the practice or testing of embodiments of the present disclosure, the exemplary, system and methods are now described.

The disclosed embodiments are merely examples of the disclosure, which may be embodied in various forms. Various modifications to the embodiment will be readily apparent to those skilled in the art and the generic principles herein may be applied to other embodiments. However, one of ordinary skill in the art will readily recognize that the present disclosure is not intended to be limited to the embodiments described but is to be accorded the widest scope consistent with the principles and features described herein.

The present subject matter introduces techniques for automating the configuration of AI pipelines. By leveraging machine learning techniques and automated search algorithms, the techniques enable efficient exploration of the configuration space to identify the settings for each stage of the AI pipeline such that the AI pipeline produces accurate output. The techniques take into account various factors, such as dataset characteristics, task requirements, computational resources, and performance objectives, to guide the configuration process.

The present subject matter discloses a system and a method for configuring an AI pipeline to perform a set of tasks. More importantly, the present invention discloses automating configuration of the AI pipeline by generating and optimizing a configuration file for the AI pipeline using a meta learning model. Initially, the system receives raw information from a database. The raw information includes data related to the set of tasks and historic information about the company. The historic information may comprise at least one of one or more AI pipelines used by the company, tasks performed using the one or more AI pipelines, historic iterations of the one or more AI pipelines, and historic iterations of the AI pipeline. Further, the system extracts one or more performance attributes from the raw information. Further, the system generates a meta dataset from the raw information for a meta learning model based on the one or more performance attributes. Further, the system trains the meta learning model based on the meta dataset, simulation data, and one or more machine learning algorithms. Furthermore, the system predicts a configuration dataset for the AI pipeline based on the meta learning model. The configuration dataset comprises at least one of a set of features, a training model, and a parameter search space. The system then generates a configuration file for the AI pipeline based on the configuration dataset, the meta learning model, and an error minimization strategy.

Certain technical challenges exist in configuring an AI pipeline. One technical challenge faced in configuring the AI pipeline is that a large amount of training data is required to train complex algorithms for high accuracy. The solution presented in the embodiments herein is to use a meta learning approach to utilise prior knowledge and simulation to train the complex algorithms. Another technical challenge is that configuring an AI pipeline requires a lot of human effort and time to adjust parameters for model training. The solution presented in the embodiments herein is to automatically generate a configuration file comprising the parameters for model training. A meta learning model is trained to generate the configuration file and continuously adjust the parameters in the configuration file to achieve a predefined accuracy. The detailed functioning of the system 102 is described below with the help of figures.

Another technical challenge in configuring the AI pipeline is that achieving and maintaining high accuracy is difficult and time-consuming because the settings are manually adjusted by trial-and-error method. The solution presented in the embodiments herein is to use simulation data. The simulation data is generated by the system by automatically adjusting the settings iteratively until a predefined accuracy is achieved. The accuracy may also be improved by using error minimization strategies such as feature engineering, model selection, hyperparameter tuning, cross-validation, and data cleaning.

While aspects of the described system and method for configuring an AI pipeline may be implemented in any number of different computing systems, environments, and/or configurations, the embodiments are described in the context of the following exemplary system.

Referring now to FIG. 1, a network implementation 100 of a system 102 for configuring an AI pipeline is disclosed. It may be noted that one or more users may access the system 102 through one or more user devices 104-2, 104-3 . . . 104-N, collectively referred to as user devices 104, hereinafter, or applications residing on the user devices 104.

Although the present disclosure is explained considering that the system 102 is implemented on a server, it may be understood that the system 102 may be implemented in a variety of computing systems, such as a laptop computer, a desktop computer, a notebook, a workstation, a virtual environment, a mainframe computer, a server, a network server, a cloud-based computing environment. It will be understood that the system 102 may be accessed by multiple users through one or more user devices 104-1, 104-2 . . . 104-N. In one implementation, the system 102 may comprise the cloud-based computing environment in which the user may operate individual computing systems configured to execute remotely located applications. Examples of the user devices 104 may include but are not limited to, a portable computer, a personal digital assistant, a handheld device, and a workstation. The user devices 104 are communicatively coupled to the system 102 through a network 106. In another implementation, the system 102 may be implemented on a user device 104 as a stand-alone system.

In one implementation, the network 106 may be a wireless network, a wired network, or a combination thereof. The network 106 can be implemented as one of the different types of networks, such as intranet, local area network (LAN), wide area network (WAN), the internet, and the like. The network 106 may either be a dedicated network or a shared network. The shared network represents an association of the different types of networks that use a variety of protocols, for example, Hypertext Transfer Protocol (HTTP), Transmission Control Protocol/Internet Protocol (TCP/IP), Wireless Application Protocol (WAP), and the like, to communicate with one another. Further, the network 106 may include a variety of network devices, including routers, bridges, servers, computing devices, storage devices, and the like.

In one embodiment, the system 102 may include at least one processor 108, an input/output (I/O) interface 110, a memory 112, and a database 114. The at least one processor 108 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, Central Processing Units (CPUs), state machines, logic circuitries, and/or any devices that manipulate signals based on operational instructions. Among other capabilities, the at least one processor 108 is configured to fetch and execute computer-readable instructions stored in the memory 112.

The I/O interface 110 may include a variety of software and hardware interfaces, for example, a web interface, a graphical user interface, and the like. The I/O interface 110 may allow the system 102 to interact with the user directly or through the client devices 104. Further, the I/O interface 110 may enable the system 102 to communicate with other computing devices, such as web servers and external data servers (not shown). The I/O interface 110 can facilitate multiple communications within a wide variety of networks and protocol types, including wired networks, for example, LAN, cable, etc., and wireless networks, such as WLAN, cellular, or satellite. The I/O interface 110 may include one or more ports for connecting a number of devices to one another or to another server.

The memory 112 may include any computer-readable medium or computer program product known in the art including, for example, volatile memory, such as static random-access memory (SRAM) and dynamic random access memory (DRAM), and/or non-volatile memory, such as read only memory (ROM), erasable programmable ROM, flash memories, hard disks, Solid State Disks (SSD), optical disks, and magnetic tapes. The memory 112 may include routines, programs, objects, components, data structures, etc., which perform particular tasks or implement particular abstract data types. The memory 112 may include programs or coded instructions that supplement applications and functions of the system 102. In one embodiment, the memory 112, amongst other things, serves as a repository for storing data processed, received, and generated by one or more of the programs or the coded instructions.

The database 114 may store information received from a user and information generated by the system 102 as historic data such that the historic data may be accessed wirelessly via the network 106.

In operation, for configuring an AI pipeline, a user may use the user device 104 to access the system 102 via the I/O interface 110. The user may register the user devices 104 using the I/O interface 110 in order to use the system 102. In one aspect, the user may access the I/O interface 110 of the system 102.

The present subject matter describes the system 102 for configuring an AI pipeline. The system 102 may generate a configuration file for an Artificial Intelligence (AI) pipeline. In order to generate the configuration file, initially, the system 102 may receive raw information. In an embodiment, the system may receive the raw information from a database or from a user, for example, through a user device. The user may be an employee of a company planning to use the AI pipeline. The raw information comprises data related to a set of tasks for which the AI pipeline may be used. In an embodiment, the raw information may comprise information of the company including past records related to the set of tasks. For example, the past records may be bills and inventory data sheets for a set of tasks including inventory management. More examples of data included in the raw information are machine generated demand forecasts for past and future time periods, baseline and incremental sales, actual demand for historical periods, promo data, holiday data, competitor information, product attributes related information.

In another embodiment, the raw information received for a set of tasks may be used for another set of tasks to be performed by the AI pipeline. For example, let us assume that a first set of tasks was inventory management for a cosmetics company. The raw information included data regarding the domain of the company, number of products sold by the company, number of products manufactured by the company, etc. Let us assume that a new task is assigned to the AI pipeline. The new task may be sales management. The data such as the number of products sold, may be used to train the AI pipeline for the new task.

In another embodiment, the raw information may comprise requirements for completion of the task and budget information for the AI pipeline. Requirements may include accuracy, type of output, and the like. Budget information may include financial data of the company, invested amount for the AI project, and the like.

In an embodiment, the raw information may be extracted from external flat files. External flat files may contain the raw information in at least one of a textual format, a semi structured format, and in a predefined syntax.

Further to receiving the raw information, the system may use a descriptive statistics module 406 to extract performance attributes and generate a meta dataset for a meta learning model 404 based on the performance attributes, the raw information, and the set of tasks. In an embodiment, the descriptive statistics module may be used to extract the one or more performance attributes from the raw information. The descriptive statistics module is a logic code which extracts key information about the data to populate a database of attributes used by the meta learning model for configuring the AI pipeline The database may be called meta learning DB. For example, the attributes may include a set of values for the mean/median, distribution, frequency, skewness and kurtosis, percentiles and the like. The descriptive statistics module may analyse the raw information and identify patterns in the raw information, characteristics, and distribution of the raw information. In an embodiment, the raw information may include the set of tasks, information regarding each task in the set of tasks, past data entries regarding the set of tasks and requirements for the set of tasks (dependent variables/target variables/expected output). The descriptive statistics module may calculate statistics for the set of tasks based on the target variables and the past data entries. The statistics may be used to find similarity between one or more tasks from the set of tasks and understand relationships between the one or more tasks. The similar tasks may be clustered together such that the raw information for the one task of the similar tasks can be used to train the meta learning model for another task of the similar tasks by determining values for attributes used to train the meta learning model.

The meta learning model may be built to configure the AI pipeline using the raw information. The meta learning model may be trained to produce an output comprising a configuration file for the AI pipeline to perform the set of tasks. The descriptive statistics module may include a logic code that may identify and extract key information from raw information. The key information may include the one or more performance attributes, and the meta dataset.

In an embodiment, the performance attributes may include granularity of output, forecast/elasticity, optimization metrics, refresh frequency, cloud cost, execution time of the AI pipeline, and the like. The meta dataset may comprise a training set of tasks, corresponding input data for the training set of tasks, and corresponding output data for the training set of tasks, wherein the training set of tasks includes tasks other than the set of tasks to be performed by the AI pipeline. The input data may vary based on the set of tasks to be performed by the AI pipeline. Consider an example, the meta learning model is used to configure an AI pipeline for a task of demand forecasting for a packaged food supplier. The input data may include a set of training set of tasks such as increasing profit, improving delivery time, and the like. Further, input data corresponding to the training set of tasks may include product portfolio of the packaged food supplier, past sales, past delivery details, target markets, and the like. The output data corresponding to the training set of tasks may include a set of features, a set of parameters, and a type of machine learning technique used in the training model for the AI pipeline to perform the training set of tasks.

In another embodiment, the input data may comprise a set of tasks to be performed by an AI pipeline, performance attributes, and expected output of the AI pipeline. The output data may comprise corresponding configuration files for the AI pipeline for the set of tasks. Another example of input data may be a set of features, a set of parameters for the AI pipeline, an accuracy, and the output data may comprise corresponding set of values for the set of features and set of parameters.

In another embodiment, the system 102 may include a machine learning module in addition or as an alternative to the descriptive statistics module to extract key information from raw information using a machine learning algorithm. The machine learning algorithm may be trained using raw information related to a plurality of tasks for a plurality of companies and corresponding key information for each task for each company.

Further to extracting the meta dataset and performance attributes, the system 102 may train the meta learning model based on the meta dataset, simulation data, and one or more machine learning algorithms. The simulation data may comprise a testing dataset, for the meta learning model, comprising a meta testing dataset for a set of testing tasks, and corresponding output testing dataset for the set of testing tasks, and wherein output of the meta learning model is compared to the output testing dataset to generate feedback and retrain the meta learning model. The testing dataset is used to evaluate the performance of the trained meta-learning model. It includes new tasks or datasets that the meta learning model has not been exposed to during training. In an embodiment, a simulation module 408 may be used to generate (i.e., synthesize) the simulation data comprising the testing dataset using one or more machine learning algorithms. The one or more machine learning algorithms may be trained to generate testing dataset based on raw information, set of tasks, and historic information. Further, the one or more machine learning algorithms may be trained to calculate accuracy of the AI pipeline configured using the meta learning model. The accuracy may be calculated by comparing the output of the AI pipeline with the expected output provided in the testing dataset. Based on the accuracy, the one or more machine learning algorithms may provide feedback for generation of simulation data using different feature lists, machine learning models, and hyperparameters specified by the meta learning model. When the accuracy matches a predefined accuracy by optimizing the machine learning model to a particular level, the meta learning model may be considered as trained.

Further to training the meta learning model, the system may predict a configuration dataset for the AI pipeline based on the meta learning model. The configuration dataset may comprise at least one of a set of features, a training model, and a parameter search space. The set of features may comprise one or more data preprocessing features, one or more model training features, one or more model evaluation features, and model development features.

The data preprocessing features may include the following:

The model training features may include the following:

The model evaluation features may include the following:

The model deployment features may include the following:

The parameter search space or the hyperparameter space, may refer to the range or set of possible values that can be assigned to the hyperparameters of a machine learning model. The hyperparameter space represents the multidimensional space in which the hyperparameters exist, and it defines the boundaries within which hyperparameter tuning is performed.

The hyperparameter space can vary depending on the specific model and the hyperparameters being considered. Each hyperparameter typically has its own range or set of valid values. For example:

In this example, the hyperparameter space includes different possible combinations of values for the learning rate, number of hidden layers, number of neurons per layer, and activation function. The hyperparameter space can be represented as a Cartesian product of the individual hyperparameter sets.

The size and complexity of the hyperparameter space depend on the number of hyperparameters and the granularity of their values. As the number of hyperparameters increases or the ranges of values become larger, the hyperparameter space expands exponentially.

The training model may be at least one of Neural Networks, Decision Trees, Random Forests, k-Nearest Neighbors, Support Vector Machines, Multi-Label Naive Bayes, Logistic Regression, and the like.

Finally, the system may generate a configuration file for the AI pipeline by populating the configuration file with the predicted configuration dataset. The configuration file may be generated based on the meta learning model and an error minimization strategy. In one example, further, to generating an initial configuration file based on the configuration dataset, the system may modify the configuration file based on the error minimization strategy. The accuracy of the AI pipeline may be improved using the error minimization strategy. The accuracy of an AI pipeline/model is represented by Weighted Mean Absolute Percentage Error (WMAPE) or Mean Absolute Percentage Error (MAPE). MAPE is a commonly used metric to measure the accuracy and performance of a forecasting or predictive AI model, particularly in the field of time series analysis and demand forecasting. MAPE represents the average percentage difference between the predicted and actual values of a variable.

The formula for calculating the MAPE is as follows:

In the formula, ‘n’ is the number of observations or data points in the dataset. ‘Actual’ represents the actual observed values of the variable being predicted. ‘Predicted’ represents the corresponding predicted values generated by the model.

The formula for calculating WMAPE is as follows:

Where ‘Weight’ represents the weight assigned to each data point. The weights can be determined based on factors like the importance or frequency of data points in the forecasting task.

WMAPE allows for prioritizing certain data points in the calculation of the percentage errors, which can be beneficial when certain observations have a more significant impact on the overall forecasting accuracy.

MAPE provides an indication of the magnitude of the errors in percentage terms. It measures the average relative difference between the predicted and actual values, which allows for comparison across different scales and magnitudes of the variable being forecasted.

The error minimization strategy may comprise at least one of the following.

In another embodiment, a non-transitory computer program product having embodied thereon a computer program for configuring an AI pipeline is disclosed. The computer program product may receive raw information. Further, the computer program product may comprise a program code for a descriptive statistics module. The descriptive statistics module may be used to analyse the raw information to extract meta dataset and one or more performance attributes to train a meta learning model. Further, the computer program product may comprise a program code for training the meta learning model using the meta dataset, simulation data, and one or more machine learning models. The computer program product may comprise a program code for a simulation module to generate simulation data including meta testing data. Further, the computer program product may comprise a program code for generating a configuration dataset based on the meta learning model. The computer program product may comprise a program code to generate a configuration file based on the configuration dataset, the meta learning model, and an error minimization strategy.

Referring now to FIG. 2, a method 200 for configuring an AI pipeline is shown, in accordance with an embodiment of the present subject matter. The method 200 may be described in the general context of computer executable instructions. Generally, computer executable instructions can include routines, programs, objects, components, data structures, procedures, modules, functions, etc., that perform particular functions or implement particular abstract data types.

The order in which the method 200 is described is not intended to be construed as a limitation, and any number of the described method blocks can be combined in any order to implement the method 200 or alternate methods for configuring an AI pipeline. Furthermore, the method 200 for configuring an AI pipeline can be implemented in any suitable hardware, software, firmware, or combination thereof. However, for ease of explanation, in the embodiments described below, the method 200 may be considered to be implemented in the above-described system 102.

At block 202, one or more performance attributes may be extracted from the raw information.

At block 204, a meta dataset, from the raw information, for the meta learning model may be generated based on the one or more performance attributes.

At block 206, the meta learning model may be trained based on the meta dataset, simulation data, and one or more machine learning algorithms.

At block 208, a configuration dataset may be predicted based on the meta learning model.

At block 210, a configuration file may be generated based on the configuration dataset, the meta learning model, and an error minimization strategy.

FIG. 3A and FIG. 3B illustrate an example configuration file 300 for an AI pipeline. The meta learning model may be used to populate the data in the configuration file. In an embodiment, the user may provide input to modify the configuration file by adding one or more parameters or features. Part 302 comprises a set of performance attributes and a set of features. Part 304 comprises a set of features selected for the AI pipeline, and information regarding the features. Part 306 comprises parameters for training the AI pipeline and the model used by the AI pipeline. Part 308 comprises information regarding ensembling strategy used for the AI pipeline. The ensembling strategy allows using data from past usage of the AI pipeline for a new task.

FIG. 4 illustrates an example 400 of the system 102 for configuring an AI pipeline. As seen in the FIG. 4, the system may comprise a Meta Learning Database (Meta Learning DB) 402, the meta learning model 404, a descriptive statistics module 406, and a simulation module 408. The Meta Learning DB may comprise a list of attributes used to configure the AI pipeline for a set of tasks. An example of the list of attributes that a meta learning DB generated by the meta learning model while configuring the AI pipeline may comprise:

The meta learning DB may comprise multiple values for the above attributes that may be generated during training of the meta learning model 404. The meta learning model may be trained by the system to configure the AI pipeline for a set of tasks.

Further, the system may comprise the descriptive statistics module 406. The descriptive statistics module may analyze the raw information to generate the meta dataset and statistical insights from the raw information to train the meta learning model.

Further, the system may comprise a simulation module 408. The simulation module may be used to train the AI pipeline such that the AI pipeline achieves a predefined accuracy. The simulation module may use one or more machine learning algorithms for generating testing datasets.

Exemplary embodiments discussed above may provide certain advantages. Though not required to practice aspects of the disclosure, these advantages may include the following.

Some embodiments of the system and the method may increase efficiency of training AI models.

Some embodiments of the system and the method may reduce human effort in AI based applications.

Some embodiments of the system and the method may reduce the amount of training data required to train an AI pipeline.

Some embodiments of the system and the method may allow an AI pipeline to be trained for a new task using information from old tasks performed by the AI pipeline.

Some embodiments of the system and the method may use a simulation model to reduce time required to train the AI pipeline and increase efficiency of the AI pipeline by testing various settings of the AI pipeline.

Although implementations for methods and system for configuring an AI pipeline have been described in language specific to structural features and/or methods, it is to be understood that the appended claims are not necessarily limited to the specific features or methods described. Rather, the specific features and methods are disclosed as examples of implementations for configuring an AI pipeline.