BIASED BASED DELEGATION IN MACHINE LEARNING MODELS

An approach is provided in which the approach receives a job request, which includes biasing parameters, from an entity operating in a distributive cognitive system. The approach evaluates the biasing parameters against a set of machine learning model bias characteristics corresponding to a set of machine learning models and selects one of the machine learning models based on the evaluating. The approach assigns the selected machine learning model to the entity to perform the job request.

BACKGROUND

A cognitive network (CN) is a new type of data network that utilizes cutting edge technology from several research areas to solve problems in current networks (e.g., machine learning, knowledge representation, etc.). Cognitive networks include a cognition plane, which is a decentralized system that enables self-management, self-control, and self-optimization of networks and service platforms. The cognition plane exploits data mining, reasoning and machine learning algorithms and techniques to extract knowledge on the status of the network and service platforms.

Cognitive models, also referred to as “cognitive entities,” are aimed to remember the past, interact with humans, continuously learn, and refine future responses. Their cognitive capabilities enrich human needs automation based on time and situation and provide more dynamic responses and user satisfaction. Machine learning models are a major component of cognitive entities.

A cognitive network typically includes many machine learning models with different functions. Machine learning models typically include input feature sets and a mathematical model to compute outcomes. The outcomes vary based on the type of machine leaning model, its algorithm, input training corpus, and other interrelated fields. A machine learning model may be classification-dependent or regression-dependent based on the need of the environment.

BRIEF SUMMARY

According to one embodiment of the present disclosure, an approach is provided in which the approach receives a job request, which includes biasing parameters, from an entity operating in a distributive cognitive system. The approach evaluates the biasing parameters against a set of machine learning model bias characteristics corresponding to a set of machine learning models and selects one of the machine learning models based on the evaluating. The approach assigns the selected machine learning model to the entity to perform the job request.

DETAILED DESCRIPTION

ExpressCard155is a slot that connects hot-pluggable devices to the information handling system. ExpressCard155supports both PCI Express and Universal Serial Bus (USB) connectivity as it connects to Southbridge135using both the USB and the PCI Express bus. Southbridge135includes USB Controller140that provides USB connectivity to devices that connect to the USB. These devices include webcam (camera)150, infrared (IR) receiver148, keyboard and trackpad144, and Bluetooth device146, which provides for wireless personal area networks (PANs). USB Controller140also provides USB connectivity to other miscellaneous USB connected devices142, such as a mouse, removable nonvolatile storage device145, modems, network cards, Integrated Services Digital Network (ISDN) connectors, fax, printers, USB hubs, and many other types of USB connected devices. While removable nonvolatile storage device145is shown as a USB-connected device, removable nonvolatile storage device145could be connected using a different interface, such as a Firewire interface, etcetera.

As discussed above, a cognitive network is a new type of data network that utilizes cutting edge technology from several research areas to solve problems in current networks. One of the common emerging ecosystems for dissimilar machine learning modeling is a 5G service orchestration layer. In a 5G domain level orchestration, the cognitive orchestration services are deployed and utilize various machine learning models to achieve real time cognition capabilities for user data.

Since machine learning models are trained using a particular training data set, machine learning models are inherently biased. Machine learning model biasing is a systematic pattern of deviation from normal or rationality in judgment based on the input parameters and feature sets of machine learning models. The machine learning models operate on the feature sets to make the decision of the real-time problems. In biased-based functionalities of a machine learning model, the machine learning model is more focused (or defocused) towards some of the parameters to make decisions. For example, if a machine learning model of loan approval uses a feature set as {A, B and C}, then in an A-based machine learning model biasing, the machine learning model produces outcomes relevant to the A feature and ignores the other features in the set. At the time of urgent job computing requirements of complex tasks, the biasing is one of the major factors in machine learning model outcome generation that is gaining popularity.

However, in today's multi-domain cognitive systems, there is no way to select a particular machine learning model for favoritism orientation (biasing) to produce a situational outcome for submitted jobs. In distributed cognitive systems, there is no way a machine learning model in the orchestration plane is discoverable and grouped based on their biasing nature and invoked when needed. In a situation where an external cognitive entity needs a job to be computed in a biased mode, there is no way that a machine learning model router (e.g., job submission router) can sense the request and route the job to respective machine learning models to produce a faster and desired outcome for the biasing expectations of the external cognitive entity.

In addition, there is no way that machine learning models in the cognitive system's orchestration plane can discover each other and collect peer information about their individual bias parameters along with the training corpus information, feature-sets and related artifacts of the machine learning model and which can be used for biased oriented delegation of the jobs. In distributed cognitive systems, or multi-domain cognitive systems in the cloud deployment with many machine learning models attachments, consequences arise when some of the jobs need to be biased on some parameters, such as for external analytics purpose or situational urgency and external software biasing requirements.

Today's machine learning model job submission router considers machine learning model activation status and workloads while assigning jobs, but the job submission router does not take into consideration the biased feature-sets and their importance values to select a target machine learning model. Because of this, customer interacting machine learning models in the orchestration plane follow the job submission router process and the jobs are submitted to the respective models. In addition, there is no way that the biasing requests are transferred to a correct basing model in the orchestration plane because of lagging metadata exchange based on biased parameters and weightage factors.

Furthermore, today's job submission routers are not able to collect information from machine learning models regarding basing capabilities and feature functions. A machine learning model may be different operating on a different feature set, running on different training data corpus, and may have deployed on a different computer infrastructure with different price/performance characters. Because of these complex factors associated with each machine learning model, it becomes very difficult to choose a correct machine learning model to compute results for a given task.

FIGS. 3 through 10depict an approach that can be executed on an information handling system that resolves the aforementioned problems by enabling the job submission router to track machine learning model biasing and dispatch new biased-based jobs to an appropriately biased machine learning model. As there are multiple machine learning models floating in various layers that are hosted with different feature alignment and biasing attributes, the approach provides a mechanism in a job submission router to discover machine learning models and capture their corresponding bias emphasis functionality.

The approach works with machine learning model and artificial intelligence (AI) job submitter functions in multi-domain cognitive systems and programmability framework by providing a primary-secondary discovery model for finding machine learning model characteristics in the cognitive system. The approach further initiates an inquiry for feature biasing and feature-set's individual weightage identification. The approach further considers the parameters for biased-based computing in the machine learning model to complete a job and create machine learning model groups in the router functions accordingly.

FIG. 3is an exemplary high-level diagram depicting a router function that learns of, and assigns, feature-biased machine learning models to job requests. System300includes cognition plane310, which embraces a monitor-analyses-plan-execute process governed by a knowledge-based approach for automated and autonomic network management. In particular, cognition plane310supports machine learning for monitoring and analysis steps, as well as for creating new knowledge. Cognition plane310focuses on determining changes in physical/virtual infrastructure350and orchestration plane320enforces the actions onto control plane340.

Orchestration plane320includes router function330, also referred to herein as a job submission router. Router function330computes feature-biased attributes in machine learning models335and articulates feature sets for augmented or deflected feature sets and accordingly selects a suitable feature-biased machine learning model for influence emphasized delegation in the router functions. Each of machine learning models335instances include demon338.

As discussed herein, demon338collects information pertaining to its corresponding machine learning model335and shares the information with router function330. In one embodiment, demon338computes the attributes, feature-sets, etc. and works with its respective machine learning model335to gather and exchange the data. Router function330further considers supporting attributes such as a training corpus, feature-set distribution for classification or regression, and fake feature exploration of the machine learning model (seeFIG. 10and corresponding text for further details).

Router function330collects biasing factors and importance assignment from each of machine learning models335using a primary-secondary discovery protocol and creates virtual groups of the machine learning models based on the biasing features and the respective weightage. Metadata classes are generated for each machine learning model that saves the biasing attributes and importance driven classification utilized during feature-biased machine learning model delegation of job requests (seeFIG. 10and corresponding text for further details).

Orchestration plane320also tracks fake feature exploration and accordingly notifies router function330of fake features to utilize during feature-biased machine learning model selection. Fake features are features used in machine learning model by value extrapolation. Fake features are not actual values, but rather added by the machine learning model itself. For example, if a loan application requires {A, B, C} as attributes and the machine learning model is trained for {A, B}, then the machine learning model inserts a false value for {C} for the next set of computations. When external cognitive entity360requests a particular biasing parameter, router function330avoids selecting a machine learning model that has the particular biasing parameter as a fake feature.

In one embodiment, router function330enables improved results in multi-user cognitive systems by tracking dynamic biasing knowledge of machine learning models and selecting the best suitable feature-biased machine learning model335for a submitted job requested by external cognitive entity360. In one embodiment, router function330provides a way to communicate with peer machine learning models to exchange feature-sets, weightage factors, and biasing attributes, and add entries in a local table that router function330utilizes to generate more accurate outcomes.

In one embodiment, router function330auto-discovers model biasing to use in transferring a submitted job based on biased-focused distancing. In this embodiment, router function330adopts dynamic machine learning model addition and deletion in a local mapping database to deliver real-time user benefits.

FIG. 4is an exemplary flowchart showing steps taken in system initialization and demon plaguing. Processing commences at400whereupon, at step410, the process detects a machine learning model activation in system300's model space. For example, the activation may be performed by a system administrator, or be part of a system initialization when other processes start. At step420, the process allocates infrastructure resources to the machine learning model from base resources, such as memory resources, compute resources, storage resources, etc.

At step430, the process initiates a demon (demon338) within the machine learning model and collects information pertaining to the machine learning model and shares the information with router function330. In one embodiment, the demon computes the attributes, feature-sets, etc. and works with the machine learning model to gather and exchange the data. At step440, the process (router function330) performs a primary-secondary machine learning model discovery while the demon performs real-time mapping of the machine learning models in the orchestration plane (seeFIG. 9and corresponding text for further details).

At step450, the process (router function330) loads the data structures for the machine learning model database and starts discovery requests to other reachable machine learning models, at which point feature-biased machine learning model discovery begins (seeFIG. 5and corresponding text for further details).FIG. 4processing thereafter ends at495.

FIG. 5is an exemplary flowchart showing steps taken in bias based machine learning model discovery. Processing commences at500whereupon, at step510, the process initiates and sends biased-oriented discovery messages to machine learning models335. In one embodiment, router function330invokes in-band authorization services and collects universally unique identifiers (UUIDs) of the machine learning models as part of the authorization service.

Machine learning model demon processing commences at520whereupon, at step530, the demon invokes an API instances for feature and bias detection and computes favorite features, number of features in the set, and importance weightage assignments. At step540, the demon validates the information in the machine learning model by parsing the configuration settings.

At step550, the demon locates bias requirements to complete a job and, at step560, the demon formulates the model type, (classification or regression), bias features and the weightage factors into a tuple and generate a RESP command, such as DISCOVERY_RESP=<feature-set, Bias-feature[ ], weightage_LIST[ ]>. At step570, the demon shares the tuple as part of a response frame to router function330's bias based router. Demon processing thereafter ends at580.

Returning back to router function processing, at step590, the router function receives responses and groups the machine learning models based on their bias orientation, weightage, and fake feature exploration (seeFIG. 10and corresponding text for further details). Router function processing thereafter ends at595.

FIG. 6is an exemplary flowchart showing steps taken to validate fake features. In one embodiment, demon338, inside machine learning model335, performs the steps shown inFIG. 6. Processing commences at600whereupon, at step610, the process receives a fake feature identification request from router function330. At step620, the process invokes a set of configuration files to gather feature-set metadata and respective attributes corresponding to the fake feature request. At step630, the process extracts the metadata of the feature for each of the features received in the feature-set.

At step640, if the feature “is_fake”=TRUE, the process adds the feature to a fake feature list. At step650, the process generates the list of all the fake features of the machine learning model (seeFIG. 10and corresponding text for further details). At step660, the process returns the fake feature list “FAKE_LIST” to router function670. At step670, the process activates a sleep mode thread and polls for further requests.FIG. 6processing thereafter ends at695.

FIG. 7is an exemplary flowchart showing steps by a job routing model to manage machine learning information. Processing commences at700whereupon, at step710, the process receives expected biasing factors and orientation weightage from the biased-based router function. At step720, the process triggers a communication message to send an inquiry response to the biased-based router functions.

At step730, upon reception of the inquiry reply by the machine learning models, the process (e.g., service job submitter router) keeps the information in a performance map local database that is used at the time of situational needs (e.g., feature-set and favoritism). At step740, the process (e.g., router function330) maintains a list of the machine learning models with their input feature-set, type (classification or regression) and other parameters such as training corpus and metadata functions with accuracy.FIG. 7processing thereafter ends at795. In one embodiment, the process iteratively learns feature-biased machine learning model selections to reach quicker selection decisions and perform incremental group updates.

FIG. 8is an exemplary flowchart showing steps taken in selecting a bias based machine learning model and dispatching a job to the bias based machine learning model.FIG. 8processing commences at800whereupon, at step810, the process receives a new job request from external cognitive entity360. At step820, the process extracts and decodes the basing parameter and value requirements of the function in the request.

At step830, the process authenticates the service request to validate the biasing permissions. At step840, the process overrides the submission logic if the service is authenticated to use bias based services. At step850, the process filters the database for the machine learning models having appropriate bias characters matching the submitted job's requirements.

At step860, the process selects relatively free machine learning models to address the submitted job once the list of all the desired machine learning models is received. At step870, the process dispatches the job to the selected model for computation and, at step880, the process shares the results with external cognitive entity360and marks the operation as completed.FIG. 8processing thereafter ends at895.

FIG. 9is an exemplary diagram depicting a router function performing a primary-secondary discovery process and requesting/receiving bias based machine learning model information.

Diagram900shows that router function330includes authentication engine910, machine learning model discovery database920, model functions930, router functions940, and biased oriented router950. Authentication engine910ensures the data/metadata and other information is exchanged between authorized entities in orchestration plane320. Machine learning model discovery database920stores data structures of the machine learning model database. Model functions930include a set of functions on which a machine learning model works, such as a mathematical computing function Y=f(x) where f is function that performs as sqrt( ) operation. Router functions940include a process that takes care of job routing in system300. When system300detects a machine learning model activation in model space900within orchestration plane320, system300initiates a demon (demon338) in the machine learning model that collects information and shares the information with router function330.

Bias oriented router950performs a primary-secondary machine learning model discovery while the demon performs real-time mapping of the machine learning models in the orchestration plane's model space900. Bias oriented router950loads the data structures for the machine learning model database into machine learning model discovery database920and starts discovery requests to other reachable machine learning models M1, M2, M3, M4, M5, and M6. In one embodiment, when a new machine learning model is activated, router function330performs group inclusion steps as discussed herein and the updates in machine learning model groups are updated so that future requests can consider new models as well.

Diagram960shows that bias oriented router950sends “DISCOVERY_INT” requests to the reachable machine learning models M1, M2, M3, M4, M5, and M6. Diagram960shows discovery responses “DISCOVERY_RESP” from the various machine learning models. In one embodiment, each DISCOVERY_RESP includes information such as a triple <feature-set, Bias-feature[ ], weightage_LIST[ ]>.

FIG. 10is an exemplary diagram depicting a model feature set table and machine learning model grouping. Router function330requests information from the machine learning models about their fake features, and in response the machine learning models send the information back to router function330. As discussed above, the fake features are the features used in machine learning model by value extrapolation. Fake features are not the actual values, but rather added by the machine learning model itself.

Router function330tracks the fake features to select appropriate machine learning models at runtime because, for example, if a job function needs to be more weighted on function X, then router function330does not select a machine learning model with fake features X.

Table1000includes weightage factors1020and fake features1010. System300uses table1000to map fake feature exploration and notify router function330of the fake features during machine learning model selection.

Metadata mapper1030shows machine learning model groupings for same feature sets and response times. Grouping1040includes M1and M6because they both have x, y, z features with similar response times. Grouping1050includes M2and M3because they both have c, d, e features with similar response times. And, grouping1060includes M4and M5because they both have p, r features.

Turning to machine learning model M6, M6has a feature set: F={w, x, y, z} where “w” is biasing feature in the model. As such, when a request is received by router function330from external cognitive entity360to compute a job with biasing of “w”, then router function330invokes M6because the model has the same biasing parameters and will therefore produce better outcomes as per the expectation of cognitive biasing.