INCREMENTAL MACHINE LEARNING USING EMBEDDINGS

An embodiment of the present invention is directed toward machine learning to produce results encompassing a new output. A machine learning model is trained to determine a candidate output from among a plurality of candidate outputs. First embeddings associated with the plurality of candidate outputs are generated from a first set of training data by an intermediate layer of the trained machine learning model. Second embeddings associated with a new candidate output are generated from a second set of training data by the intermediate layer of the trained machine learning model. A third embedding is determined for input data by the intermediate layer of the trained machine learning model. A resulting candidate output for the input data is predicted from a group of the plurality of candidate outputs and the new candidate output based on distances for the third embedding to the first and second embeddings.

BACKGROUND

1. Technical Field

Present invention embodiments relate to machine learning, and more specifically, to incremental machine learning of a new aspect (e.g., new output, new class, etc.) using embeddings without retraining a machine learning model.

2. Discussion of the Related Art

Classification is an area of machine learning that is used for classifying different inputs into a certain number of groups or classes. Classification is widely used in various areas, such as computer vision, natural language processing, and data analysis. Typically, classification models are created with an output layer of a fixed size including a number of nodes that equals the number of classes to predict (e.g., each node of an output layer corresponds to a predicted class).

If a new class is desired for addition to the classification model, the classification model will need to be reconfigured, reset, and retrained in order to include the new class. Accordingly, when the number of classes changes frequently, the classification model will constantly be reconfigured, reset, and retrained.

SUMMARY

According to one embodiment of the present invention, a system for machine learning to produce results encompassing a new output comprises at least one processor. A machine learning model is trained to determine a candidate output from among a plurality of candidate outputs. First embeddings associated with the plurality of candidate outputs are generated from a first set of training data. The first embeddings are produced from an intermediate layer of the trained machine learning model. Second embeddings associated with a new candidate output are generated from a second set of training data. The second embeddings are produced from the intermediate layer of the trained machine learning model. A third embedding is determined for input data by the intermediate layer of the trained machine learning model. A resulting candidate output for the input data is predicted from a group of the plurality of candidate outputs and the new candidate output based on distances for the third embedding to the first and second embeddings. Embodiments of the present invention further include a method and computer program product for machine learning to produce results encompassing a new output in substantially the same manner described above.

Present invention embodiments enable a new candidate output to be learned based on training data for that candidate output and without retraining the machine learning model, thereby conserving computing and memory resources and significantly increasing the speed of machine learning.

An embodiment of the present invention may employ a classification model as the machine learning model, where the plurality of candidate outputs includes classes, and the new candidate output includes a new class. This enables new classes to be added for classification without retraining the classification model, thereby improving computing performance especially for cases where classes change frequently.

An embodiment of the present invention may further predict the resulting candidate output by determining, from the first and second embeddings, a plurality of embeddings closest to the third embedding and determining the resulting candidate output based on candidate outputs associated with the determined plurality of embeddings. The embeddings and prediction are used in order to add new candidate outputs without retraining the machine learning model. The use of the embeddings (and the distances therebetween) further provides sufficient accuracy of the prediction to avoid retraining.

An embodiment of the present invention may also determine the resulting candidate output based on a candidate output associated with a majority of the determined plurality of embeddings. The use of the majority ensures an appropriate (and closest) candidate output is selected for enhancing accuracy of the prediction to avoid retraining the machine learning model.

An embodiment of the present invention may further determine a training score based on the first and second embeddings and retrain the machine learning model in response to the training score failing to satisfy a threshold. This enables the performance of the embeddings to be monitored to selectively trigger embedding retraining to increase the performance of the embeddings. Thus, the frequency of training the machine learning model can be significantly decreased, thereby improving computer performance.

An embodiment of the present invention may also have the first embeddings form a plurality of first clusters each associated with a corresponding candidate output and the second embeddings form a second cluster associated with the new candidate output. In this case, the training score may be determined based on distances between the second embeddings within the second cluster and distances between each of the second embeddings and the plurality of first clusters. This enables the performance of the embeddings to be monitored based on quality of the cluster formed by the embeddings for the new candidate output. When the cluster for the new candidate output is insufficient to distinguish the new candidate output from the plurality of candidate outputs, embedding retraining is conducted to increase the performance of the embeddings. Thus, the frequency of training the machine learning model can be significantly decreased, thereby improving computer performance.

In an embodiment of the present invention, the machine learning model comprises a neural network including an input layer, the intermediate layer, and an output layer for the plurality of candidate outputs, and the first, second, and third embeddings are generated by an embedding model. The embedding model includes the neural network of the trained machine learning model without the output layer. This structurally changes the neural network to form an embedding model that generates the embeddings based on the trained neural network. The embeddings enable new candidate outputs to be learned without retraining the neural network, thereby significantly increasing the speed of machine learning.

An embodiment of the present invention may also add the new candidate output to the group for predicting the resulting candidate output without retraining the machine learning model. This conserves computing and memory resources and significantly increases the speed of machine learning.

DETAILED DESCRIPTION

Present invention embodiments are directed to incremental machine learning of a new aspect (e.g., new output, new class, etc.) using embeddings without retraining a machine learning model. For example, when a new class is to be added for a machine learning classification model, the classification model will need to be reconfigured, reset, and retrained in order to include the new class. Accordingly, when the number of classes changes frequently, the classification model will constantly be reconfigured, reset, and retrained, thereby consuming significant computing and memory resources. However, present invention embodiments use embeddings for incremental machine learning (e.g., for deep learning models, classification models, etc.) when a new aspect (e.g., new output, new class, etc.) is added. Thus, incremental machine learning may be performed to learn the new aspect without retraining a machine learning model.

In order to prevent repetitive retraining, present invention embodiments include an embedding model to generate embeddings for data and a prediction model to predict a new added aspect (e.g., new output, new class, etc.) based on the embeddings from the embedding model using pattern recognition. This approach prevents retraining of the embedding model when a new aspect is added.

According to one embodiment of the present invention, a system for machine learning to produce results encompassing a new output comprises at least one processor. A machine learning model is trained to determine a candidate output from among a plurality of candidate outputs. First embeddings associated with the plurality of candidate outputs are generated from a first set of training data. The first embeddings are produced from an intermediate layer of the trained machine learning model. Second embeddings associated with a new candidate output are generated from a second set of training data. The second embeddings are produced from the intermediate layer of the trained machine learning model. A third embedding is determined for input data by the intermediate layer of the trained machine learning model. A resulting candidate output for the input data is predicted from a group of the plurality of candidate outputs and the new candidate output based on distances for the third embedding to the first and second embeddings. Embodiments of the present invention further include a method and computer program product for machine learning to produce results encompassing a new output in substantially the same manner described above.

Present invention embodiments enable a new candidate output to be learned based on training data for that candidate output and without retraining the machine learning model, thereby conserving computing and memory resources and significantly increasing the speed of machine learning.

An embodiment of the present invention may employ a classification model as the machine learning model, where the plurality of candidate outputs includes classes, and the new candidate output includes a new class. This has the advantage of adding new classes for classification without retraining the classification model, thereby improving performance especially for cases where classes change frequently.

An embodiment of the present invention may further predict the resulting candidate output by determining, from the first and second embeddings, a plurality of embeddings closest to the third embedding and determining the resulting candidate output based on candidate outputs associated with the determined plurality of embeddings. The embeddings and prediction are used in order to add new candidate outputs without retraining the machine learning model. The use of the embeddings (and the distances therebetween) further provides sufficient accuracy of the prediction to avoid retraining.

An embodiment of the present invention may also determine the resulting candidate output based on a candidate output associated with a majority of the determined plurality of embeddings. The use of the majority ensures an appropriate (and closest) candidate output is selected for enhancing accuracy of the prediction to avoid retraining the machine learning model.

An embodiment of the present invention may further determine a training score based on the first and second embeddings and retrain the machine learning model in response to the training score failing to satisfy a threshold. This enables the performance of the embeddings to be monitored to selectively trigger embedding retraining to increase the performance of the embeddings. Thus, the frequency of training the machine learning model can be significantly decreased, thereby improving computer performance.

An embodiment of the present invention may also have the first embeddings form a plurality of first clusters each associated with a corresponding candidate output and the second embeddings form a second cluster associated with the new candidate output. In this case, the training score may be determined based on distances between the second embeddings within the second cluster and distances between each of the second embeddings and the plurality of first clusters. This enables the performance of the embeddings to be monitored based on quality of the cluster formed by the embeddings for the new candidate output. When the cluster for the new candidate output is insufficient to distinguish the new candidate output from the plurality of candidate outputs, embedding retraining is conducted to increase the performance of the embeddings. Thus, the frequency of training the machine learning model can be significantly decreased, thereby improving computer performance.

In an embodiment of the present invention, the machine learning model comprises a neural network including an input layer, the intermediate layer, and an output layer for the plurality of candidate outputs, and the first, second, and third embeddings are generated by an embedding model. The embedding model includes the neural network of the trained machine learning model without the output layer. This structurally changes the neural network to form an embedding model that generates the embeddings based on the trained neural network. The embeddings enable new candidate outputs to be learned without retraining the neural network, thereby significantly improving the speed of machine learning.

An embodiment of the present invention may also add the new candidate output to the group for predicting the resulting candidate output without retraining the machine learning model. This conserves computing and memory resources and significantly increases the speed of machine learning.

An example environment100for use with present invention embodiments is illustrated inFIG.1. Specifically, the environment includes one or more server systems110, and one or more client or end-user systems114. Server systems110and client systems114may be remote from each other and communicate over a network112. The network may be implemented by any number of any suitable communications media (e.g., wide area network (WAN), local area network (LAN), Internet, Intranet, etc.). Alternatively, server systems110and client systems114may be local to each other, and communicate via any appropriate local communication medium (e.g., local area network (LAN), hardwire, wireless link, Intranet, etc.).

Client systems114enable users to submit information (e.g., inputs for machine learning models, new aspects (e.g., new outputs, new classes, etc.) for machine learning models, training data, etc.) to server systems110to manage and utilize machine learning models to produce outputs for a desired task (e.g., classification, etc.). The server systems include a machine learning module116to train, utilize, and maintain machine learning models for various tasks (e.g., classification, etc.) as described below. A database system118may store various information for the tasks (e.g., machine learning models and corresponding configurations and parameters, training data, results, etc.). The database system may be implemented by any conventional or other database or storage unit, may be local to or remote from server systems110and client systems114, and may communicate via any appropriate communication medium (e.g., local area network (LAN), wide area network (WAN), Internet, hardwire, wireless link, Intranet, etc.). The client systems may include an interface module120or browser to present a graphical user (e.g., GUI, etc.) or other interface (e.g., command line prompts, menu screens, etc.) to solicit information from users (e.g., inputs for machine learning models, new aspects (e.g., new outputs, new classes, etc.) for machine learning models, training data, etc.), and may provide reports including results (e.g., classification, outputs from the machine learning models, etc.).

Server systems110and client systems114may be implemented by any conventional or other computer systems preferably equipped with a display or monitor, a base, optional input devices (e.g., a keyboard, mouse or other input device), and any software (e.g., conventional or other server/communications software, conventional or other browser software, machine learning module116and interface module120of present invention embodiments, etc.). The base may include at least one hardware processor115(e.g., microprocessor, controller, central processing unit (CPU), etc.), one or more memories135and/or internal or external network interfaces or communications devices125(e.g., modem, network cards, etc.).

Alternatively, one or more client systems114may manage and utilize machine learning models to produce outputs for a desired task (e.g., classification, etc.) when operating as a stand-alone unit. In a stand-alone mode of operation, the client system stores or has access to the data (e.g., inputs for machine learning models, new aspects (e.g., new outputs, new classes, etc.) for machine learning models, training data, machine learning models and corresponding configurations and parameters, training data, results, etc.), and includes machine learning module116to train, utilize, and maintain machine learning models for various tasks (e.g., classification, etc.) as described below. Interface module120may generate the graphical user (e.g., GUI, etc.) or other interface (e.g., command line prompts, menu screens, etc.) to solicit information from a corresponding user (e.g., inputs for machine learning models, new aspects (e.g., new outputs, new classes, etc.) for machine learning models, training data, etc.), and provide reports including results (e.g., classification, outputs from the machine learning models, etc.).

Machine learning module116and interface module120may include one or more modules or units to perform the various functions of present invention embodiments described below. The various modules (e.g., machine learning module116, interface module120, etc.) may be implemented by any combination of any quantity of software and/or hardware modules or units, and may reside within memory135of the server and/or client systems for execution by processor115.

Referring now toFIG.2, a schematic of an example of a computing device210of computing environment100(e.g., implementing server system110, client system114, database system118, and/or other computing devices) is shown. The computing device is only one example of a suitable computing device for computing environment100and is not intended to suggest any limitation as to the scope of use or functionality of embodiments of the invention described herein. Regardless, computing device210is capable of being implemented and/or performing any of the functionality set forth herein.

Computer system212may be described in the general context of computer system executable instructions, such as program modules, being executed by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, and so on that perform particular tasks or implement particular abstract data types.

As shown inFIG.2, computer system212is shown in the form of a general-purpose computing device. The components of computer system212may include, but are not limited to, one or more processors or processing units115, a system memory135, and a bus218that couples various system components including system memory135to processor115.

Computer system212typically includes a variety of computer system readable media. Such media may be any available media that is accessible by computer system212, and it includes both volatile and non-volatile media, removable and non-removable media.

Program/utility240, having a set (at least one) of program modules242(e.g., machine learning module116, interface module120, etc.) may be stored in memory135by way of example, and not limitation, as well as an operating system, one or more application programs, other program modules, and program data. Each of the operating system, one or more application programs, other program modules, and program data or some combination thereof, may include an implementation of a networking environment. Program modules242generally carry out the functions and/or methodologies of embodiments of the invention as described herein.

Computer system212may also communicate with one or more external devices214such as a keyboard, a pointing device, a display224, etc.; one or more devices that enable a user to interact with computer system212; and/or any devices (e.g., network card, modem, etc.) that enable computer system212to communicate with one or more other computing devices. Such communication can occur via Input/Output (I/O) interfaces222. Still yet, computer system212can communicate with one or more networks such as a local area network (LAN), a general wide area network (WAN), and/or a public network (e.g., the Internet) via network adapter125. As depicted, network adapter125communicates with the other components of computer system212via bus218. It should be understood that although not shown, other hardware and/or software components could be used in conjunction with computer system212. Examples, include, but are not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

A conventional machine learning classification model may be trained to classify data into a quantity or number of different classes. By way of example, the classification model may be trained to classify the data among one-hundred (100) different classes, where the training set includes ten-thousand (10,000) data items for each class (or 100 classes×10,000 data items=1 million (1,000,000) total data items). The classification model includes an output layer with one-hundred nodes each corresponding to a class.

When an additional class is to be added for the trained classification model, a new classification model is needed with an output layer having one-hundred and one (101) nodes for the one-hundred and one (101) classes. In addition, the new classification model will need to be trained for the one-hundred and one (101) classes with the initial training set for the one-hundred (100) classes and an additional training set (of 10,0000 data items) for the new class (or 101 classes×10,000 data items=1,010,000 total data items). Accordingly, as the number of classes changes, the classification model will need to be reconfigured and retrained (e.g., with a training set in excess of one million (1,000,000) data items) that consumes significant computing and memory resources.

Accordingly, present invention embodiments use embeddings for incremental machine learning (e.g., for deep learning models, classification models, etc.) when a new aspect (e.g., new output, new class, etc.) is added. The incremental machine learning may be performed for the new aspect without retraining a machine learning model. In the case of the above example, an embedding model of a machine learning system of a present invention embodiment is initially trained for the one-hundred (100) classes and modified to produce embeddings. When a new class is to be added, a training set of the new class (e.g., 10,000 data items) is applied to the embedding model to produce embeddings that define the new class and enable a prediction model of the machine learning system to predict the new class as an output. Thus, the new class is added without reconfiguring and retraining the embedding model with the training set for one-hundred and one (101) classes, thereby conserving computing and memory resources and significantly increasing the speed of machine learning. In addition, present invention embodiments introduce flexibility and scalability with respect to handling frequently changing classes.

A machine learning system according to an embodiment of the present invention is illustrated inFIG.3. In particular, a machine learning system300receives input data and produces an output indicating one or more from among a plurality of candidate outputs. By way of example, the machine learning system may be a classifier that classifies input data pertaining to an object (e.g., image, text, etc.) into one or more classes, groups, or categories.

Machine learning system300includes an embedding model310and a prediction model340. Embedding model310receives the input data to the machine learning system and produces an embedding for each data item of the input data. The prediction model applies pattern recognition to the embedding from embedding model310to produce as output a predicted candidate output (e.g., class, etc.) for the data item corresponding to the embedding.

Embedding model310includes a machine learning model320to produce the embeddings. Basically, a data item may be represented by an embedding that includes a vector having numeric elements corresponding to a plurality of dimensions. Data items with similar features or characteristics have similar embeddings or vector representations. By way of example, machine learning model320may be implemented by a neural network to produce the emdeddings. The neural network includes an input layer322, intermediate or hidden layers324,326, and an output layer328. Each layer includes one or more neurons or nodes329, where the input layer neurons receive input data (e.g., data pertaining to objects), and may be associated with weight values. The neurons of the intermediate and output layers are connected to one or more neurons of a preceding layer, and receive as input the output of a connected neuron of the preceding layer. Each connection is associated with a weight value, and each neuron produces an output based on a weighted combination of the inputs to that neuron. The output of a neuron may further be based on a bias value for certain types of neural networks (e.g., recurrent types of neural networks, etc.).

The weight (and bias) values may be adjusted based on various training techniques. For example, the training may be performed with data items of the training set as input and corresponding known results as outputs, where the neural network attempts to produce the known results and uses an error from the output (e.g., difference between produced and known outputs) to adjust weight (and bias) values (e.g., via backpropagation or other training techniques). The output layer of the neural network preferably includes a neuron329for each candidate output (e.g., class, etc.) and indicates the candidate output for corresponding input data. By way of example, the output layer neurons may indicate a specific candidate output (e.g., class, etc.) or an identifier of the specific candidate output. Further, output layer neurons may indicate a probability of the associated candidate output (e.g., class, etc.) being the result for the input data. The candidate output (e.g., class, etc.) associated with the highest probability is preferably selected for the input data. However, machine learning model320may include any quantity of any type of machine learning models (e.g., feed-forward, recurrent, convolutional or other neural networks, etc.). Further, the neural network may include any quantity of layers having any quantity of neurons or nodes.

Neural network320is initially trained on a training set for an initial set of candidate outputs (e.g., classes, etc.). Once trained, output layer328is removed from the neural network (e.g., to form or serve as embedding model310), and the preceding intermediate layer326serves as an embedding layer that produces the embeddings. The elements or dimensions of the embedding correspond to the number of neurons in the embedding layer (e.g., each neuron of the embedding layer provides a value for an element or dimension of the embedding). However, the embeddings may include any quantity of elements or dimensions pertaining to any desired features (e.g., image features, textual features, etc.).

When a new candidate output (e.g., class, etc.) is to be added, embedding model310is provided with training data350for the new candidate output to generate embeddings representing and defining the new candidate output that enable prediction model340to predict the new candidate output for corresponding input data. Thus, in order to add a new candidate output (e.g., class, etc.), a training set for the new candidate output is applied to the embedding model without retraining of the embedding model, thereby conserving computing and memory resources and significantly increasing the speed of machine learning.

FIG.4illustrates an example of data items within a space defined by dimensions of embeddings. Data items405are grouped or clustered within the embedding space, thereby forming clusters corresponding to candidate outputs (e.g., classes, etc.). For example, cluster410is associated with Class 1, cluster420is associated with Class 2, cluster430is associated with Class 3, and cluster440is associated with Class 4 (e.g., as viewed inFIG.4). A data item405within a cluster is assigned to the class associated with that cluster (e.g., a data item in cluster410is assigned to Class 1, a data item in cluster420is assigned to Class 2, a data item in cluster430is assigned to Class 3, and a data item in cluster440is assigned to Class 4). These classes may represent an initial set of classes with which machine learning model320(with output layer328) is trained to provide the embeddings.

When the embedding model is used with the training data for the new class, the resulting embeddings form a new cluster450associated with the new class (e.g., Class 5 as viewed inFIG.4). Clusters410,420,430,440, and450are used by prediction model340to predict the candidate output (e.g., class, etc.) for a data item.

Referring back toFIG.3, the embedding produced from embedding model310for a data item is provided to prediction model340to produce a predicted candidate output (e.g., class, etc.) for the data item. The prediction model uses pattern recognition techniques (e.g., k nearest neighbor (kNN), etc.) to predict the candidate output (e.g., class, etc.).

FIG.5illustrates an example of prediction model340predicting a candidate output (e.g., class, etc.) for a data item based on an embedding of the data item and clusters410,420,430,440, and450. By way of example, a data item460(e.g., represented as an “X” inFIG.5) is shown in the embedding space among clusters410,420,430,440, and450based on an embedding of the data item produced by embedding model310. A pattern recognition technique is used for predicting the appropriate cluster (and, hence, the candidate output (e.g., class, etc.)). For example, a k nearest neighbor (kNN) technique may be applied to determine the nearest neighbors of data item460in the embedding space. This technique may determine the distance between data item460and the other points in clusters410,420,430,440, and450, and identify k nearest neighbors (or k closest points) which reside in the embedding space based on the determined distances to predict the candidate output (e.g., class, etc.) for data item460. The predicted candidate output (e.g., class, etc.) may be the cluster having a majority of the nearest neighbors. The distance may be determined based on various techniques (e.g., Euclidean distance, Manhattan distance, Hamming distance, etc.).

In the case of using three nearest neighbors (k=3), the three nearest neighbors of data item460may be a first data item in cluster420(Class 2), a second data item in cluster420(Class 2), and a third data item in cluster430(Class 3). The predicted candidate output (e.g., class, etc.) may be determined from the cluster having a majority of the nearest neighbors. In this case, since a majority of the nearest neighbors are in cluster420(e.g., 2 out of the 3 nearest neighbor data items), the predicted candidate output (e.g., class, etc.) is the class associated with cluster420(or Class 2). The k nearest neighbor technique may be applied with any desired number of nearest neighbors (e.g., k may be any suitable values), where any suitable techniques may be used to determine a resulting cluster (e.g., majority, distances to nearest neighbors, etc.).

A manner of learning a new aspect by incremental machine learning (e.g., via machine learning module116and server system110and/or client system114) according to an embodiment of the present invention is illustrated inFIG.6. In particular, embedding model310(FIG.3) is produced using a training set for an initial set of candidate outputs (e.g., classes, categories, results, etc.) at operation610. The set of training data includes a data item and a corresponding known result (e.g., a data item and a known class to which the data item belongs, etc.). The embedding model includes a machine learning model preferably derived from a neural network including input layer322, intermediate or hidden layers324,326, and an output layer328(e.g. as described above forFIG.3). The output layer of the neural network preferably includes a neuron329for each candidate output, and indicates the candidate output for corresponding input data. The neural network (including output layer328) is initially trained on the training set for the initial set of candidate outputs to produce a corresponding candidate output from output layer328. Once trained, output layer328is removed from the neural network, where the remaining portions of the neural network (e.g., input layer322and intermediate layers324,326) form the embedding model with intermediate layer326serving as an embedding layer for producing the embeddings. The elements or dimensions of the embedding correspond to the number of neurons in intermediate layer326(e.g., each neuron of the embedding layer provides a value for an element or dimension of the embedding). However, the embedding may include any quantity of dimensions, where any intermediate or other layer of the neural network may provide the embedding.

Once trained embedding model310is produced, the set of training data is applied to the embedding model to produce embeddings for data items of the set of training data at operation620. Each embedding of a data item of the training set is associated (or labeled) with the corresponding known candidate output and stored (e.g., in database system118) at operation630. The stored embeddings define the corresponding candidate outputs (e.g., classes, etc.) for prediction model340. However, the same or a different training set may be used to produce the embeddings defining the candidate outputs.

When a new candidate output (e.g., class, category, result, etc.) is to be added as determined at operation635, embedding model310(FIG.3) is provided with a set of training data for the new candidate output to generate embeddings representing the new candidate output at operation640. The set of training data for the new candidate output includes data items corresponding to the new candidate output (e.g., a data item and the new candidate output, etc.) Each embedding of a data item of the training set for the new candidate output is associated (or labeled) with the new candidate output and stored (e.g., in database system118). Thus, in order to add a new candidate output, incremental machine learning is performed where embedding model310produces embeddings defining the new candidate output from a training set for the new candidate output. This avoids the embedding model being retrained (on a large training set encompassing the new and previous candidate outputs), thereby conserving computing and memory resources and significantly increasing the speed of machine learning when a new candidate output is added.

A training score is determined based on the embeddings produced by embedding model310from the training set for the new candidate output at operation642to determine sufficiency of the incremental machine learning. The training score may be derived from a measure of quality of a cluster formed by the embeddings for the new candidate output. By way of example, a Silhouette method that measures similarity of an object to its own cluster relative to other clusters may be used to determine a score for an embedding for the new candidate output based on the following expression:

where i represents an embedding for the new candidate output, a(i) is an intra-cluster distance for the embedding relative to other embeddings in the cluster for the new candidate output, b(i) is an inter-cluster distance between the embedding in the cluster for the new candidate output and the embeddings in clusters for other candidate outputs, and max is a maximum function (e.g., providing the greater value of a(i) and b(i)).

For example, an inter-cluster distance, a(i), may be the average distance between an embedding and other embeddings in the cluster for the new candidate output. An inter-cluster distance, b(i), may be the minimum of the average distances for clusters for other candidate outputs. The average distance for a cluster for another candidate output is the average of distances between the embedding in the cluster for the new candidate output and each embedding in the cluster for the other candidate output. The distances may be determined based on various techniques (e.g., Euclidean distance, Manhattan distance, Hamming distance, etc.). The training score may be a combination of the scores for the embeddings for the new candidate output, such as an average of the scores for the embeddings for the new candidate output.

By way of further example, an embedding space may include: Class 1 with points 1A and 1B; Class 2 with point 2A, and a New Class with points 3A, 3B, and 3C. The distances between the points within the New Class may include: 2 between points 3A and 3B; and 4 between points 3A and 3C. In this case, the intra-cluster distance, a(i), for point 3A would be the average of the distances to the other points (3B and 3C) within the New Class or ((Distance 3A to 3B)+(Distance 3A to 3C))/2=(2+4)/2=3. The distances between point 3A and points in the other clusters may be: 12 between point 3A and point 1A of Class 1; 10 between point 3A and point 1B of Class 1; and 12 between point 3A and point 2A of Class 2. The inter-cluster distance, b(i), for point 3A would be the minimum of the average distance for each cluster between point 3A and the points in that cluster, or min ((12+10)/2=11 for Class 1, 12/1=12 for Class 2)=11. The score for point 3A would be (b(i)−a(i))/max {a(i), b(i)}=(11−3)/11=0.72. The training score for the New Class would be the average of the scores for points 3A, 3B, 3C, or (score 3A+score 3B+score 3C)/3.

When the training score does not exceed a threshold as determined at operation645, the embeddings for the new candidate output are insufficient to enable the candidate output to be added without reconfiguring and retraining of the embedding model. In other words, the embeddings do not sufficiently distinguish the new candidate output from the other candidate outputs. Accordingly, embedding model310is produced based on a training set including the training set for the previous candidate outputs and the training set for the new candidate output at operation610in substantially the same manner described above (e.g., starting with a machine learning model320(or neural network) with an output layer having a neuron corresponding to the new candidate output).

When the training score exceeds the threshold, the embeddings for the new candidate output are sufficient to enable the candidate output to be added without reconfiguring and retraining of the embedding model. In other words, the embeddings sufficiently distinguish the new candidate output from the other candidate outputs. The new candidate output is added to the machine learning system (via incremental machine learning using the embeddings) without retraining of the embedding model, thereby improving the speed of the machine learning and conserving computing and memory resources. Accordingly, when the incremental machine learning is sufficient as determined at operation645, or an absence of a new candidate output is determined at operation635, data to be processed is provided to embedding model310at operation650to generate embeddings for the data.

The embedding produced from embedding model310for a data item is provided to prediction model340to produce a predicted candidate output as a result for the data item at operation660based on clusters of embeddings of data items of the training sets for the candidate outputs formed in the embedding space. The prediction model uses a pattern recognition technique (e.g., k nearest neighbor (kNN), etc.) to predict the candidate output. For example, a k nearest neighbor (kNN) technique may be applied to determine the nearest neighbors of a data item in the embedding space. This technique may determine the distance between the data item and the points in the clusters (representing the candidate outputs including any new candidate outputs), and identify k nearest neighbors (or k closest points) which reside in the embedding space based on the determined distances. The nearest neighbors are used to predict the candidate output (e.g., class, category, result, etc.) as a result for the data item. The predicted candidate output may be the cluster having a majority of the nearest neighbors. The distance may be determined based on various techniques (e.g., Euclidean distance, Manhattan distance, Hamming distance, etc.). The k nearest neighbor technique may be applied with any desired number of nearest neighbors (e.g., k may be any suitable values), where any suitable techniques may be used to determine a resulting cluster (e.g., majority, distances to nearest neighbors, etc.). Additional candidate outputs may be added in substantially the same manner described above.

An example of operation of the machine learning system according to an embodiment of the present invention is illustrated inFIGS.7and8. Initially, machine learning system300is implemented as a classifier to classify images into classes of categories of cat710and dog720. The machine learning system includes embedding model310and prediction model340as described above. Embedding model310is trained using a training set for the initial set of classes (e.g., cat710and dog720). The set of training data includes a data item (e.g., image, etc.) and a corresponding known result (e.g., a data item and a known class (cat or dog) to which the data item belongs, etc.). The embedding model includes a machine learning model320preferably derived from a neural network including input layer322, intermediate or hidden layers324,326, and an output layer328(e.g., as described above forFIG.3). The output layer of the neural network preferably includes a neuron329for each class, and indicates the candidate output for corresponding input data. The neural network (including output layer328) is initially trained on the training set for the initial set of classes to produce a corresponding candidate output from output layer328. Once trained, output layer328is removed from the neural network, where the remaining portions of the neural network (e.g., input layer322and intermediate layers324,326) form the embedding model with intermediate layer326serving as an embedding layer for producing the embeddings.

Once trained embedding model310is produced, the set of training data is applied to the embedding model to produce embeddings for data items of the set of training data (e.g., Cat 1 [0.01, 0.00, 0.23, . . . 0.20], Cat N [0.00, 0.02, 0.21, . . . 0.22], Dog 1 [0.02, 0.52, 0.13, . . . 0.38], Dog N [0.02, 0.50, 0.11, . . . 0.40]; for N data items in each class of cat and dog). Each embedding of a data item of the training set is associated (or labeled) with the corresponding known class and stored (e.g., in database system118).

When a new class, rabbit730, is to be added, embedding model310is provided with a set of training data for the new rabbit class to generate embeddings representing the new class. The set of training data for the new class includes data items (e.g., images, etc.) corresponding to the new class (e.g., a data item and the new class, etc.). Each embedding of a data item of the training set for the new class is associated (or labeled) with the new class and stored (e.g., in database system118). Thus, in order to add a new class, embedding model310generates embeddings defining the new class for the prediction model rather than being reconfigured and retrained for the cat, dog, and rabbit classes, thereby conserving computing and memory resources and significantly increasing the speed of machine learning.

Example data items for the cat, dog, and rabbit classes are shown within a space defined by dimensions of embeddings (FIG.8). The data items are grouped or clustered within the embedding space, thereby forming clusters corresponding to the cat, dog, and rabbit classes (e.g., cluster810is associated with cat class710, cluster820is associated with dog class720, and cluster830is associated with rabbit class730. A data item within a cluster is assigned to the class associated with that cluster (e.g. a data item in cluster810is assigned to cat class710, a data item in cluster820is assigned to dog class720, and a data item in cluster830is assigned to rabbit class730).

An image840of a cat may be provided to embedding model310to generate an embedding for the image. The embedding produced from embedding model310for the image is provided to prediction model340to produce a predicted class for the image based on clusters810,820, and830of embeddings of data items of the training sets for the cat, dog, and rabbit classes formed in the embedding space. The prediction model may use a k nearest neighbor (kNN) technique to determine the nearest neighbors of the cat image in the embedding space. This technique may determine the k nearest neighbors which reside in the embedding space to predict the class. The predicted class may be determined from the cluster having a majority of the nearest neighbors.

For example, image840is represented by a data item850(e.g., shown by an “X” inFIG.8) in the embedding space among clusters810,820, and830based on an embedding of the image produced by embedding model310. The k nearest neighbor (kNN) technique may be applied to determine the nearest neighbors of image840in the embedding space. In the case of using three nearest neighbors (k=3), the three nearest neighbors of image840may be a first data item in cluster810(cat class710), a second data item in cluster810(cat class710), and a third data item in cluster830(rabbit class730). The predicted class may be determined from the cluster having a majority of the nearest neighbors. In this case, since a majority of the nearest neighbors are in cluster810(e.g., 2 out of the 3 nearest neighbor data items), the predicted class is the cat class710associated with cluster810.

It will be appreciated that the embodiments described above and illustrated in the drawings represent only a few of the many ways of implementing embodiments for incremental machine learning using embeddings.

A report may include any information arranged in any fashion, and may be configurable based on rules or other criteria to provide desired information to a user (e.g., machine learning models, aspects (e.g., types of outputs, classes, etc.), classification, outputs from the machine learning models, etc.).

The embedding model may be derived from any machine learning model (e.g., neural networks, statistical models, etc.). The neural network may be of any type (e.g., feed-forward, recurrent, convolutional, etc.), include any quantity of layers, and any quantity of neurons or nodes in each layer. Any layer of the neural network may serve as the embedding layer producing the embeddings, where the embedding model includes the neural network or any portion of the neural network (e.g., the embedding layer and layers prior to the embedding layer, etc.). The embeddings may be represented by any type of structure (e.g., vector, array, etc.), and may include any quantity of elements or dimensions with any numeric or other values (e.g., alphanumeric, etc.) pertaining to any desired features (e.g., image features, textual features, etc.). Any quantity of embeddings may be generated for a data item (e.g., each embedding is associated with corresponding one or more features of the data item, etc.). The training sets for candidate outputs (including new candidate outputs) may include any quantity of data items of any type (e.g., image, text, video, audio, etc.), and may further include any indications of known results (e.g., known output, known class, known result, etc.). Any conventional or other training techniques may be applied for any quantity of training sets to train or retrain the machine learning model (e.g., backpropagation, etc.).

The prediction model may employ any techniques to predict a candidate output (e.g., machine learning, pattern recognition, kNN, etc.). Any quantity of nearest neighbors may be used for the prediction based on any suitable distances or other similarity measurements (e.g., cosine similarity, etc.). The distance may be determined based on various conventional or other techniques (e.g., Euclidean distance, Manhattan distance, Hamming distance, etc.). The candidate output may be selected based on the nearest neighbors in any fashion (e.g., candidate output associated with a majority of the nearest neighbors, candidate output associated with closest nearest neighbor, distances of nearest neighbors to other clusters, etc.).

The training score may be determined based on any cluster quality metrics and/or distances or other similarity measures (e.g., inter-cluster distance, intra-cluster distance, average distances, cosine similarity, etc.). The distance may be determined based on various conventional or other techniques (e.g., Euclidean distance, Manhattan distance, Hamming distance, etc.). The score for individual embeddings may be combined in any fashion to determine the training score (e.g., average, median, standard deviation, etc.). Further, the embeddings for the new candidate output may combined in any fashion (e.g., average, median, maximum or minimum values for certain features or elements, etc.) to form a representative embedding used to determine the training score. The threshold may be set to any suitable value in any value range to indicate the cluster for the new candidate output is sufficiently distinguished from clusters of other candidate outputs. For example, the inter-cluster distance should be large to distinguish from other clusters, while the intra-cluster distance should be small to provide a cohesive cluster. For the above expression for the training score, a score for an individual embedding towards or near 1.0 indicates a sufficiently distinguishable cluster of embeddings (e.g., for a large inter-cluster distance b(i) and a small intra-cluster distance a(i), the score approaches 1.0). Thus, an example threshold for the training score (e.g., based on the expression above for an average or median of individual embedding scores or a score for the representative embedding, etc.)) may be any value in a range from 0.7 to 1.0 (or similar values for other value ranges based on the manner the individual embedding scores are combined or normalized). However, any suitable value or value range may be used for the threshold.

The present invention embodiments are not limited to the specific tasks or algorithms described above, but may be utilized for learning any types of new aspects (e.g., new outputs, new classes, new groups, new categories, new value ranges, new selections from any group, etc.) for various machine learning systems (e.g., deep learning, classification, pattern matching/recognition, image/video/vision analysis, natural language/text analysis, audio analysis, etc.).