Generative adversarial network with dynamic capacity expansion for continual learning

A method may include training a machine learning model to perform a first task before training the machine learning model to perform the second task. The machine learning model includes a generator network and a discriminator network. The training includes training, based on a first training sample associated with the first task, the discriminator network to perform the first task. The generator network may be trained to generate a first synthetic training sample emulating the first training sample. The discriminator network trained to perform the first task may be reinitialized in order for the discriminator network to be trained, based on a second training sample, to perform the second task. The reinitialized discriminator network may be further retrained, based on the first synthetic training sample, to perform the first task. Related systems and articles of manufacture, including computer program products, are also provided.

TECHNICAL FIELD

The subject matter described herein relates generally to machine learning and more specifically to the training of a machine learning model.

BACKGROUND

Machine learning models may be trained to perform a variety of cognitive tasks including, for example, object identification, natural language processing, information retrieval, speech recognition, and/or the like. A deep learning model such as, for example, a neural network, may be trained to perform a classification task by at least assigning input samples to one or more categories. The deep learning model may be trained to perform the classification task based on training data that has been labeled in accordance with the known category membership of each sample included in the training data. Alternatively and/or additionally, the deep learning model may be trained to perform a regression task. The regression task may require the deep learning model to predict, based at least on variations in one or more independent variables, corresponding changes in one or more dependent variables.

SUMMARY

Systems, methods, and articles of manufacture, including computer program products, are provided for continual learning. In one aspect, there is provided a system. The system may include at least one data processor and at least one memory. The at least one memory may store instructions that result in operations when executed by the at least one data processor. The operations may include: training a machine learning model to perform a first task and a second task, the machine learning being trained to perform the first task before the machine learning model is trained to perform the second task, the machine learning model including a generator network and a discriminator network, and the training of the machine learning model including training, based at least on a first training sample associated with the first task, the discriminator network to perform the first task, training the generator network to generate a first synthetic training sample emulating the first training sample associated with the first task, and reinitializing the discriminator network trained to perform the first task, the discriminator network being reinitialized in order for the discriminator network to be trained, based at least on a second training sample, to perform the second task, the reinitialized discriminator network further being retrained, based at least on the first synthetic training sample, to perform the first task; and deploying the trained machine learning model to perform the first task and the second task.

In some variations, one or more of the features disclosed herein including the following features can optionally be included in any feasible combination. The discriminator network may be further trained to differentiate between the first synthetic training sample and the first training sample. The generator network may be trained, based at least on an output of the discriminator network, to generate the first synthetic training sample such that the discriminator network is unable to differentiate between the first synthetic training sample and the first training sample.

In some variations, the generator network may be further trained to generate a second synthetic training sample emulating the second training sample associated with the second task. The generator network may be configured to generate a binary mask identifying a first neuron in the generator network as being reserved for the first task. The training of the generator network to generate the second synthetic training sample may include changing a state of a second neuron in the generator network in response to the binary mask identifying the first neuron in the generator network as being reserved for the first task.

In some variations, the state of the second neuron may be changed further in response to the binary mask identifying the second neuron as a free neuron.

In some variations, a capacity of the generator network may be expanded by at least adding the second neuron in response to the binary mask indicating that no free neurons are available in the generator network.

In some variations, changing the state of the second neuron may include adjusting one or more weights applied by the second neuron in order to minimize a difference in between the second synthetic training sample generated by the generator network and the second training sample associated with the second task.

In some variations, the training of the generator network to generate the second synthetic training sample may include reusing the first neuron without changing a state of the first neuron.

In some variations, the generator network may generate the first synthetic training sample and the second synthetic training sample without storing the first training sample associated with the first task or the second training sample associated with the second task.

In some variations, the first task and the second task may be classification tasks.

In some variations, the machine learning model may be a generative adversarial network having the generator network and the discriminator network.

In another aspect, there is provided a method for continual learning. The method may include: training a machine learning model to perform a first task and a second task, the machine learning being trained to perform the first task before the machine learning model is trained to perform the second task, the machine learning model including a generator network and a discriminator network, and the training of the machine learning model including training, based at least on a first training sample associated with the first task, the discriminator network to perform the first task, training the generator network to generate a first synthetic training sample emulating the first training sample associated with the first task, and reinitializing the discriminator network trained to perform the first task, the discriminator network being reinitialized in order for the discriminator network to be trained, based at least on a second training sample, to perform the second task, the reinitialized discriminator network further being retrained, based at least on the first synthetic training sample, to perform the first task; and deploying the trained machine learning model to perform the first task and the second task.

In some variations, one or more of the features disclosed herein including the following features can optionally be included in any feasible combination. The discriminator network may be further trained to differentiate between the first synthetic training sample and the first training sample. The generator network may be trained, based at least on an output of the discriminator network, to generate the first synthetic training sample such that the discriminator network is unable to differentiate between the first synthetic training sample and the first training sample.

In some variations, the generator network may be further trained to generate a second synthetic training sample emulating the second training sample associated with the second task. The generator network may be configured to generate a binary mask identifying a first neuron in the generator network as being reserved for the first task. The training of the generator network to generate the second synthetic training sample may include changing a state of a second neuron in the generator network in response to the binary mask identifying the first neuron in the generator network as being reserved for the first task.

In some variations, the state of the second neuron may be changed further in response to the binary mask identifying the second neuron as a free neuron.

In some variations, the method may further include expanding a capacity of the generator network by at least adding the second neuron in response to the binary mask indicating that no free neurons are available in the generator network.

In some variations, changing the state of the second neuron may include adjusting one or more weights applied by the second neuron in order to minimize a difference in between the second synthetic training sample generated by the generator network and the second training sample associated with the second task.

In another aspect, there is provided a computer program product that includes a non-transitory computer readable storage medium. The non-transitory computer-readable storage medium may include program code that causes operations when executed by at least one data processor. The operations may include: training a machine learning model to perform a first task and a second task, the machine learning being trained to perform the first task before the machine learning model is trained to perform the second task, the machine learning model including a generator network and a discriminator network, and the training of the machine learning model including training, based at least on a first training sample associated with the first task, the discriminator network to perform the first task, training the generator network to generate a first synthetic training sample emulating the first training sample associated with the first task, and reinitializing the discriminator network trained to perform the first task, the discriminator network being reinitialized in order for the discriminator network to be trained, based at least on a second training sample, to perform the second task, the reinitialized discriminator network further being retrained, based at least on the first synthetic training sample, to perform the first task; and deploying the trained machine learning model to perform the first task and the second task.

DETAILED DESCRIPTION

A machine learning model may be trained to perform a task by being exposed to a corresponding corpus of labeled training samples, each of which including data (e.g., text, image, and/or the like) and at least one ground-truth label corresponding to a correct label for the data. Training the machine learning model may include adjusting the machine learning model to minimize the errors present in the output of the machine learning model. For example, training the machine learning model may include adjusting the weights applied by the machine learning model in order to minimize a quantity of incorrect labels assigned by the machine learning model. Nevertheless, although a conventional machine learning model may be successfully trained to perform a first task, it may be unable to learn from additional training samples associated with a second task while maintaining the parameters learned for the first task. For instance, subjecting the machine learning model to additional training, including by exposing the machine learning model to training samples associated with the second task, may diminish the performance of the machine learning model for the first task and/or the second task.

In some example embodiments, a machine learning model may be implemented as a generative adversarial network having a generator network and a discriminator network operating in tandem to ensure that the machine learning model is able to learn incrementally to perform multiple tasks. Moreover, the capacity of the generator network may expand, for example, through the addition of neurons, to accommodate knowledge that is acquired with the learning of additional tasks. By including the generative adversarial network having an adaptive network capacity, the machine learning model may avoid the phenomenon of catastrophic forgetting, which may occur when a conventional machine learning model is trained to perform multiple tasks.

A conventional machine learning model may be incapable of learning continuously to perform multiple tasks because the neurons of the conventional machine learning model lack plasticity. Instead of locking to a state to retain the parameters learned for a first task, the state of the neurons in a conventional machine learning model may vary when the machine learning model is exposed to training samples associated with a second task. Moreover, a conventional machine learning model may have a fixed capacity. Thus, then the quantity of available neurons becomes depleted, the state of neurons already configured for one task may be altered in order to accommodate additional tasks. This lack of plasticity and scalability may result in catastrophic forgetting, in which the exposure to training samples associated with a subsequent task supplants the knowledge that the machine learning model had already acquired for one or more previously learned task. In some example embodiments, the inclusion of the generative adversarial network having an adaptive network capacity may lend plasticity as well as scalability to the machine learning model such that the machine learning model is able to learn continuously to perform multiple tasks.

As noted, the generative adversarial network included in the machine learning model may include a generator network and a discriminator network. In some example embodiments, the generator network may be trained to generate synthetic training samples that emulate the actual training samples used to train the discriminator network to perform the first task. Meanwhile, the discriminator network may be trained based on the synthetic training samples generated by the generator network and training samples associated with a second task such that the machine learning model is able to learn the second task without forgetting the first task. For example, the discriminator network may be trained to differentiate between a synthetic training sample associated with the first task and an actual training sample associated with the second task. Subsequent to being trained to perform the first task, the discriminator network may be reinitialized before being trained, based on training samples associated with the second task, to perform the second task. In addition to being trained to perform the second task, the discriminator network may be retrained, based on synthetic training samples associated with the first task, to perform the first task.

In some example embodiments, as part of generating synthetic training samples for training the discriminator network, the generator network may be configured to memorize a data distribution associated with the first task without storing the actual training samples associated with the first task. For example, the generator network may generate a binary mask identifying parameters of the generator network that may not be modified when the generator network encounters additional training samples associated with the second task. The binary mask may lend plasticity to the generator network. For instance, although a neuron in the generator network configured for the first task may be reused for the second task, the binary mask may prevent the state of that neuron from being changed when the generator network encounters training samples associated with the second task. Instead, if the training associated with the second task requires changing the state of the neuron, these changes may be applied to a different neuron in the generator network that the binary mask indicates as being free.

Furthermore, in some example embodiments, the capacity of the generator network may expand in order to accommodate knowledge that is acquired with the learning of additional tasks. For example, the capacity of the generator network may expand to include additional neurons if the binary mask indicates that no existing neurons may be modified based on the training samples associated with the second task. As such, the state of the neurons configured for the first task may remain fixed while changes corresponding to the second task may be applied to the additional neurons added to the generator network.

FIG.1Adepicts a system diagram illustrating an example of a machine learning system100, in accordance with some example embodiments. Referring toFIG.1A, the machine learning system100may include a machine learning controller110, a machine learning application120, and a client130. The machine learning controller110, the machine learning application120, and the client130may be communicatively coupled via a network140. It should be appreciated that the client130may be a processor-based device including, for example, a smartphone, a tablet computer, a wearable apparatus, a virtual assistant, an Internet-of-Things (IoT) appliance, and/or the like. The network140may be any wired network and/or a wireless network including, for example, a wide area network (WAN), a local area network (LAN), a virtual local area network (VLAN), a public land mobile network (PLMN), the Internet, and/or the like.

The machine learning controller110may incrementally train a machine learning model150to perform multiple tasks. For example, the machine learning controller110may train the machine learning model150to perform a first task before training the machine learning model150to perform a second task. Examples of the first task and the second task may include object identification tasks, natural language processing tasks, information retrieval tasks, speech recognition tasks, and/or the like. As used here, the machine learning model150may be trained to perform an individual “task” by being exposed to a corresponding corpus of training samples. For instance, the machine learning model150may be trained to perform the first task based on a first corpus of training samples and the second task based on a second corpus of training samples. When the machine learning model150is trained “incrementally” to perform the first task and the second task, the first corpus of training samples is not available when the machine learning model150trained to perform the first task is subsequently being trained to perform the second task. The trained machine learning model150may be deployed to perform the first task and the second task in order to implement one or more functionalities of the machine learning application120.

For instance, the machine learning application120may be a machine learning based communication application such as, for example, a chatbot, an issue tracking system, and/or the like. As such, the trained machine learning model150may be deployed to perform various natural language processing tasks that includes determining a sentiment, a topic, and/or an intent of text received at the machine learning application120. The result of the natural language processing tasks may enable the machine learning application120to further determine an appropriate response to the text received at the machine learning application120.

In some example embodiments, the machine learning model150may be a generative adversarial network with adaptive network capacity. The plasticity and scalability of the machine learning model150may enable the machine learning model150to learn incrementally, for example, to perform the second task after the first task without forgetting the first task. To further illustrate,FIG.1Bdepicts a block diagram illustrating an example of the machine learning model150, in accordance with some example embodiments. As shown inFIG.1B, the machine learning model150may be a generative adversarial network that includes a generator network GθGand a discriminator network DθD.

In some example embodiments, training the machine learning model150may include jointly training the generator network GθGand the discriminator network DθDincrementally to perform, for example, the first task before the second task. For example, the training samples associated with an individual task k∈K may be denoted as Dk={(xik,yik)}i=1nk, wherein xik∈X may denote a single training sample and yik∈ykmay denote the ground truth labels associated with the training sample. In order to train the machine learning model150to perform each of the K quantity of tasks incrementally, the training samples Dkassociated with each individual task may become available to the machine learning model150continuously and progressively instead of all at once. For instance, while the machine learning model150is being trained to perform a first task k1, the training samples Dkmay include training samples associated with the first task k1and/or originating from a single class. That is, the ground truth labels yik∈ykof the training samples Dkmay exclude labels belonging to other tasks such as, for example, a second task k2. Nevertheless, when deployed for testing and/or for production, the output space Yk=∪j=1kyjof the machine learning model150may span all off the labels encountered by the machine learning model150, including labels belonging to multiple tasks such as the first task k1and the second task k2.

In order to learn each individual task k∈K incrementally, the machine learning model150may, at each time t, learn the parameters θ of a predictive model fθ: X→Yt,

wherein Yt=∪j=1ty3may denote all of the labels encountered by the machine learning model150thus far. Learning the parameters θ of the predictive model fθ: X→Ytmay require solving the problem expressed as Equation (1) below.

minθ⁢ℒ⁡(θ,Dt)+Ω⁡(θ)(1)
wherein t may denote a task index,may denote the loss function, and Ω(θ) may correspond to a regularization term.

If the training samples Dt={(xk,yt)} of a task t includes only a single class of labels (e.g., |{y}|=1), then the loss function cannot be computed without referring to classes from training samples already encountered by the machine learning model150. Nevertheless, storing any training samples may violate a strictly incremental training setup. Accordingly, in some example embodiments, the generator network GθGmay be configured to generate synthetic training samples that emulate the actual training samples already encountered by the discriminator network DθD, for example, while the discriminator network DθDwas trained to perform one or more previously learned tasks. Meanwhile, the discriminator network DθDmay be trained to differentiate between synthetic training samples and actual training samples such that the output of the discriminator network DθDmay be used to train the generator network GθGadversarially to generate synthetic training samples that emulate actual training samples. For example, the generator network GθGmay be trained adversarially to generate synthetic training samples that the differentiator network DθDis incapable of differentiating from actual training samples.

According to some example embodiments, the generator network GθGmay be configured to generate a binary mask M identifying parameters in the generator network GθGthat cannot be modified when the generator network GθGtrained to generate synthetic training samples for the first task k1is subsequently trained to generate synthetic training samples for the second task k2. The binary mask M may lend plasticity to the generator network GθGincluding by preventing the state of neurons configured for the first task k1from being changed when the generator network GθGencounters training samples associated with the second task k2. For example, if the training associated with the second task k2requires changing the state of a neuron the binary mask indicates as reserved for the first task k1, these changes may be applied to a different neuron in the generator network GθGthat the binary mask indicates as being free.

Training the generator network GθGto generate synthetic training samples that emulate actual training samples associated with the first task k1may include performing a stochastic gradient descent that includes determining a gradient of an error function (e.g., mean squared error (MSE), cross entropy, and/or the like) associated with the generator network GθG. The gradient of the error function associated with the generator network GθGmay be determined, for example, by backward propagating the error in an output of the generator network GθG, which may correspond to a difference between a synthetic training sample generated by the generator network GθGand an actual training sample associated with the first task k1. Meanwhile, the error in the output of the generator network GθGmay be minimized by at least updating one or more weights applied by the neurons in the generator network GθGuntil the gradient of the error function converges, for example, to a local minimum and/or another threshold value.

The state of a neuron configured for the first task k1may include one or more weights applied to the inputs of the neuron before the inputs are passed through an activation function associated with the neuron. Preserving the state of the neuron may therefore include preventing the weights applied by the neuron from being changed when the generator network GθGis subsequently trained to generate synthetic training samples that emulate actual training samples associated with the second task k2. For example, training the generator network GθGto generate synthetic training samples that emulate actual training samples associated with the second task k2may include minimizing the error in the output of the generator network GθGby adjusting the weights applied by the neurons in the generator network GθG. Instead of adjusting the weights of any neuron indiscriminately, the binary mask M may identify neurons that are reserved for the first task k1such that the state of these neurons remain unchanged by the training associated with the second task k2. As such, weight adjustments necessitated by the training associated with the second task k2may be applied, based on the binary mask M, to free neurons included in the generator network GθG.

In some example embodiments, each stochastic gradient descent step t may include learning the binary mask Mt=[m1t, . . . , mlt] based on the previous layer's activations of the generator network GθG. The binary mask Mtmay subsequently be modulated with the layer activation to yield an output for a fully connected layer l expressed by Equation (2) below.
ylt=mlt⊙(Wl·x)  (2)
wherein mltmay denote an n-element vector and Wlmay denote a weight matrix between layer l and l−1 shaped as an m×n matrix.

Referring again toFIG.1B, the discriminator network DθDmay be trained to differentiate between the synthetic training samples generated by the generator network GθGto emulate training samples associated with previously learned tasks such as the first task k1and actual training samples associated with a current task such as the second task k2. The discriminator network DθDmay be further trained to perform the first task k1and the second task k2including, for example, by assigning labels belonging to the first task k1and the second task k2. In some example embodiments, the discriminator network DθDmay be reinitialized for every task k before being trained for a current task as well as the previously learned tasks. For example, subsequent to being trained to perform the first task k1, the discriminator network DθDmay be reinitialized before being trained, based on training samples associated with the second task k2, to perform the second task k2. In addition to being trained to perform the second task k2, the discriminator network DθDmay be retrained, based on synthetic training samples associated with the first task k1, to perform the first task k1.

In some example embodiments, to generate the binary mask M, the generator network GθGmay apply real valued mask embedding elt, which may be scaled by a positive scaling parameter s, and passed through a threshold function σ(x)∈[0,1]. As such, the binary mask M generated by the generator network GθGmay be given as mlt=σ(selt). A sigmoid function may be applied as a pseudo step-function such in order to ensure a gradient flow to train the mask embedding elt. Meanwhile, the scaling parameters may control the degree of binarization of the mask M. For example, the value of s may be directly proportional to the degree of binarization (e.g., mlt→{0,1} for s→∞, mlt→0.5 for s→0). Over the course of training, the scaling parameters may be incrementally annealed from 0 to smaxin accordance with Equation (3) below.

s=1smax+(smax-1smax)⁢i-1I-1(3)
wherein i may correspond to a current training epoch and I may correspond to the total quantity of training epochs.

In order to avoid overwriting of the knowledge associated with previously learned tasks when the generator network GθGis trained for additional tasks, the gradients glof the weights of each layer l may be multiplied by the inverse of the cumulated binary masks for all previously learned tasks as shown by Equations (4) and (5) below.
ml≤t=max(mlt,mlt-1)  (4)
ml≤t=max(mlt,mlt-1)  (5)
wherein g′lmay correspond to a new weight gradient and ml,m×n≤tmay denote a cumulated mask expanded to the shape of gl(e.g., n times duplication of m≤tto match sizes).

In the cumulated attention mask ml,m×n≤t, the neurons that are important for the previously learned tasks may be masked with a first value (e.g., “1” or a value close to “1”) to indicate these neurons as reserved for the previously learned tasks. Contrastingly, free neurons that can be modified during training for subsequent tasks may be masked with a second value (e.g., “0”). Although neurons reserved for previously learned tasks (e.g., neurons masked with the first value in the cumulated attention mask ml,m×n≤t) may still be reused during training for subsequent tasks, it should be appreciated that the state of the reserved neurons (e.g., the weights applied by the reserved neurons) may not be modified. The higher the sparsity of the cumulated attention mask ml,m×n≤t, the higher the quantity of neurons that may be available for modification during subsequent training.

In some example embodiments, sparsity of the cumulated attention mask ml,m×n≤tmay be promoted by adding a regularization term Rtto the loss function LGof the generator network GθGas shown in Equation (6) below. The regularization term Rtmay be added in order to increase the efficiency in the allocation of neurons across various tasks.

Rt⁡(Mt,Mt-1)=Σl=1L-1⁢Σi=1Ni⁢ml,it⁡(1-ml,i<t)Σl=1L-1⁢Σi=1Ni⁢1-ml,i<t(6)
wherein Ntmay denote the quantity of neurons of the layer l. Neurons reserved for previously learned tasks may not be subject to regularization whereas free neurons may be subject to regularization. Accordingly, the addition of the regularization term Rtto the loss function LGmay increase the efficiency of neuron allocation across different tasks including by promoting the reuse of reserved neurons over the allocation of free neurons.

As noted, in some example embodiments, the generator network GθGand the discriminator network DθDmay be jointly trained such that the generator network GθGand the discriminator network DθDmay operate in tandem to ensure that the machine learning model150is able to learn incrementally to perform multiple tasks including, for example, the first task k1followed by the second task k2. For example, the generator network GθGmay be trained to generate synthetic training samples that emulate the training samples associated with one or more previously learned tasks (e.g., the first task k1). Meanwhile, the discriminator network DθDmay be trained to differentiate between the synthetic training samples and actual training samples such that the output of the discriminator network DθDmay be used to train the generator network GθGadversarially to generate synthetic training samples that the differentiator network DθDis incapable of differentiating from actual training samples. Moreover, the discriminator network DθDmay be trained to perform additional tasks based on actual training samples associated with the additional tasks while being retrained to perform the previously learned tasks based on the synthetic training samples generated by the generator network GθG.

For example, using task labels as conditions, the generator network GθGmay learn from a training set Xt={X1t, . . . , XNt} to generate synthetic training samples for a task t previously learned by the discriminator network DθD. The synthetic training samples generated by the generator network GθGmay be expressed by Equation (7) below.
xt=GθGt(t,z,Mt)  (7)
wherein θGtmay denote the parameters of the generator network GθGconfigured for the task t, z may denote a random noise vector, and Mtmay denote the computed binary mask for the task t.

To train the generator network GθGadversarially, the discriminator DθDmay be trained to perform a discriminative task that includes determining whether a training sample xfis synthetic training sample generated by the generator network GθGor an actual training sample. The discriminator network DθDmay also be trained to perform a classification task in which the discriminator network DθDdetermines whether the training sample xfmay be labelled as part of the task t. To achieve both, the final layer of the discriminator network xfmay include a first branch corresponding to the discriminative task and a second branch corresponding to the classification task. The base network added to each layer may be parameterized with θHtand θCt, respectively. The parameters corresponding to the three tasks, θGt, θHt, θCt, may be optimized in an alternating fashion. Accordingly, the training of the machine learning model150, including the joint training of the generator network GθG, may be an optimization problem that requires minimizing Equation (8) below.
G+st−ct+λRURt(8)
whereincmay denote a classification error on the auxiliary output,smay denote a discriminative loss function used on the binary output layer of the network, and λRURtmay correspond to the regularization term Rtincluded in in Equation (6).

To promote efficient allocation of neuron in the generator network GθGincluding those already reserved for previously learned tasks, the regularization weight λ may be multiplied by the ratio

α=StSfree,
wherein Stmay denote the size of the generator network GθGprior to the training associated with the task t, and Sfreemay denote the quantity of free neurons remaining in the generator network GθG. The ratio α may ensure that less neurons are reused during the beginning stages of training and more neurons are reused during the later stages of training.

For a given task t, its corresponding binary mask Mtmay be initialized with the scaling parameter s=0. As observed in the graph400depicted inFIG.4, at task initialization, the binary mask Mtmay be completely non-binary, with no new neurons set aside for learning a new task. However, as the training of the generator network GθGprogresses for the given task (e.g., over successive training epochs), the scaling parameters and regularization scaling parameter λ may increase and become annealed. With heavy regularization and increasing binarization, the generator network GθGmay be encouraged to minimize the quantity of reserved neurons, as illustrated in region (A-2) of the graph400.

But with most mask values near 0, the ability of the generator network GθGto acquire new knowledge may be greatly curtailed. An optimization process may push the binary mask Mtto become less binary so that neurons may have better mobility, and a “short-term” memory may be formed as a result. For example, the quantity of mask values corresponding to free neurons may be steadily increased with the binarization bias, a trend observed in region (B) of the graph400as the standard deviation of the mask values corresponding to each neuron. This behavior may be seen as a state transition in which the generator network GθGselects and reserves the neurons that are most representative of previously learned tasks while other neurons are left free for learning subsequent tasks.

Accordingly, for a given task t, the neurons in the generator network GθGmay be divided into three categories: (i) free neurons that are not used at all (U) and may thus be considered as free capacity for the generator network GθG, (ii) neurons that are reserved for the task t (NB), (iii) neurons reserved for previously learned tasks and are being reused for the task t without being changed (R).FIGS.5A-Bdepict the evolution of the ratio of free neurons (NB) and reused neurons (R) relative to the total quantity of neurons in the generator network GθG. As shown graph500inFIG.5shows, the ratio of reused neurons (R) may be increasing across successive tasks whereas the ratio of reserved neurons may be decreasing across successive tasks. These trends may indicate that the generator network GθGmay be learning to generalize better over successive tasks, thus enabling a more efficient allocation of neurons for each successive task.

Nevertheless, one issue with reserving the representative neurons of each task is that the generator network GθGmay eventually run out of capacity. As such, in some example embodiments, the capacity of the generator network GθGmay expand in order to maintain a constant quantity Nfreeof free neurons. This may be achieved by expanding the generation network GθGafter each task with a quantity of reserved neurons denoted as NB(t). The growth of the generator network GθGmay therefore be linked to how efficiently neurons are being reserved for each task, the latter being further dependent on how well the generator network GθGis able to generalize from previously learned tasks.FIG.6depicts a graph600comparing the efficiency of growing the capacity of the generator network GθGin the aforementioned manner to a worst case scenario in which a same quantity of neurons Ninitis added to the generator network GθGfor every task.

As noted, the discriminator network DθDmay be trained jointly with the generator network GθGto differentiate between synthetic training samples and actual training samples as well as to perform various tasks. Accordingly, the training of the discriminator network DθDmay also be an optimization problem that includes minimizing Equation (9) below.
D=ct−st+λGPgpt(9)
whereingptmay denote a gradient penalty term to ensure a more stable training process.

The more significant the domain shift between the training samples associated with different tasks, the quicker the capacity of the generator network GθGmay become exhausted and the sooner the effects of catastrophic forgetting may begin to manifest. This may be attributed to the overall decline in the sparsity of the accumulated mask ml≤tas the machine learning model150is exposed incrementally to training samples associated with different tasks. In order to avoid this effect, the sparsity of the binary mask may be kept substantially the same for each training cycle t.

For example, a layer l of the generator network GθGmay be associated with an input vector of size m and an output vector of size n. At the beginning of an initial training cycle for a task t based on the corresponding training samples D0, the binary mask mlmay be initialized at size n, with zero sparsity. That is, all neurons in the layer l may be available for use, with all n values of the binary mask mlset to a value of 0.5 (e.g., real-valued embeddings e are initialized with 0).

After the initial training cycle with the regularization term R0, the sparsity of the binary mask mlmay decrease to n−δt, wherein δtmay correspond to the quantity of neurons reserved for the task t. In order to avoid exhausting the capacity of the generator network GθG, the capacity of the generator network GθGmay be expanded subsequent to the initial training cycle. For example, the quantity of neurons in the layer l of the generator network GθGmay be expanded by

δtm
in order to guarantee that the capacity of the layer l (e.g., the quantity of free neurons that may be modified to accommodate additional tasks) is kept constant at n for each successive learning cycle during which the generator network GθGis exposed to an additional task.

The classification accuracy of the machine learning model150, implemented as a generative adversarial network having the jointly trained generator network GθGand discriminator network DθDmay exceed that of conventional machine learning models. For example, the classification accuracy of the machine learning model150in a strictly incremental setup may be measured on various benchmark datasets including, for example, the Modified National Institute of Standards and Technology (MNIST) database, the Street View House Number (SVHN) dataset, and the Canadian Institute for Advanced Research (CIFAR-10) collection. Table 1 below depicts a quantitative comparison of the performance of the machine learning model150and other machine learning architectures. As shown in Table 1, the machine learning model150may outperform even machine learning model with access to previously observed training samples.

The performance of the machine learning model150may be evaluated for different types of tasks. For example,FIGS.2A-Bdepict examples of images generated by the machine learning model150trained incrementally to perform multiple tasks, in accordance with some example embodiments.FIG.3Adepicts a graph300illustrating a change in the classification accuracy of the machine learning model150as the machine learning model150is trained incrementally to perform multiple classification tasks. Meanwhile,FIG.3Bdepicts a graph350illustrating a change in the perceptual quality of images generated by the machine learning model150as the machine learning model150is trained incrementally to perform multiple image generation tasks. It should be appreciated that inFIG.3B, the perceptual quality of images generated by the incrementally trained machine learning model150may be assessed based on a Frechet Inception Distance (FID) metric. Moreover, as shown inFIGS.3A-B, the performance of the machine learning model150does not deteriorate as the machine learning model150is trained incrementally to perform different tasks.

FIG.7depicts a flowchart illustrating a process700for incrementally training a machine learning model, in accordance with some example embodiments. Referring toFIGS.1A-Band7, the process700may be performed by the machine learning controller110, for example, to incrementally train the machine learning model150to perform a succession of different tasks.

At702, the machine learning controller110may train, based at least on a first training sample associated with a first task, the machine learning model150to perform the first task. For example, the machine learning controller110may train, based at least on training samples associated with the first task k1, the discriminator network DθDincluded in the machine learning model150to perform the first task k1. In some example embodiments, the machine learning model150may be trained in a strictly incremental setup such that the training samples associated with the first task k1are not stored. Accordingly, to avoid catastrophic forgetting in which the discriminator network DθDforgets knowledge associated with the first task k1when the discriminator network DθDis subsequently trained to perform a second task k2, training the machine learning model150may include training the generator network GθGto generate synthetic training samples emulating the actual training samples associated with the first task k1.

In some example embodiments, in addition to being trained to perform the first task k1, the discriminator network DθDmay be further trained to differentiate between synthetic training samples generated by the generator network GθGand actual training samples. The output of the discriminator network DθDmay therefore be used to train the generator network GθGadversarially to generate synthetic training samples that emulate actual training samples. For example, the generator network GθGmay be trained adversarially to generate synthetic training samples that the differentiator network DθDis incapable of differentiating from actual training samples.

At704, the machine learning controller110may train, based at least on a second training sample associated with a second task, the machine learning model150to perform the second task after the machine learning model150is trained to perform the first task. In some example embodiments, subsequent to being trained for the first task k1, the discriminator network DθDmay be reinitialized before being trained, based on training samples associated with the second task k2, to perform the second task k2. In addition to being trained to perform the second task k2, the discriminator network DθDmay also be retrained, based on the synthetic training samples generated by the generator network GθGfor first task k1, to perform the first task k1. While the discriminator network DθDis trained to perform the second task k2based on training samples associated with the second task k2, the generator network GθGmay be further trained to generate synthetic training samples that emulate the actual training samples associated with the second task k2such that the discriminator network DθDmay subsequently be retrained to perform the second task k2.

In some example embodiments, in order to avoid catastrophic forgetting in which knowledge associated with the second task k2supplants knowledge associated with the first task k1, the generator network GθGmay generate a binary mask M identifying neurons in the generator network GθGthat cannot be modified when the generator network GθGtrained to generate synthetic training samples for the first task k1is subsequently trained to generate synthetic training samples for the second task k2. The binary mask M may lend plasticity to the generator network GθGincluding by preventing the state of neurons configured for the first task k1from being changed when the generator network GθGis exposed to training samples associated with the second task k2. For example, if the training associated with the second task k2requires changing the state of a neuron the binary mask indicates as reserved for the first task k1, these changes may be applied to a different neuron in the generator network GθGthat the binary mask indicates as being free. In the event no free neurons are available in the generator network GθG, the capacity of the generator network GθGmay expand, for example, through the addition of neurons, in order to accommodate the changes necessitated by the second task k2.

At706, the machine learning controller110may deploy the trained machine learning model150to perform the first task and the second task. For example, the machine learning model150may be deployed to perform the first task and the second task in order to implement one or more functionalities of the machine learning application120. Examples of the first task and the second task may include object identification tasks, natural language processing tasks, information retrieval tasks, speech recognition tasks, and/or the like. For instance, the machine learning application120may be a machine learning based communication application such as, for example, a chatbot, an issue tracking system, and/or the like. As such, the trained machine learning model150may be deployed to perform various natural language processing tasks that includes determining a sentiment, a topic, and/or an intent of text received at the machine learning application120. The result of the natural language processing tasks may enable the machine learning application120to further determine an appropriate response to the text received at the machine learning application120.

FIG.8depicts a block diagram illustrating a computing system800, in accordance with some example embodiments. Referring toFIGS.1A-Band8, the computing system800can be used to implement the machine learning controller110and/or any components therein.

As shown inFIG.8, the computing system800can include a processor810, a memory820, a storage device830, and input/output devices840. The processor810, the memory820, the storage device830, and the input/output devices840can be interconnected via a system bus850. The processor810is capable of processing instructions for execution within the computing system800. Such executed instructions can implement one or more components of, for example, the machine learning controller110. In some implementations of the current subject matter, the processor810can be a single-threaded processor. Alternately, the processor810can be a multi-threaded processor. The processor810is capable of processing instructions stored in the memory820and/or on the storage device830to display graphical information for a user interface provided via the input/output device840.

The memory820is a computer readable medium such as volatile or non-volatile that stores information within the computing system800. The memory820can store data structures representing configuration object databases, for example. The storage device830is capable of providing persistent storage for the computing system800. The storage device830can be a floppy disk device, a hard disk device, an optical disk device, or a tape device, or other suitable persistent storage means. The input/output device840provides input/output operations for the computing system800. In some implementations of the current subject matter, the input/output device840includes a keyboard and/or pointing device. In various implementations, the input/output device840includes a display unit for displaying graphical user interfaces.