Patent Description:
Lifelong learning, the problem of continual learning where tasks arrive in sequence, has been lately attracting more attention in computer vision and artificial intelligence research. The aim of lifelong learning is to develop a system that can learn new tasks while maintaining the performance on the previously learned tasks. However, there are two obstacles for lifelong learning of deep neural networks: catastrophic forgetting and capacity limitation.

In many real-world applications, batches of data arrive periodically (e.g., daily, weekly, or monthly) with the data distribution changing over time. This presents an opportunity (or demand) for lifelong learning or continual learning and is an important issue in improving artificial intelligence. The primary goal of lifelong learning is to learn consecutive tasks without forgetting the knowledge learned from previously trained tasks and leverage the previous knowledge to obtain better performance or faster convergence on the newly coming task. One simple way is to finetune the model for every new task. However, such retraining typically degenerates the model performance on both new tasks and the old ones. If the new tasks are largely different from the old ones, it might not be possible to learn the optimal model for the new tasks. Meanwhile, the retrained representations may adversely affect the old tasks, causing them to drift from their optimal solution. This can cause "catastrophic forgetting"- a phenomenon where training a model to perform new tasks interferes the previously learned old knowledge. This leads to a performance degradation or even overwriting of the old knowledge by the new knowledge. Another issue for lifelong learning is resource consumption. A model that is continually trained may increase dramatically in terms of consumed resources (e.g., model size), which may be disadvantageous in applications where resources are limited, for example, in mobile device or mobile computing applications. Non-patent document <NPL>,Retrieved from the Internet:URL:https://arxiv. org/pdf/<NUM>. pdf discloses a method called "Deep Adaptation Networks (DAN)" that constrains newly learned filters to be linear combinations of existing ones.

The present disclosure provides for multi-task based lifelong learning.

According to the invention, there is provided a computer-implemented method for lifelong learning in the field of computer vision according to claim <NUM>, an electronic device for lifelong learning in the field of computer vision according to claim <NUM> and a computer-readable medium comprising program code according to claim <NUM>.

Those skilled in the art will understand that the principles of the present disclosure may be implemented in any suitably-arranged system or device.

Embodiments of the present disclosure recognize that there are two primary obstacles for lifelong learning of deep neural networks: catastrophic forgetting and capacity limitations. Accordingly, various embodiments of the present disclosure provide for multi-task based lifelong learning framework, with some embodiments involving non-expansive automatic ML (AutoML). In some embodiments, this framework is referred to as a regularize, expand and compress (REC) framework. In these embodiments, the REC framework includes at least one of the three stages: <NUM>) allowing the ML model to continually learn new tasks without access to the learned tasks' data via multi-task weight consolidation (MWC), <NUM>) expanding the neural network architecture for the ML model to learn the new tasks with improved model capability and performance by network-transformation based AutoML, and <NUM>) compressing the expanded ML model after learning the new tasks to maintain model size, efficiency, and performance. In this disclosure, the terms of "neural network" and "network" are used interchangeably.

Some embodiments of the present disclosure automatically expand the neural network architecture for lifelong learning, with higher performance and less parameter redundancy than other network architectures. To better facilitate both automatic knowledge transfer without human expert tuning and model design with optimized model complexity, some embodiments apply AutoML for lifelong learning while taking learning efficiency into consideration.

As used herein, AutoML is automatically learning a suitable ML model for a given task. Neural architecture search (NAS) is a subfield of AutoML for deep learning, which searches for optimal hyperparameters of a neural network architecture using a reinforcement learning framework. Neural network architecture defines how different ML algorithms in the ML model work together and process inputs, perform tasks, and provide results. Reinforcement learning framework involves observing a network's performance on a validation set as a reward signal, and giving higher probabilities to network architectures that have higher performances than network architectures that have lower performances to adapt the network model. Embodiments of the present disclosure recognize that using a reinforcement learning framework directly in the lifelong learning setting would result in forgetting the model's knowledge of old tasks and may be wasteful because each new task network architecture would need to be searched from scratch, which ignores correlations between previously learned tasks and the new task.

Various embodiments of the present disclosure provide an efficient AutoML algorithm for lifelong learning. In some embodiments, this efficient AutoML algorithm is referred to as a Regularize, Expand and Compress (REC). In these embodiments, REC involves first searching a best new neural network architecture for the given tasks in a continuous learning mode. Tasks may include image classification, image segmentation, object detection and/or many other computer vision tasks. The best neural network architecture can solve multiple different tasks simultaneously, without catastrophic forgetting of old tasks' information, even when there is no access to old tasks' training data. Next, in these embodiments, REC involves exploring the network architecture via AutoML. For example, this may involve using a net2net transformation based AutoML to reduce the search space and the training time of the new neural network architecture. REC is an efficient lifelong learning algorithm, which offers accurate task performance on mobile devices without or with little memory increase after learning different new tasks.

Various embodiments of the present disclosure are implemented by or within part or all of a computing system. <FIG> illustrates an example computing system <NUM> in which various embodiments of the present disclosure may be implemented. The embodiment of the system <NUM> shown in <FIG> is for illustration only. Other embodiments of the system <NUM> can be used without departing from the scope of the present disclosure.

In this illustrative example, the computing system <NUM> is a system in which the lifelong learning techniques of the present disclosure may be implemented. The system <NUM> includes network <NUM> that facilitates communication between various components in the system <NUM>. For example, network <NUM> can communicate Internet Protocol (IP) packets, frame relay frames, or other information between network addresses. The network <NUM> includes one or more local area networks (LANs), metropolitan area networks (MANs), wide area networks (WANs), all or a portion of a global network such as the Internet, or any other communication system or systems at one or more locations.

The network <NUM> facilitates communications between a server <NUM> and various client devices <NUM>-<NUM>. The client devices <NUM>-<NUM> may be, for example, a smartphone, a tablet computer, a laptop, a digital assistant device, a voice-controlled speaker, a television, a personal computer, a wearable device, smart watch, a head-mounted display (HMD), etc. The server <NUM> can represent one or more servers. Each server <NUM> includes any suitable computing or processing device that can provide computing services for one or more client devices. Each server <NUM> could, for example, include one or more processing devices, one or more memories storing instructions and data, and one or more network interfaces facilitating communication over the network <NUM>. As described in more detail below, in various embodiments, the server <NUM> may train models for multi-task based lifelong learning. In other embodiments, the server <NUM> may be a webserver to provide or access deep-learning neural networks, training data, and/or any other information to implement multi-task based lifelong learning embodiments of the present disclosure.

Each client device <NUM>-<NUM> represents any suitable computing or processing device that interacts with at least one server or other computing device(s) over the network <NUM>. In this example, the client devices <NUM>-<NUM> include a desktop computer <NUM>, a mobile telephone or mobile device <NUM> (such as a smartphone), a personal digital assistant (PDA) <NUM>, a laptop computer <NUM>, a tablet computer <NUM>, and a digital assistant device <NUM>. However, any other or additional client devices could be used in the system <NUM>. As described in more detail below, each client device <NUM>-<NUM> may train models for multi-task based lifelong learning.

In this example, some client devices <NUM>-<NUM> communicate indirectly with the network <NUM>. For example, the client devices <NUM> and <NUM> (mobile devices <NUM> and PDA <NUM>, respectively) communicate via one or more base stations <NUM>, such as cellular base stations or eNodeBs (eNBs). Mobile device <NUM> includes smartphones. Also, the client devices <NUM>, <NUM>, and <NUM> (laptop computer, tablet computer, and digital assistant device, respectively) communicate via one or more wireless access points <NUM>, such as IEEE <NUM> wireless access points. Note that these are for illustration only and that each client device <NUM>-<NUM> could communicate directly with the network <NUM> or indirectly with the network <NUM> via any suitable intermediate device(s) or network(s).

Although <FIG> illustrates one example of a system <NUM>, various changes can be made to <FIG>. For example, the system <NUM> could include any number of each component in any suitable arrangement. In general, computing and communication systems come in a wide variety of configurations, and <FIG> does not limit the scope of the present disclosure to any particular configuration. While <FIG> illustrates one operational environment in which various features disclosed in this patent document can be used, these features could be used in any other suitable system.

<FIG> and <FIG> illustrate example devices in a computing system in accordance with various embodiments of the present disclosure. In particular, <FIG> illustrates an example electronic device <NUM>, and <FIG> illustrates an example electronic device <NUM>. The electronic device <NUM> could represent the server <NUM> of <FIG>, and the electronic device <NUM> could represent one or more of the client devices <NUM>-<NUM> of <FIG>. The embodiments of the electronic devices <NUM> and <NUM> shown in <FIG> and <FIG> are for illustration only, and other embodiments could be used without departing from the scope of the present disclosure. The electronic devices <NUM> and <NUM> can come in a wide variety of configurations, and <FIG> or <FIG> do not limit the scope of the present disclosure to any particular implementation of an electronic device.

Electronic device <NUM> can represent one or more servers or one or more personal computing devices. As shown in <FIG>, the electronic device <NUM> includes a bus system <NUM> that supports communication between at least one processor(s) <NUM>, at least one storage device(s) <NUM>, at least one communications interface <NUM>, and at least one input/output (I/O) unit <NUM>.

The processor <NUM> executes instructions that can be stored in a memory <NUM>. The instructions stored in memory <NUM> can include ML model(s) for performing tasks instructions, neural network architectures for ML algorithms of such ML models, and algorithms to implement a framework for multi-task based lifelong learning. The processor <NUM> can include any suitable number(s) and type(s) of processors or other devices in any suitable arrangement. Example types of processor(s) <NUM> include microprocessors, microcontrollers, digital signal processors, field programmable gate arrays, application specific integrated circuits, and discrete circuitry.

The memory <NUM> and a persistent storage <NUM> are examples of storage devices <NUM> that represent any structure(s) capable of storing and facilitating retrieval of information (such as data, program code, or other suitable information on a temporary or permanent basis). The memory <NUM> can represent a random-access memory or any other suitable volatile or non-volatile storage device(s). The persistent storage <NUM> can contain one or more components or devices supporting longer-term storage of data, such as a ready-only memory, hard drive, Flash memory, or optical disc.

The communications interface <NUM> supports communications with other systems or devices. For example, the communications interface <NUM> could include a network interface card or a wireless transceiver facilitating communications over the network <NUM> of <FIG>. The communications interface <NUM> can support communications through any suitable physical or wireless communication link(s). The I/O unit <NUM> allows for input and output of data. For example, the I/O unit <NUM> can provide a connection for user input through a keyboard, mouse, keypad, touchscreen, motion sensors, or any other suitable input device. The I/O unit <NUM> can also send output to a display, printer, or any other suitable output device.

Note that while <FIG> is described as representing the server <NUM> of <FIG>, the same or similar structure could be used in one or more of the various client devices <NUM>-<NUM>. For example, a desktop computer <NUM> or a laptop computer <NUM> could have the same or similar structure as that shown in <FIG>.

The electronic device <NUM> can be any personal computing device, such as, for example, a wireless terminal, a desktop computer (similar to desktop computer <NUM> of <FIG>), a mobile device (similar to mobile device <NUM> of <FIG>), a PDA (similar to PDA <NUM> of <FIG>), a laptop (similar to laptop computer <NUM> of <FIG>), a tablet (similar to tablet computer <NUM> of <FIG>), a digital assistant device (similar to digital assistant device <NUM> of <FIG>), a smart television, a wearable device, smartwatch, and the like.

As shown in <FIG>, the electronic device <NUM> includes an antenna <NUM>, a radiofrequency (RF) transceiver <NUM>, a transmit (TX) processing circuitry <NUM>, a microphone <NUM>, and a receive (RX) processing circuitry <NUM>. The electronic device <NUM> also includes a speaker <NUM>, a one or more processors <NUM>, an input/output (I/O) interface (IF) <NUM>, an input <NUM>, a display <NUM>, and a memory <NUM>. The memory <NUM> includes an operating system (OS) <NUM>, one or more applications <NUM>, and ML model(s) <NUM>.

The RF transceiver <NUM> receives, from the antenna <NUM>, an incoming RF signal transmitted by another component on a system. For example, the RF transceiver <NUM> receives RF signal transmitted by a BLUETOOTH or WI-FI signal from an access point (such as a base station, WI-FI router, BLUETOOTH device) of the network <NUM> (such as a WI-FI, BLUETOOTH, cellular, <NUM>, LTE, LTE-A, WiMAX, or any other type of wireless network). The received signal is processed by the RX processing circuitry <NUM>. The RX processing circuitry <NUM> may transmit the processed signal to the speaker <NUM> (such as for voice data) or to the processor <NUM> for further processing (such as for web browsing data). The TX processing circuitry <NUM> receives voice data from the microphone <NUM> or other outgoing data from the processor <NUM>. The outgoing data can include web data, e-mail, or interactive video game data. The TX processing circuitry <NUM> processes the outgoing data to generate a processed signal. The RF transceiver <NUM> receives the outgoing processed signal from the TX processing circuitry <NUM> and converts the received signal to an RF signal that is transmitted via the antenna <NUM>.

The processor <NUM> can include one or more processors or other processing devices and execute the OS <NUM> stored in the memory <NUM> in order to control the overall operation of the electronic device <NUM>. Example types of processor <NUM> include microprocessors, microcontrollers, digital signal processors, field programmable gate arrays, application specific integrated circuits, and discrete circuitry.

The processor <NUM> is also capable of executing other applications <NUM> resident in the memory <NUM>, such as for multi-task based lifelong learning. The processor <NUM> can move data into or out of the memory <NUM> as required by an executing process. In some embodiments, the processor <NUM> is configured to execute the plurality of applications <NUM> based on the OS <NUM> or in response to signals received from eNBs (similar to the base stations <NUM> of <FIG>) or an operator. The processor <NUM> is also coupled to the I/ O IF <NUM> that provides the electronic device <NUM> with the ability to connect to other devices, such as client devices <NUM>-<NUM>. The I/O IF <NUM> is the communication path between these accessories and the processor <NUM>.

The processor <NUM> is also coupled to the input <NUM>. The operator of the electronic device <NUM> can use the input <NUM> to enter data or inputs into the electronic device <NUM>. Input <NUM> can be a keyboard, touch screen, mouse, track-ball, voice input, or any other device capable of acting as a user interface to allow a user in interact with electronic device <NUM>. For example, the input <NUM> can include voice recognition processing thereby allowing a user to input a voice command via microphone <NUM>. For another example, the input <NUM> can include a touch panel, a (digital) pen sensor, a key, or an ultrasonic input device. The touch panel can recognize, for example, a touch input in at least one scheme among a capacitive scheme, a pressure sensitive scheme, an infrared scheme, or an ultrasonic scheme.

The processor <NUM> is also coupled to the display <NUM>. The display <NUM> can be a liquid crystal display (LCD), light-emitting diode (LED) display, organic LED (OLED), active matrix OLED (AMOLED), or other display capable of rendering text and/or graphics, such as from websites, videos, games, images, and the like.

Part of the memory <NUM> could include a random-access memory (RAM), and another part of the memory <NUM> could include a Flash memory or other read-only memory (ROM). The memory <NUM> can include persistent storage that represents any structure(s) capable of storing and facilitating retrieval of information (such as data, program code, and/or other suitable information on a temporary or permanent basis). The memory <NUM> can contain one or more components or devices supporting longer-term storage of data, such as a ready only memory, hard drive, Flash memory, or optical disc. In various embodiments, the electronic device <NUM> includes the ML model <NUM> to perform ML tasks. This ML model <NUM> can be adapted using one or more of the techniques for multi-task based lifelong learning disclosed herein. For example, the electronic device <NUM> may implement a framework for multi-task based lifelong learning to efficiently adapt the ML model <NUM> to perform new tasks while maintaining performance on old tasks and model size.

Electronic device <NUM> can further include one or more sensors <NUM> that meter a physical quantity or detect an activation state of the electronic device <NUM> and convert metered or detected information into an electrical signal. For example, sensor(s) <NUM> may include one or more buttons for touch input, one or more cameras, a gesture sensor, an eye tracking sensor, a gyroscope or gyro sensor, an air pressure sensor, a magnetic sensor or magnetometer, an acceleration sensor or accelerometer, a grip sensor, a proximity sensor, a color sensor, a bio-physical sensor, a temperature/ humidity sensor, an illumination sensor, an Ultraviolet (UV) sensor, an Electromyography (EMG) sensor, an Electroencephalogram (EEG) sensor, an Electrocardiogram (ECG) sensor, an infrared (IR) sensor, an ultrasound sensor, a fingerprint sensor, and the like. The sensor(s) <NUM> can further include a control circuit for controlling at least one of the sensors included therein.

For example, in various embodiments, the camera in in the sensor(s) <NUM> may be used to capture images and/or videos of objects for tasks such as object detection and/ or classification for multi-task based lifelong learning. In other embodiments, the microphone <NUM> may be used to capture voice inputs and/or audio for an audio recognition model which is adapted using multi-task based lifelong learning.

Although <FIG> and <FIG> illustrate examples of devices in a computing system, various changes can be made to <FIG> and <FIG>. For example, various components in <FIG> and <FIG> could be combined, further subdivided, or omitted and additional components could be added according to particular needs. In addition, as with computing and communication networks, electronic devices and servers can come in a wide variety of configurations, and <FIG> and <FIG> do not limit the present disclosure to any particular electronic device or server.

Embodiments of the present disclosure recognize that expanding the network architecture for more and more new tasks may lead to a much larger model size, as compared with an initial model, which leads to inefficiencies including memory requirements, power usage, and processing requirements. Many network-expanding-based lifelong learning algorithms increase the model capability but also decrease the learning efficiency in terms of memory cost and power usage. To address these issues, various embodiments provide for model compression after completing the learning of each new task-compressing the expanded model to the initial model size, while focusing on negligible performance loss on both old and new tasks.

<FIG> illustrates a conceptual example of network expansion and compression for multi-task based lifelong learning in accordance with various embodiment of the present disclosure. The conceptual example in <FIG> is for illustration only. Other embodiments could be used without departing from the scope of the present disclosure.

In this example, network architecture <NUM> is an illustration of the architecture for a neural network with connections between various ML algorithms as nodes. Network architecture <NUM> has been trained to perform multiple tasks (i.e., old or existing task(t-<NUM>) <NUM> and new task(t) <NUM>). In this example, the bolded network nodes and lines represent the old or existing network architecture originally in place to perform the old task <NUM>, while the non-bolded nodes and lines represent the expansions to the old or existing network architecture in order to perform the new task <NUM>. As discussed in greater detail below, embodiments of the present disclosure compress the expanded network to increase model efficiency. This compression is illustrated by the dashed network layers and nodes that are removed from the network architecture <NUM>.

In various embodiments, the REC framework of the present disclosure is able to provide for reduced or limited model size from a memory requirement standpoint, can retain old task performance without the original training data for the old tasks, can expand the network capacity to perform new tasks, and provides for AutoML.

<FIG> illustrates an example of a system <NUM> for multi-task based lifelong learning in accordance with various embodiment of the present disclosure. For example, the system <NUM> may be implemented by either of the electronic device <NUM> or <NUM> in <FIG> or <FIG>. The embodiment of the system <NUM> is for illustration only. Other embodiments could be used without departing from the scope of the present disclosure.

In this illustrative example, system <NUM> provides an exemplary AutoML framework called REC for lifelong learning. In various embodiments, REC includes three steps: (<NUM>) regularize using multi-task weight consolidation (MWC), (<NUM>) expand the network architecture using AutoML, and (<NUM>) compress the expanded network architecture. The system <NUM> includes a controller <NUM> that, upon identifying a new task to be performed, uses deeper and wider operators <NUM> and <NUM>, respectively, (e.g., such as Net2Deeper and Net2Wider operators) to generate a plurality of child network architectures <NUM>. The deeper operator <NUM> adds layer(s) to the existing network architecture to perform the new task, while the wider operator <NUM> widens existing layer(s) of the existing network architecture to perform the new task. The child network architectures <NUM> are expanded versions of the original/prior/existing network architecture that are generated to perform the new tasks. In these embodiments, the child networks architecture <NUM> include both wider and deeper child network architectures 520w and 520D, respectively, and child network architectures that are combinations thereof.

Thereafter, the system <NUM> uses MWC and AutoML, as discussed in greater detail below, to search for the best child network architecture 520B for new task, REC using the Net2deeper and Net2wider operators <NUM> and <NUM>, respectively, in the controller <NUM> of AutoML. Then, the system <NUM> compresses the expanded best child network architecture to the same or a similar size as the original/prior/existing network architecture to maintain model size. Thereafter, the system <NUM> repeats the process for additional new tasks that are identified to provide continual or lifelong learning for multiple tasks.

In one embodiment, the system <NUM> continually and automatically learns on sequential data sets. With a given small network, the system <NUM> learns an initial model on a first given task. In this embodiment, using the REC framework, the system <NUM> then searches for the best network architecture by network transformation based AutoML for the upcoming new tasks without access to the old tasks' data using an MWC algorithm and compresses the expanded network to the initial network's size.

In various embodiments, to overcome catastrophic forgetting for the old tasks, the system <NUM> uses a novel loss function MWC. Using MWC, the system <NUM> considers the discriminative weight subset by incorporating inherent correlations between old tasks and new task and learns the newly added or widened layer as a task-specific layer for the new task. In one embodiment, to narrow down the architecture searching space and save training time, network transformation based AutoML is utilized to accelerate the new network searching.

Furthermore, unlike network-expanding-based lifelong learning algorithms, in one embodiment, using REC, the system <NUM> compresses the model after learning every new task to guarantee the model efficiency. The final model is a space-efficient model, but with an enhanced performance caused by network expansion before the compression. In various embodiments, the system <NUM> may use knowledge distillation to preserve knowledge of old tasks and compress model size for lifelong learning.

In particular, in various embodiments, the system <NUM> is able to provide for lifelong learning based on an unknown number of tasks with unknown distributions, being identified sequentially. The system <NUM> provides a deep-learning model for lifelong learning without catastrophic forgetting. Given a sequence of T tasks, task at time point t = <NUM>, <NUM>, · · · , T with Nt images comes with dataset <MAT>. <MAT> is the label for the ith sample <MAT> in task t, where R represents the real number space and dt is a dimension of R. The training data matric is denoted by X t for D t, ie. , X t=(x <NUM>t,x <NUM>t,. When the dataset of task t is identified, the previous training datasets D <NUM>,. , D t-<NUM> may not be available any more, but the deep model parameter <MAT> can be obtained. For example, the lifelong learning problem at time point t when given data Dt can be defined as solving:
<MAT>
where F is the loss function of solving θ t, θ tis the parameter for task t. That the number of the upcoming tasks can be finite or infinite - for simplification, one embodiment considers the finite scenario.

Embodiments of the present disclosure recognize that ecstatic weight consolidation (EWC), which involves a quadratic penalty on the difference between the parameter θ t and θ t-<NUM>, may be used to slow down the catastrophic forgetting for previously learned tasks. For example, the posterior distribution ρ is used to describe the problem by the Bayes' rule:
<MAT>
where the posterior probability logρ(θt|Dt-<NUM>) embeds the information from task t-<NUM>. However, the equation <NUM> is intractable so that EWC approximates it as a Gaussian distribution with mean of parameter θ̂t-<NUM> and a diagonal I of the fisher information matrix F. The matrix F is computed by F i= <MAT>.

Therefore, for example, the problem of EWC on task t can be written as follows:
<MAT>
where Ft is the loss function for task t, λ denotes how important the task t-<NUM> is compared to the task t and i labels each weight of the parameter θ.

Embodiments of the present disclosure recognize that one issue with using EWC is that EWC only enforces task t close to task t-<NUM>. This will ignore the inherent correlations between task t-<NUM> and task t and such relationship might potentially help overcome catastrophic forgetting on the previously learned tasks.

Accordingly, embodiments of the present disclosure recognize that learning multiple related tasks jointly can improve performance relative to learning each task separately. Redefining equation <NUM> using multi-task learning (MTL), a new objective function (e.g., equation <NUM>) is provided to improve the ability of overcoming catastrophic forgetting from multiple tasks simultaneously:
<MAT>
where λ2 is the non-negative regularization parameter and
∥[θt; θt-<NUM>]∥<NUM>,<NUM> = ∥∥θt∥<NUM>, ∥θt-<NUM>∥<NUM>∥<NUM> is the l2,<NUM>-norm regularization to learn the related representations. Here, multi-task learning is leveraged by incorporating <NUM>,<NUM>-norm to capture the common subset of relevant parameters from each layer for task t-<NUM> and task t.

<FIG> illustrates a conceptual example of the use of MWC in network expansion and compression for multi-task based lifelong learning in accordance with various embodiment of the present disclosure. The conceptual example in <FIG> is for illustration only. Other embodiments could be used without departing from the scope of the present disclosure.

In this example, similarly to network architecture <NUM> in <FIG>, network architecture <NUM> has been expanded and compressed as discussed above to perform multiple tasks (i.e., old or existing task(t-<NUM>) <NUM> and new task(t) <NUM>). In this embodiment, the system <NUM> uses MWC to retrain the network architecture learned on previous tasks while regularizing the network architecture to prevent forgetting from the original model. In graph <NUM>, line <NUM> illustrates an MWC approach that learns better parameter representations to overcome catastrophic forgetting by studying MTL with the sparsity-inducing norm (indicated by line <NUM>) and EWC (indicated by line <NUM>). Line <NUM> indicates a no penalty approach for low error between the existing and new tasks.

In one embodiment, additional parameters are considered that have better representation power corresponding to a subset of tasks. The MTL with sparsity-inducing norm can select discriminative parameter subsets, by incorporating inherent correlations among multiple tasks. To this end, the <NUM> sparse norm is imposed to learn the new task-specific parameters while learning task relatedness among multiple tasks. Therefore, for example, the objective function for task t becomes:
<MAT>
where λ3 is the non-negative regularization parameter. This algorithm incorporates MWC because it includes the discriminative weight subset by incorporating inherent correlations among multiple tasks.

Various embodiments of the present disclosure provide AutoML for lifelong learning with MWC. MWC is a regularization-based lifelong learning algorithm. In various embodiments, the system <NUM> uses AutoML with MWC as a loss function to expand the network architecture if the task is very different from the existing task or the network capacity suffers from sparsity. In one embodiment, the system <NUM> provides AutoML for lifelong learning, in an algorithm (REC) that is summarized in Table <NUM> below. The details of the network transformations in AutoML for REC are outlined in Table <NUM>.

In one embodiment, the system <NUM> applies wider and deeper operators in the controller <NUM> as discussed above. In one example, the wider network transformation function is denoted as:
<MAT>
where Ol represents the outputs of the original layer <NUM>. And the deeper network transformation function is denoted as:
<MAT>
where the constraint γ holds for the rectified linear activation. The system <NUM> learns a meta-controller to generate network transformation actions (Eq. <NUM> and Eq. <NUM>) when given the initial network architecture. Specifically, in one example, the system <NUM> uses an encoder network, which is implemented with an input embedding layer and a bidirectional recurrent neural network, to learn a low-dimensional representation of the initial network and be embedded into different operators to generate different network transformation actions. The system <NUM> may also use a shared sigmoid classifier to make the wider decision according to a hidden state of the layer learned by the bidirectional encoder network and the wider network can be further combined with a deeper operator.

MWC (e.g., as expressed in Eq. <NUM>) is integrated into above AutoML system for lifelong learning. After learning the network θ t-<NUM> on the data D t-<NUM>, the system <NUM> automatically searches for the best child network θt by wider and deeper operators when it is necessary to expand the network while keeping the model performance on task t-<NUM> based on equation <NUM>. If the controller <NUM> decides to expand the network architecture, the newly added layer(s) will not have the previous tasks' Fisher Information. Accordingly, the system <NUM> considers the newly added layer as a new task specific layer, <NUM>. Regularization is adopted to promote sparsity in the new weight so that each neuron in the network architecture is only connected with few neurons in the layer below. This will efficiently learn the best (or an improved) representation for the new task while reducing the computation overhead. For example, the modified MWC in this network expanding scenario is as follows:<MAT>
where the subscript deeper and wider refer to the newly added layer in task t.

After the controller <NUM> generates the child network architecture(s), the child network architecture(s) achieve an accuracy Aval on the validation set of task t and this will be used as the reward signal Rt to update the controller <NUM>. The controller <NUM> maximizes (or increases) the expected reward to find the optimal, desired, or best child network architecture. For example, the empirical approximation of this AutoML reinforcement can be expressed as follows:
<MAT>
where m is the number of child network architectures that the controller C samples in one batch and as and gs represents the action and state of predicting s-th hyper-parameter to design a child network architecture, respectively. T is the transition function in Table <NUM> below. Since Rt is non- linear transformation tan (Aval × π/<NUM>) on validation set of task t , the transformed value is used as the reward. An exponential moving average of previous rewards with a decay of <NUM> is used to reduce the variance. To balance the old task and new task knowledge, for example, maximum expanding layers are set as <NUM> and <NUM> on the wider and deeper operators, respectively.

If the network keeps expanding as more and more tasks will be given, the model will suffer inefficiency and extra memory cost. Thus, the system <NUM> uses a model compression technique to reduce the memory cost and generate a non-expansive model. In one embodiment, the system <NUM> uses soft-labels (the logits) as knowledge distillation instead of the hard labels to train the model for the child network architectures. In one embodiment, the system <NUM> trains the model to minimize (or reduce) the mean of the l2 loss on the training data <MAT>, where <MAT> is the logits of the child network architecture θ t i-th training sample. For example, the child network architecture θ t can be compressed to the same size model as θ <NUM> by knowledge distillation loss as follows:
<MAT>
where <MAT> is the weights of the child network architecture and <MAT> is the prediction of task i-th training sample. The final child network architecture <MAT> is trained to convergence with hard and soft labels by the following loss function:
<MAT>
where F is the loss function (cross-entropy in this work) for training with ground truth yt of task t.

Various embodiments of the present disclosure may be implemented by or for use with digital assistant devices and programs, for example, Bixby digital assistant by Samsung Electronics, Co. Digital assistant supporting computer vision programs are continuously updated to add new tasks, for example, for recognizing different types of items and places. Typically, the addition of each new task may require the deployment of a new deep learning model. Many different tasks being added overtime may require deployment of many models, potentially resulting in high storage and energy (e.g., battery) cost.

Embodiments of the present disclosure enable lifelong learning allowing continuous addition of such additional tasks for digital assistant (e.g., digital assistant device <NUM> in <FIG>) while deploying one deep learning model. For example, in these embodiments, the system <NUM> identifies the initial/existing model for the initial/existing task(s). This model can be a compressed model or a non-compressed model. The system <NUM> then obtains the parameters from the existing model and adapts the existing model according to MWC (e.g., using equation <NUM> above) based AutoML (e.g., using Table <NUM>: step <NUM> above) to expand the deep learning model to suit additional new task(s) beyond the initial/existing task(s). The system <NUM> then compresses the expanded network (e.g., using equation <NUM> above with Table <NUM>: step <NUM> above).

In one embodiment, the compressed model will have a same or similar model size as the initial deep model. For example, it may not be advantageous to compress the model as small as possible, because a smaller model size may lead to accuracy loss. Instead, in this embodiment, the system <NUM> compresses the model little by little. For example, whenever a new task is added into the model, after expanding to accommodate the added task, the system <NUM> compresses the model into its original (or similar) model size. This compression and model size consistency is a trade-off between model complexity and model accuracy. The final deep model will have the same (or similar) model size as the initial model, but the final model can work on additional tasks as compared with the initial model. The system <NUM> can continually add new tasks, no matter the number and keep adapting the model as necessary whenever a new task is added.

In one embodiment, an initial version of computer vision for a digital assistant program is developed with a certain deep learning model for visual recognition, e.g., food recognition. The initial deployed deep learning model may be trained by developers or a third-party. If the initial model is pre-trained by a third-party, access to the training data may not be available-just the model itself. However, using model hyperparameters, the system <NUM> can adapt the model to perform new tasks without requiring access to the old tasks' training data. Additionally, for example, in mobile device applications where computation, storage, and battery constraints may be present, this initial model might have been compressed to the minimal size/capacity. Next, given the desire to add new visual recognition tasks/capabilities (e.g., particular item or place recognitions), there may be a few constraints in adding the new task. For example, in some embodiments, deploying multiple deep learning models, each corresponding to a different task, on the mobile device may be undesirable. Also, in various embodiments, the existing model should be adapted to perform multiple tasks. Further, in various embodiments, the adapted model's size should be increased as minimal as possible (e.g., by using Net2Net based AutoML as discussed above). Additionally, without training data for the old task, embodiments of the present disclosure can reduce or eliminate accuracy reduction on the old task by using MWC (e.g., by using equation <NUM> above). Further, in some embodiments, given that a trade-off exists between the model size and the model accuracy, the adapted output model remains the same or similar size as the initial model without forgetting the old task and also has the capacity to perform for multiple tasks.

In particular, for one embodiment, the system <NUM> can reuse the digital assistant's existing model to add new tasks/capabilities as follows. First, when a new task (e.g., wine recognition) is identified to be added, the system <NUM> prepares training and validation dataset/images for the new tasks (e.g., from a database or by internet image searches). The system <NUM> also obtains the previously trained model or model hyperparameters as the initial/existing model, for example, without access to old training database. Second, the system <NUM> uses one or more of the AutoML for Lifelong Learning with MWC techniques disclosed herein (e.g., Table <NUM>: step <NUM>) to adaptively train the existing deep learning model (e.g., a food recognition model) to perform the additional task. During the training, MWC (e.g., using equation <NUM>) provides that even without training data for the old/existing task, the model can still maintain the existing accuracy/performance. The system <NUM> uses AutoML if, for example, the accuracy on the new task does not meet desired accuracy requirements/standards, for example, the model size/capacity may be too small to accommodate two tasks. The system <NUM> may then expand the deep learning model size, for example, according to the algorithm in Table <NUM>. Third, when the accuracy requirement/standards are met and the model size has been increased significantly, the system <NUM> can compress the adapted model (e.g., using the compression according to equation <NUM> above and the algorithm in Table <NUM> and step <NUM>). As a result, the computation, memory, and power cost on the adapted model is not greatly increased or is maintained.

<FIG> illustrates a flowchart of a process <NUM> for multi-task based lifelong learning in accordance with various embodiments of the present disclosure. For example, the process depicted in FIG. <NUM> may be performed by the electronic device <NUM> in <FIG>, the electronic device <NUM> in <FIG>, and the system <NUM> in <FIG>, respectively, generally referred to here as the system.

The process begins with the system identifying a new task for a ML model to perform (step <NUM>). For example, in step <NUM>, the ML model may be trained to perform an existing task such as, for example, a computer vision task such as image classification or object detection tasks.

The system then adaptively trains a network architecture of the machine learning model to generate an adapted machine learning model based on incorporating inherent correlations between the new task and the existing task (step <NUM>). For example, in various embodiments, in step <NUM>, the system may generate and identify an adapted network architecture based on MWC as discussed above. In using MWC, the system may incorporate inherent correlations between the existing task and the new task and identify the added layer as a task-specific layer for the new task. Also, for example, the system may train the ML model to perform the new task using training data for the new task without access to the training data for the old task.

In some embodiments, to adapt the network architecture in step <NUM>, the system may expand the network architecture for the ML model to perform the new task using AutoML, for example, by training child network architectures using wider and deeper operators as discussed with regard to <FIG> above. The expanded network architecture may include adding a layer to the network architecture and expanding one or more existing layers of the network architecture. After network expansion, the system may compress the expanded network architecture to generate the adapted ML model. For example, the system may remove and shrink layers from the selected best child network architecture to compress the child network architecture. To compress the expanded child network architecture, the system may use the training data for the new task to provide for accuracy in performing the task after compression. In some embodiments, the system may compress the generated child network architecture for the adapted ML model to have a reduced size or a same/similar size as the initial ML model to reduce memory constraints.

In some embodiments, to adapt the network architecture in step <NUM>, the system may first adapt the network architecture using MWC as discussed above. Then, if the system determines that performance accuracy or standards cannot be maintained, for example, on either or both of the existing or new tasks, the system may then use AutoML as discussed above to expand the network architecture to achieve the desired performance accuracy or standards. Thereafter, the system may compress the network architecture to maintain model size.

The system then uses the adapted ML model to perform both existing and new tasks (step <NUM>). Also, after step <NUM>, the system may return to step <NUM> to continuously adapt the network architecture for additional new tasks for the lifelong learning for each new task identified.

Although <FIG> illustrates an example processes for multi-task based lifelong learning, various changes could be made to <FIG>. For example, while shown as a series of steps, various steps in each figure could overlap, occur in parallel, occur in a different order, or occur multiple times.

Although the figures illustrate different examples of user equipment, various changes may be made to the figures. For example, the user equipment can include any number of each component in any suitable arrangement. In general, the figures do not limit the scope of the present disclosure to any particular configuration(s). Moreover, while figures illustrate operational environments in which various user equipment features disclosed in this patent document can be used, these features can be used in any other suitable system.

Claim 1:
A computer-implemented method for lifelong learning in the field of computer vision, the method comprising:
receiving a new task in a sequence of tasks for a machine learning model to perform, the machine learning model trained to perform an existing task;
for the new task, searching the best network architecture for the machine learning model by network transformation based AutoML without access to data of the existing task using a multi-task weight consolidation approach to learn a discriminative weights subset by incorporating inherent correlations between the new task and the existing task to provide an adapted machine learning model having an expanded network architecture size;
and
using the adapted machine learning model to perform both the existing task and the new task,
wherein the existing task and the new task comprise tasks relating to at least one of image classification, image segmentation, object detection and other computer vision tasks.