Meta Learning: Learning to Learn in Neural Networks
By: Atul Shah
Tags: Neural Networks, Deep Learning, Meta Learning, Learning

Formalizing Meta-Learning

This ‘learning-to-learn’ is very much aligned with human and animal learning in which learning methods incrementally improve over a period of time. This approach has advantages of data and compute efficiency.

Meta learning algorithm can be understood as made up of 2 levels of learning — inner and outer algorithm. Inner learning is similar to conventional learning algorithm such as improving image classification. During meta-learning, inner algorithm learns the model while also improving the outer objective (meta-objective).This objective could be generalization performance or learning speed of the inner algorithm. Learning iterations of the base task can be thought of as providing the stimulus needed by the outer algorithm in order to learn the base learning algorithm.

Terminology

Meta learning algorithm objective function can be mathematically expressed as

L:a function that measures the match between true labels and those predicted by f(θ) θ: parameter for inner algorithm

D: the Dataset under consideration.

ω: meta-knowledge to denote dependence on θ and class of labels.

History

Meta-learning first appears in the literature in 1987 in two separate and independent pieces of work, by J. Schmidhuber and G. Hinton. They set the theoretical foundations for a new family of algorithms that can learn how to learn, using self-referential learning.

Then in 2001, proposals for training meta-learning systems using gradient descent and backpropagation were first made. Finally in 2012 Meta-learning was re-introduced in the modern era of deep neural networks, which marked the beginning of modern meta-learning of the type discussed here.

Taxonomy

Meta Learning landscape can be broadly categorized as follows, each identifying parameters and functioning of the algorithms:

Meta-Representation

This classification is based on representation of meta-knowledge ω. This includes estimation optimizer initialization parameters. These parameters can be domain specific.

Meta-Optimizer

This classification is based on the choice of optimizer to be used for the outer level algorithm. This includes methods such as gradient-descent, reinforcement learning and evolutionary search.

Meta-Objective

This classification is based on the goal of meta-learning which is determined by choice of meta objective(loss function), task distribution p(T), and dataflow between the two levels.

Overview of the meta-learning landscape

Let us explore few of the models shown in the figure above.

Parameter Initialization

If we start with good initialization parameters, then solution to any task T drawn from p(T ) can be found in just a few gradient steps. Here the goal is to find the find family of methods ω which correspond to the initial parameters of a neural network. Applications of this approach include few-shot learning where models can be trained using few examples without overfitting.

Optimizer-centric approaches

Here the goal is to learn the inner algorithm optimizer by training a function that takes as input optimization states such as θ and produces the optimization step for each base learning iteration.

Embedding Functions (Metric Learning)

In this case, embedding network ω transforms raw inputs into a representation compatible with recognition with the help of simple similarity comparison between query and support instances using cosine similarity or euclidean distance.

Black-Box Models (Recurrent, Convolutional, HyperNetwork)

These methods rather than relying on gradient iterative optimization of θ, train ω that provide a feed-forward mapping directly from the support set to the parameters required to classify test instances.

Hyper-parameters

In these methods optimizing ω represents optimizing hyperparameters of the base learning algorithm such as regularization strength, per-parameter regularization, task relatedness in multi-task learning, or sparsity strength in data cleansing

Losses and Auxiliary Tasks

Here the small neural network takes inputs to losses (e.g., predictions, features, or model parameters) and outputs a scalar quantity as a loss for the inner task optimizer.

Evolution

Evolution learning methods have found interesting applications where they can optimize any type of base model and meta objective with no requirement on differentiability. They do not rely on backpropagation, which takes care of both gradient degradation issues and minimizes the cost of high-order gradient computation as required by conventional gradient-based methods.

Reinforcement Learning

When base learner or the meta-objective is non-differentiable, reinforcement learning is used. This estimates the gradient of Loss function, typically using the policy gradient theorem.

Many vs Few-Shot Episode

Inner loop learning episodes are defined with many or few examples per-task depending upon whether the goal is to improve few- or many-shot performance.

Multi vs Single Task

Mutli Task: If the goal is to tune the learner to better solve any task drawn from a given family, then inner loop learning episodes correspond to a randomly drawn task from a family of tasks.

Simple Task: If the goal is to tune the learner to simply solve one specific task better, then the inner loop learning episodes all draw data from the same underlying task.

Fast Adaptation vs Asymptotic Performance

Asymptomatic Performance: Validation loss is computed at the end of the inner learning episode, and this leads to better final performance of the base task.

Fast Adaptation: Validation loss is computed as the sum of the validation loss after each inner optimization step, and this leads to faster learning in the base task.

Applications

Computer Vision and Graphics

Meta-learning-based on few-shot learning methods train algorithms that enable powerful deep networks to successfully learn on small datasets. Applications include Object Detection, Landmark Prediction, Object Segmentation, Image Generation, Video Synthesis, Density Estimation.

Meta Reinforcement Learning and Robotics

Reinforcement learning is typically concerned with learning control policies that enable an agent to obtain high reward in achieving a sequential action task within an environment. Meta learning has proved to be effective in enabling Reinforcement learning.

Meta-knowledge for the ability to stand up for a humanoid robot is a transferable skill for all family of tasks that require locomotion, while meta-knowledge of a maze layout is transferable skill for all navigation tasks in a maze.

Environment Learning and Sim2Real

In Sim2Real we are interested in training a simulation a model that is able to generalize the real-world.

Meta learning algorithms are particularly useful where the inner-level optimization learns a simulation model, the outer-level optimization algorithm evaluates the model’s performance in the real-world, and the meta-representation ω corresponds to the parameters of the simulation environment.

Neural Architecture Search

Architecture search involves finding ω which represents optimal architecture of a neural network. While the outer optimization searches for architectures with good validation performance, the inner optimization trains networks with the architecture found by outer algorithm.

Bayesian Meta Learning

Rather than direct learning of optimization parameters, Bayesian Hierarchical modeling and Bayesian inference emphasizes on learning by inference. Also, it provides uncertainty measures for the θ parameters, and hence measures of prediction uncertainty.

Continual, Online and Adaptive Learning

Continual: Meta-learning has been applied to improve continual learning in which human-like learning capability can be achieved where new tasks are learned better given past experience, without forgetting previously learned tasks, and without needing to store all past data for rehearsal against forgetting.

Online and Adaptive Learning: Online and Adaptive Learning also consider tasks arriving in a stream, but are concerned with the ability to effectively adapt to the current task in the stream, rather than remembering the old tasks.

Challenges

Meta-generalization in Meta-learning encounters a generalization challenge across family of tasks analogous to the challenge of generalising across instances in conventional machine learning.

Mutli-modality

Many meta learning algorithms assume unimodal task distribution whereas in reality the distribution is often multi-modal and single learning strategy may not give good results.

Computation Cost

Meta-learning algorithms often lead to a quadratic number of learning steps, since each outer step requires multiple inner steps. Moreover, there are a large number of inner steps in the case of many-shot experiments, and these need to be stored in memory. For this reason most meta-learning frameworks are extremely expensive in both time and memory resources, and often limited to small architectures.

Cross-modal transfer and heterogeneous tasks

To be able to able to extract knowledge from set of tasks each having its own modality and transfer it into another heterogeneous task with a unique modality is an open challenge and a very fertile research area.

Conclusion

There has been an exponential growth in meta learning algorithms and its easy to confuse one with its related fields. With the help of taxonomy and broad classification which focuses on functioning and goal, we are able to clearly identify and benchmark the performance of these algorithms and effectively evaluate their applications.

References

Meta-Learning in Neural Networks: A Survey

Learning to learn by gradient descent by gradient descent

From zero to research — An introduction to Meta-learning