Learning to Learn like the Human Brain
By: L Éclairé
Tags: Research, Machine Learning, Artificial Intelligence, Data Science, Neuroscience

This post has been written with equal contribution by the authors of éclairé, Rishika Bharti and Jai Kotia.

The post explains an idea borrowed from an evolutionary mechanism-Synaptic Plasticity that has recently been used in the field of deep learning and has either outperformed former models or given competitive results in various tasks such as complex pattern memorisation, one-shot learning and reinforcement learning.

Important Terms

Background

As we all know, throughout our lives we are shaped by our experiences. This is not only true for our behaviour but also for the way our brain functions, from a neuroscience perspective. The essence of this concept is best captured by Canadian psychologist Donald Hebb’s famous postulate which colloquially states that:

“neurons that fire together, wire together.”

It simply means that if two neurons are repeatedly activated together, the pathway between them is strengthened making it more likely to get triggered in the future. The opposite is true for neurons that fire apart in an uncoordinated manner weakening their junctions, also known as synapses which are responsible for selectively storing memories for a long-term.

Synaptic Plasticity

To put it simply, plasticity is flexibility or being shaped or moulded. Synaptic plasticity is the ability of the connections between the neurons in the brain to strengthen or weaken over time based on various cognitive tasks performed by the brain.

It is said that synaptic plasticity is the necessary and sufficient element for memory formation and it’s encoding and tracing of its various types within the brain. It contributes a great deal to long-term learning within the brain.

Self-Directed or Autonomous Learning

Learning from ongoing experiences, at one’s own rate is called self-directed learning. In this the learner has the power to regulate and control their learning activities. Biological agents like animals learn the path to nearby food and water sources and humans adopt particular mannerisms over the course of time in this manner. This helps in quicker and a more efficient learning in unpredictable situations.

When introduced in deep learning networks, autonomous learning can give these networks the power to deal with new features that we not encountered during training.

Simplified concept behind the paper

To implement synaptic plasticity into neural network, the paper proposes the concept of Differentiable Plasiticity. Similar to how synapses develop in the brain, the neurons in the computation graph are to be modelled by assigning a variable weight to each of them.

This concept revolves around the following parameters:

Between two neurons i and j in the neural network, w(i, j) represents the fixed non-plastic weight component which is exactly the same as that used in traditional back-propagation algorithms. This is taken as the baseline weight for a particular connection.

The Hebbian trace Hebb(i, j) stores the non-plastic component of the connection. This component varies throughout the lifetime of the learning process and is varied at fixed intervals. It depends on the current inputs and outputs through the neuron. It’s computation depends upon the exact nature of the problem that we are looking at.

The plasticity coefficient α(i, j) determines the degree of plasticity of a connection. It can be fixed for all connections in a model or different.

So in this paper, it is proposed that the connections between the neurons be a sum of the weight and the Hebbian trace. The new equation is given by:

Note: A connection with no plasticity can be designed by setting α=0 and a completely plastic connection with w=0.

The drawback of the above equation is that η appears as a weight decay term. This means that its value will go down to zero in absence of input for a prolonged period of time. To overcome this, Oja’s rule is incorporated into the equation as:

This concept was applied and tested in various tasks by the authors with promising results. The tasks performed were:

Recurrent Networks: Pattern recognition and memorization

Meta Learning: One-shot classification

Reinforcement Learning: Exploration task

Effectively, through this implementation the network achieves long-term learning capacity. The synaptic plasticity ensures that the connections between neurons tend to favour long-term task success, and are not too influenced by short-term variations.

Why is this paper significant?

Standard or non-plastic networks are designed in such a way that they perform their best at a particular complex task when trained extensively with large amounts of data. But the downside is that after training, the understanding that these networks have is permanent. If the agent is required to learn a different task after this point, it must undergo training all over again with plenty of examples. Also, it will not be able to retain the only task once it is trained for a new one. This phenomenon is known as catastrophic forgetting where the network has an extremely short memory lifetime and inevitably forgets.

For the deep learning models to adapt to data changes over domains through the course of learning, it must overcome this hurdle. The aforementioned paper provides a work around for just that. Owing to its promising results in a wide range of tasks, the authors have claimed that this concept could possibly be a solution for the learning-to-learn problem.

Link to paper: https://arxiv.org/pdf/1905.08233