Using Information Gain for the Unsupervised Training of Excitatory Neurons
By: Lukas Molzberger
Tags: Machine Learning, Information Gain, Neural Networks, Unsupervised Learning, Recurrent Neural Network

Looking for a biologically more plausible way to train a neural network.

Traditionally, artificial neural networks have been trained using the Delta rule and backpropagation. But this contradicts the findings that the neurosciences have made on the function of the brain. There simply is no gradient error signal that is propagated backwards through biological neurons (see here and here). Besides, the human brain can find patterns in its audiovisual training data by itself without the need for training labels. When a parent shows a cat to a child, the child doesn’t use this information to learn every detail of what constitutes a cat, but only to associate a name with the concept of a cat. A deep neural network, on the other hand, requires thousands of images with and without cats, together with accurate training labels for these images. The deep neural network would then try to learn the concept of a cat in its output layer. But it would also try to learn lower-level features in its intermediate layers. The network could, for example, try to learn concepts like cat ears, whiskers, paws, cat eyes and so on to help the recognition of a whole cat. The problem, however, is that these lower-level concepts are learned in a very indirect and fuzzy fashion from the labels telling which of the images show a cat and which don’t. In my opinion, this isn’t a good way to learn these concepts. Probably a better way would be to learn these concepts directly from the data without relying on the output labels. If we take a closer look at the cat’s eye, we’ll see that there are many lower-level features, like the shape of the pupil, the shape of the eyelids, the context in which the eye occurs and so on, that co-occur and that make up the pattern of what an eye is. So why don’t we use this co-occurrence of features to learn the pattern constituting the eye?

If we go back to the very beginning of research in artificial neural networks, we’ll stumble across some very interesting ideas by Donald O. Hebb. In his book, ‘The Organization of Behavior’ Hebb suggests that the internal representation of a real-world object consists of cortex cells activated by the external stimulus. He calls this group of simultaneously activated neurons a cell assembly. Hebb imagines that all these cells are interconnected reciprocally. The internal representation of the object would remain in the short-term memory as long as the activity circulates through the circuits in the neuron cell assembly. Furthermore, he hypothesizes that if the activation of the neuron cell assembly only lasted long enough, it would lead to a consolidation through a ‘growth process’ through which the reciprocal circuits become more effective; neurons that fire together would also be wired together. Consequently, if only a fraction of the cells of the assembly is activated by a later stimulus, the now amplified reciprocal circuits would cause the entire cell assembly to become active again, recalling the entire internal representation of the external stimulus. His theory became known as Hebbian theory and the models that follow this theory are said to exhibit ‘Hebbian learning’. The equation for the Hebb rule is as follows where wi is the synapse weight, α is the learn rate, y is the output activation of the neuron and xi is the input activation of synapse i:

As we can easily see, this equation is overly simple and doesn’t lead to a stable training model, because the weights are only increased.

Excitatory and inhibitory neurons

So how could we arrive at a model of Hebbian learning that avoids this problem? First, we need to recognize that Hebbian learning only describes a learning mechanism for a certain type of neuron, namely the excitatory neuron. Hebbian learning is suitable for the learning of co-occurring features but isn’t applicable to the learning of categories. Excitatory neurons are conjunctive in nature. They allow the representation of patterns consisting of many individual features. Inhibitory neurons, on the other hand, are disjunctive in nature and can group excitatory neurons into categories. They’re called inhibitory neurons because they can form negative feedback loops that suppress excitatory neurons. The biological role models for excitatory and inhibitory neurons are the spiny pyramidal cell and the aspiny stellate cell of the cerebral cortex. The pyramidal cells usually exhibit an excitatory characteristic and some possess long-ranging axons that connect to other parts of the brain. The stellate cells, on the other hand, are usually inhibitory interneurons with short axons that form circuits with nearby neurons.

Information gain

As we’re only trying to train excitatory neurons, let’s take a step back and look at the problem from an information-theoretic perspective. If we view the neurons as data sources and the neuron activations as events, we can use the Shannon entropy to measure the average rate at which information is produced by neurons. (In our case k only iterates over the two options active and inactive.)

As an example consider a set of neurons, each representing a single letter where each neuron has its own entropy value. If we view these neurons as independent, we can simply add up the entropy values. But are these neurons really independent? Certainly not. There are the words in which a subset of these letter neurons are often activated together. So, we could probably reduce the overall entropy of the network by introducing excitatory neurons representing the words or other types of patterns. These excitatory neurons then compress the information contained in the letter neuron activations. In other words, we need less information to represent a word in a single-word neuron than we would need to represent it as individual-letter neurons. If we follow the goal to reduce the overall entropy of the network, we can probably use this information-theoretic framework to come up with a training rule for our excitatory neurons. To adjust the synapse weights of our excitatory neurons in such a way that reduces the overall entropy, we first need to define a cost function for our excitatory neuron that we can use during the optimization process. Let’s start to formulate such a cost function by looking at the relative entropy that is gained through a single synapse i of an excitatory neuron. This relative entropy is also called the Kullback-Leibler divergence and can be stated as follows:

Where Xi is the discrete random variable of the input neuron and Y is the discrete random variable of the output neuron. For the distribution Q, the joint probability distribution of Xi and Y is assumed to be independent. This is the reference distribution to which the relative entropy is calculated. So, if the distribution P(Xi,Y) is independent as well, then the information gain is 0.

So, the overall information gain we achieve for the neuron looks like this:

Since we’ve added a new excitatory neuron to the network to reduce the overall entropy, we need to take the entropy of this new neuron into account as well:

Another thing that we need to take care of is to only consider those synapses that are actually contributing to the compression. If an input synapse of the excitatory neuron has a weight of 0, it should also have no influence on the cost function. To achieve this, we’re using the coverage term covi to determine how strong the influence of the synapse i is on the output neuron. We’ll go into more detail on how covi is calculated later. Now our final cost function looks like this:

The model of the excitatory neuron

Before we show how the cost function is used to optimize the synapse weights, let’s take a closer look at the model of the excitatory neuron. The model is actually pretty straightforward in that it computes the weighted sum over all input synapses and uses a bias value to set the threshold.

The result is then sent through an activation function φ to compute the output activation value y.

The only difference between an excitatory neuron and an ordinary neuron is that the bias value of the former can be split as follows:

So, if all the inputs of the excitatory neuron are fully active, then net will be equal to b. The activation function φ we use is the positive half of the hyperbolic tangent function.

Its derivative, which we’ll need later, is given by:

Finally, the coverage term covi, which we’ve already mentioned, is calculated as follows:

This indicates how well the synapse i can suppress the activation of the excitatory neuron. If the synapse weight wi is equal to or larger than the bias b, then the covi value will be 1.

Counting frequencies

Since the calculation of the entropy relies on probability distributions, we should take a short walk through how these are computed. First, the frequencies are counted by summing up the input activations xd and output activations yd over all training instances d.

The probability distributions are then calculated by dividing the frequency by the number of training instances N.

However, there are two problems with this simple form of frequency counting. The first is that the sample might be too small to be reliable. In this case, the Dirichlet distribution might help to estimate how reliable the distribution is, but this is getting pretty complicated. The second problem is that the distribution is a moving target. Since we constantly adjust the weights of the excitatory neuron, this means that the probability distributions are also changing. This problem can be tackled by using a moving average to calculate the probabilities.

The derivative of the cost function

To get to a training rule with which the synapse weights can be adjusted, the derivation of the cost function must now be calculated. As a first step, we remove the information content parts of the equation and consider these as constants during the following steps:

Under the assumption that covi is constant for all but the current training instance d, we can derive the following outer derivation:

When it comes to the derivative of the neuron’s output value y′l, there are several variables that we might want to adjust. There are the synapse weights wj of all input synapses j and there is the bias. And when adjusting the synapse weight wj, we need to consider either b as constant or bc depending on whether we’re looking at the case of an active input synapse or an inactive one. Of course, one bias value must also change if the other is assumed to be constant. The case of an active input synapse looks like this:

And the case of an inactive input synapse looks like this:

The derivation of the bias value is given by this equation:

When considering the case of an active input synapse, we need to introduce an asymmetry. If the cost function of the excitatory neuron wants to increase the activity of the neuron, only the weight of the synapse is increased. This is the case of ‘what fires together wires together’. But if the cost function wants to decrease the activity of the neuron, only the bias value is decreased. This is the case when an input letter occurs outside the word represented by the excitatory neuron. In this case, we only want to reduce the activity of the neuron, without reducing the strength of the synapse.

The final training rule would then look like this, where α is the learn rate:

So in order to train the synapse j all the other input synapses i of the excitatory neuron are used to calculate the target direction in which to adjust the weight of the synapse j. The cov′i part of the equation G′ acts as a kind of bootstrapping mechanism for weak synapses. This part of the equation also prevents the bias from becoming too large, because if that happens, suddenly many synapses become weak synapses for which covi becomes active again. The derivation G′ also captures quite nicely the intuition, that frequently activated input synapses should be considered less relevant. There’s even the possibility to train the excitatory neurons in a supervised way by connecting negative feedback synapses as inputs.

Open questions

Some of the questions that are not answered so far are:

How can new excitatory neurons be induced?

To tackle this question we could start with a single mature input neuron and start looking at whether there are other neurons firing at the same time.

To tackle this question we could start with a single mature input neuron and start looking at whether there are other neurons firing at the same time. How can duplicates be prevented?

For this question, we’ll probably need the inhibitory neurons to form a feedback loop that can suppress potential duplicates.

For this question, we’ll probably need the inhibitory neurons to form a feedback loop that can suppress potential duplicates. What does the training of the inhibitory neurons look like?

The update rule described in this article only covers the training based on co-occurring features. But the training of the category-based inhibitory neurons requires an entirely different training approach. One possibility would be to cluster excitatory neurons based on shared properties.

Implementation and related work

An implementation of this training rule can be found in the ExcitatoryNeuron class of the Aika project on GitHUB. This article gives some context in which the excitatory neurons can be used. And finally, this article describes how excitatory neurons can be embedded into feedback loops.