Warning! Unsupervised Neuroscience Ahead
By: Mark Humphries
Tags: Machine Learning, Neuroscience, Artificial Intelligence, Science, Data Science

The warning: Care is needed. We are experts at finding patterns in noise, and so are our algorithms.

Take clustering. The problem with clustering is that it returns clusters. I mean, I know that’s what it’s supposed to do, but that’s actually the problem. Give a clustering algorithm the phone numbers of everyone in Llandudno, ask it to find four clusters in the data, and it will. Now you have four clusters of Welsh people, and are none the wiser as to what to do with them, and neither are they. The mere existence of clusters does not mean there is actual cluster structure in the data.

Unsupervised algorithms are about making sense of data for us, the observer. What they find does not have to coincide with reality. Reality doesn’t have a ground-truth, because it does not have clean, neat carves at the joints except in special cases. And most of those special cases are from artificial systems, with the joints built in. Run a different clustering algorithm on your data, and you’ll get different clusters; run a different dimension reduction algorithm on your data, and you’ll get different dimensions. As Ulrike von Luxburg and friends have so clearly argued, clustering is an art.

The unsupervised organisation of data is just a description of that data. Just because we can cluster neurons into groups doesn’t mean there exists actual meaningful groups of neurons in the brain; just because we can cluster behaviour into discrete elements — into states, motifs, syllables, or whatever term you prefer — doesn’t mean behaviour is actually discrete. To find that it means something, we have to link that discovered organisation to reality, show it has meaning. In neuroscience, that typically means we have to link that organisation of data to something happening in the world, or elsewhere in the brain, or both.

And that acid test is passed by the best attempts at unsupervised neuroscience. In the paper from Alon and friends, they showed the neurons’ “tuning” to the hippocampus’ internal dynamics had meaning as location in space (and repeated the same trick for neurons that code for head direction in the rodent thalamus). In the study from Adam Kepecs’ lab on the orbitofrontal cortex, their discrete groups of neurons in turn each encoded a meaningful variable in the decision-making process. Even better, they re-did the whole analysis with another cohort of animals with more neurons, reusing all parameters from first cohort, and ended up with the same results. These studies could show us a mapping between the unsupervised structure of the data and the real world.

Terrific work, but those are the “easy” ways of doing unsupervised neuroscience — by relating what we found to what we already know. We already know that hippocampus has place cells, and that there is a head direction system in the rodent thalamus. We already know the neurons in orbitofrontal cortex are heavily implicated in decision making, and to work out what their groups of neurons were encoding Kepec’s lab interpreted their activity as the variables within a mathematical model of decision making. If that model is wrong, the mapping between variables and activity is of little consequence in building our confidence that the clustered neurons are really there. And others, of course, may find different answers: when Anne Churchland’s group went looking for discrete groups of coding neurons in posterior parietal cortex, for example, they found none.

The ultimate test for unsupervised neuroscience is discoveries that could not be found any other way. There are some examples of that too. For example, we took large-scaling recordings of neurons in the motor system of the venerable sea-slug Aplysia. Using a fully-unsupervised pipeline to analyse that data we discovered its motor system was doubly discrete: on one level, groups of neurons with correlated activity were laid out contiguously in the motor system, beautifully tesselating the bit of brain they were in. On another level, mass populations of neurons with clearly different dynamics were in different parts of the motor system, including a distinct population of oscillating neurons in one spot that were most likely the pattern generating network for movement — a discovered hypothesis waiting to be tested. Joshua Vogelstein and friends discovered a richly detailed map of the relationship between neural activity and the resulting behaviour of Drosophila maggots, by individually stimulating each of 1054 types of neurons, videoing the behaviour, and clustering it. They thus revealed 29 different types of behaviour and which neurons drove each. But these are discoveries of relationships, of structure; they are not yet that final step of unsupervised discovery of a theory for how a bit of brain works.