The gradient clusteron: A model neuron that learns to solve classification tasks via dendritic nonlinearities, structural plasticity, and gradient descent
Authors: Toviah Moldwin, Menachem Kalmenson, Idan Segev
Journal: PLOS Computational Biology
Accepted: 2021-04-28T00:00:00Z


Synaptic clustering on neuronal dendrites has been hypothesized to play an important role in implementing pattern recognition. Neighboring synapses on a dendritic branch can interact in a synergistic, cooperative manner via nonlinear voltage-dependent mechanisms, such as NMDA receptors. Inspired by the NMDA receptor, the single-branch clusteron learning algorithm takes advantage of location-dependent multiplicative nonlinearities to solve classification tasks by randomly shuffling the locations of “under-performing” synapses on a model dendrite during learning (“structural plasticity”), eventually resulting in synapses with correlated activity being placed next to each other on the dendrite. We propose an alternative model, the gradient clusteron, or G-clusteron, which uses an analytically-derived gradient descent rule where synapses are "attracted to" or "repelled from" each other in an input- and location-dependent manner. We demonstrate the classification ability of this algorithm by testing it on the MNIST handwritten digit dataset and show that, when using a softmax activation function, the accuracy of the G-clusteron on the all-versus-all MNIST task (~85%) approaches that of logistic regression (~93%). In addition to the location update rule, we also derive a learning rule for the synaptic weights of the G-clusteron (“functional plasticity”) and show that a G-clusteron that utilizes the weight update rule can achieve ~89% accuracy on the MNIST task. We also show that a G-clusteron with both the weight and location update rules can learn to solve the XOR problem from arbitrary initial conditions.
Author summary: Artificial neural networks (ANNs) have become among the most powerful tools in artificial intelligence and machine learning, enabling computers to solve complex tasks like image recognition. Inspired by the brain, ANNs are composed of simple neuron-like units that perform a weighted sum of their synaptic inputs. Artificial neurons can “learn” by modifying the weight of each input. Biological neurons, however, are more complex. Synapses in a real neuron can interact nonlinearly with each other in a distance-dependent manner due to voltage-dependent mechanisms, such as the NMDA receptor. We created a model neuron, called the Gradient clusteron (G-clusteron) which can use distance-dependent nonlinearities to learn to solve classification tasks by making its synapses attract or repel each other. We tested the G-clusteron on its ability to recognize handwritten digits from the MNIST dataset and showed that it can achieve a high level of classification accuracy (~85%) just by updating its synaptic locations. We also derive a rule for updating the synaptic weights of the G-clusteron and show that having the ability to update both its synaptic weights and synaptic locations allows the G-clusteron to solve the “exclusive or” (XOR) problem, which is famously impossible for a linear neuron.