Original: Neocortical plasticity: an unsupervised cake but no free lunch
Revised: Neocortical plasticity: an unsupervised cake but no free lunch

Original: Abstract
Revised: Abstract

Original: The ﬁelds of artiﬁcial intelligence and neuroscience have a long history of fertile bidirectional interactions.
Revised: The ﬁelds of artiﬁcial intelligence and neuroscience have a long history of fertile bidirectional interactions.

Original: On the one hand, important inspiration for the developmentof artiﬁcial intelligence systems has come from the study of natural systems ofintelligence, the mammalian neocortex in particular.
Revised: On the one hand, important inspiration for the development of artiﬁcial intelligence systems has come from the study of natural systems of intelligence, the mammalian neocortex in particular.

Original: On the other, importantinspiration for models and theories of the brain have emerged from artiﬁcialintelligence research.
Revised: On the other, important inspiration for models and theories of the brain have emerged from artiﬁcial intelligence research.

Original: A central question at the intersection of these two areas isconcerned with the processes by which neocortex learns, and the extent to whichthey are analogous to the back-propagation training algorithm of deep networks.
Revised: A central question at the intersection of these two areas is concerned with the processes by which neocortex learns, and the extent to which they are analogous to the back-propagation training algorithm of deep networks.

Original: Matching the data efﬁciency, transfer and generalisation properties of neocorticallearning remains an area of active research in the ﬁeld of deep learning.
Revised: Matching the data efﬁciency, transfer and generalization properties of neocortical learning remains an area of active research in the ﬁeld of deep learning.

Original: Recentadvances in our understanding of neuronal, synaptic and dendritic physiology of the neocortex suggest new approaches for unsupervised representation learning,perhaps through a new class of objective functions, which could act alongside or inlieu of back-propagation.
Revised: Recent advances in our understanding of neuronal, synaptic and dendritic physiology of the neocortex suggest new approaches for unsupervised representation learning, perhaps through a new class of objective functions, which could act alongside or in lieu of back-propagation.

Original: Such local learning rules have implicit rather than explicitobjectives with respect to the training data, facilitating domain adaptation andgeneralisation.
Revised: Such local learning rules have implicit rather than explicit objectives with respect to the training data, facilitating domain adaptation and generalization.

Original: Incorporating them into deep networks for representation learningcould better leverage unlabelled datasets to offer signiﬁcant improvements in dataefﬁciency of downstream supervised readout learning, and reduce susceptibility toadversarial perturbations, at the cost of a more restricted domain of applicability.
Revised: Incorporating them into deep networks for representation learning could better leverage unlabelled datasets to offer signiﬁcant improvements in data efﬁciency of downstream supervised readout learning, and reduce susceptibility to adversarial perturbations, at the cost of a more restricted domain of applicability.

Original: Unsupervised neocortex
Revised: Unsupervised neocortex

Original: The neocortex is the canonically 6-layered sheet of cells forming the grey matter surface of the mammalian cerebrum.
Revised: The neocortex is the canonically 6-layered sheet of cells forming the grey matter surface of the mammalian cerebrum.

Original: It is composed of a densely interconnected network of subregions responsiblefor learning sensory processing, speech and language, motor planning and many of the highercognitive processes associated with rational thought.
Revised: It is composed of a densely interconnected network of sub-regions responsible for learning sensory processing, speech and language, motor planning and many of the higher cognitive processes associated with rational thought.

Original: The human neocortex contains an estimated 100trillion synapses, the points of communication between neurons which undergo persistent changesin strength and topology as a function of signals local to the synapse and a complex biochemicalprogram (Holtmaat and Svoboda, 2009).
Revised: The human neocortex contains an estimatedtrillion synapses, the points of communication between neurons which undergo persistent changes in strength and topology as a function of signals local to the synapse and a complex biochemical program (Holtmaat and Svoboda, 2009).

Original: These processes, broadly known as synaptic plasticity, arethought to be the basis of learning and memory in the brain.
Revised: These processes, broadly known as synaptic plasticity, are thought to be the basis of learning and memory in the brain.

Original: An important task of synaptic plasticity in sensory neocortical areas is to learn disentangled invariantrepresentations (DiCarlo et al., 2012).
Revised: An important task of synaptic plasticity in sensory neocortical areas is to learn disentangled invariant representations (DiCarlo et al., 2012).

Original: For example, the ventral stream of primate visual cortex,the collection of areas responsible for visual object recognition, computes hierarchically organizedrepresentations much like state-of-the art convolutional neural networks (CNNs) optimized for thetask (Yamins et al., 2014).
Revised: For example, the ventral stream of primate visual cortex, the collection of areas responsible for visual object recognition, computes hierarchically organized representations much like state-of-the art convolutional neural networks (CNNs) optimized for the task (Yamins et al., 2014).

Original: While there are impressive similarities in the learned representations between the ventral stream and
Revised: While there are impressive similarities in the learned representations between the ventral stream and

Original: CNNs, there are important differences in how those representations are learned.
Revised: CNNs, there are important differences in how those representations are learned.

Original: While CNNs aretrained in a supervised manner using a gradient descent optimization algorithm with an explicit global objective on large labelled datasets, the ventral stream learns from a much larger dataset (visualexperience) but with only very sparse labelling.
Revised: While CNNs are trained in a supervised manner using a gradient descent optimization algorithm with an explicit global objective on large labelled datasets, the ventral stream learns from a much larger dataset (visual experience) but with only very sparse labelling.

Original: The latter property of cortical learning is attractive toemulate in CNNs, and more broadly across deep learning models.
Revised: The latter property of cortical learning is attractive to emulate in CNNs, and more broadly across deep learning models.

Original: Attractive, not only because of theability to make use of unlabelled data during learning, but also because it will impart the models withsuperior generalization and transfer properties, as discussed below.
Revised: Attractive, not only because of the ability to make use of unlabelled data during learning, but also because it will impart the models with superior generalization and transfer properties, as discussed below.

Original: The monkey’s paw effect: the problem with specifying what withoutspecifying how
Revised: The monkey’s paw effect: the problem with specifying what without specifying how

Original: A well known and often encountered pitfall of numerical optimization algorithms for high dimensionalproblems, such as evolutionary algorithms, simulated annealing and also gradient descent, is that theyregularly yielda solutions matchingto-the-letter what your objective speciﬁes, but far from how youintended (Lehman et al., 2018).
Revised: A well known and often encountered pitfall of numerical optimization algorithms for high dimensional problems, such as evolutionary algorithms, simulated annealing and also gradient descent, is that they regularly yield solutions matching what your objective speciﬁes to the letter, but far from how you intended (Lehman et al., 2018).

Original: The short story “The Monkey’s Paw” by W. W. Jacobs provides a compelling metaphor.
Revised: The short story “The Monkey’s Paw” by W. W. Jacobs provides a compelling metaphor.

Original: In thatstory, the new owner of  magical mummiﬁed monkey’s paw of Indian origin is granted three wishes.
Revised: In that story, the new owner of a magical mummiﬁed monkey’s paw of Indian origin is granted three wishes.

Original: The owner ﬁrst wishes for $200, and his wish is eventually granted to the penny, but with the graveside effect that it is granted through a goodwill payment from his son’s employer in response to hisuntimely death in a terrible machinery accident (Jacobs and Parker, 1910).
Revised: The owner ﬁrst wishes for $200, and his wish is eventually granted to the penny, but with the grave side effect that it is granted through a goodwill payment from his son’s employer in response to his untimely death in a terrible machinery accident (Jacobs and Parker, 1910).

Original: The Monkey’s Paw effect is also applicable to gradient descent-based optimization of deep neuralnets.
Revised: The Monkey’s Paw effect is also applicable to gradient descent-based optimization of deep neural nets.

Original: The relative data-hungriness of current supervised learning strategies, and the use of dataaugmentation to improve generalization reﬂect the precarious position we are in of needing tomicromanage the learning processes.
Revised: The relative data-hungriness of current supervised learning strategies, and the use of data augmentation to improve generalization reﬂect the precarious position we are in of needing to micromanage the learning processes.

Original: Adversarial examples (Moosavi-Dezfooli et al., 2016) are evidence that the monkey’s paw effectnone-the-less persists.
Revised: Adversarial examples (Moosavi-Dezfooli et al., 2016) are evidence that the monkey’s paw effect none-the-less persists.

Original: It is temping to continue with the current paradigm and re-inject adversarialexamples back into the learning data stream.
Revised: It is temping to continue with the current paradigm and re-inject adversarial examples back into the learning data stream.

Original: Extrapolating, this goes in the direction of specifying thenegative space of the objective, all those things the optimization should not do to solve the problem,which is potentially inﬁnite, and rather risky in production environments like self-driving cars.
Revised: Extrapolating, this goes in the direction of specifying the negative space of the objective, all those things the optimization should not do to solve the problem, which is potentially inﬁnite, and rather risky in production environments like self-driving cars.

Original: Adversarial examples represent an opportunity to address the issue in a more fundamental way(Yamins and DiCarlo, 2016).
Revised: Adversarial examples represent an opportunity to address the issue in a more fundamental way (Yamins and DiCarlo, 2016).

Original: It has been argued by Bengio (2012) that if we could design deeplearning systems with the explicit objective of “disentangling the underlying factors of variation” inan unsupervised manner, then there is much to be gained for generalization and transfer.
Revised: It has been argued by Bengio (2012) that if we could design deep learning systems with the explicit objective of “disentangling the underlying factors of variation” in an unsupervised manner, then there is much to be gained for generalization and transfer.

Original: Such an approach offers a promising solution to the Monkey’s Paw effect, as there is an explicitobjective of learning good representations, from which generalization and transfer follow by deﬁnition.
Revised: Such an approach offers a promising solution to the Monkey’s Paw effect, as there is an explicit objective of learning good representations, from which generalization and transfer follow by deﬁnition.

Original: 1 One small challenge remains: how to express the objective of learning good representations?
Revised: 1 One small challenge remains: how to express the objective of learning good representations?

Original: If we restrict ourselves to the subset of all possible inputs for which the neocortex learns goodrepresentations, the local processes of synaptic plasticity may provide valuable clues.
Revised: If we restrict ourselves to the subset of all possible inputs for which the neocortex learns good representations, the local processes of synaptic plasticity may provide valuable clues.

Original: Neocortical plasticity
Revised: Neocortical plasticity

Original: The neocognitron model (Fukushima, 1980), the original CNN architecture, learned visual featuresthrough self-organization using local rules.
Revised: The neocognitron model (Fukushima, 1980), the original CNN architecture, learned visual features through self-organization using local rules.

Original: Since its conception, our understanding of the neocortexand its neurons and synapses has progressed considerably.
Revised: Since its conception, our understanding of the neocortex and its neurons and synapses has progressed considerably.

Original: Recent insights into the local plasticity rules for learning in the neocortex offer new inspirationfor deep representation learning paradigms that learn “disentangled representations” from largeunlabelled datasets in an unsupervised manner.
Revised: Recent insights into the local plasticity rules for learning in the neocortex offer new inspiration for deep representation learning paradigms that learn “disentangled representations” from large unlabelled datasets in an unsupervised manner.

Original: A selection of recent insights into the systems ofplasticity of the neocortex is shown in Fig.
Revised: A selection of recent insights into the systems of plasticity of the neocortex is shown in Fig.

Original: A new dendrite-centric view of synaptic plasticity isemerging with the discovery of the NMDA spike, a non-linear mechanism hypothesized to associateco-activated synapses through potentiation or structural changes driven by the resulting calciumcurrents (Schiller et al., 2000; Graupner and Brunel, 2010; Holtmaat and Svoboda, 2009) (Fig.
Revised: A new dendrite-centric view of synaptic plasticity is emerging with the discovery of the NMDA spike a non-linear mechanism hypothesized to associate co-activated synapses through potentiation or structural changes driven by the resulting calcium currents (Schiller et al., 2000; Graupner and Brunel, 2010; Holtmaat and Svoboda, 2009) (Fig.

Original: 1A-B).
Revised: 1A-B).

Original: Such associations, in the form of co-coding clusters of synapses, have recently been experimentallyobserved using optical techniques (Wilson et al., 2016)
Revised: Such associations, in the form of co-coding clusters of synapses, have recently been experimentally observed using optical techniques (Wilson et al., 2016)

Original: (Fig.
Revised: (Fig.

Original: 1C).
Revised: 1C).

Original: Moreover neurons in the neocortexare known to form small cliques of all-to-all connected neurons which drive co-coding (Reimann et al.,2017), a process that would be self-reinforced through dendritic clustering by NMDA spikes (Fig.
Revised: Moreover neurons in the neocortex are known to form small cliques of all-to-all connected neurons which drive co-coding (Reimann et al., 2017), a process that would be self-reinforced through dendritic clustering by NMDA spikes (Fig.

Original: 1D).
Revised: 1D).

Original: Martinotti neurons, which are activated by such cliques of pyramidal neurons, and subsequentlyinhibit pyramidal dendrites (Silberberg and Markram, 2007) provide well-timed inhibition to blockfurther NMDA spikes(Doron et al., 2017), and put a limit on the maximal pyramidal clique size,but also suppress activation of competing cliques (e g. WTA dynamics).
Revised: Martinotti neurons, which are activated by such cliques of pyramidal neurons and subsequently inhibit pyramidal dendrites (Silberberg and Markram, 2007) provide well-timed inhibition to block further NMDA spikes (Doron et al., 2017) and put a limit on the maximal pyramidal clique size, but also suppress activation of competing cliques (e.g. Winner-take-all (WTA) dynamics).

Original: Together, such plasticitymechanisms appear to form basic building blocks for representation learning in the feed-forwardpathway of the neocortex using local learning rules.
Revised: Together, such plasticity mechanisms appear to form basic building blocks for representation learning in the feedforward pathway of the neocortex using local learning rules.

Original: However, working models have yet to beproposed.
Revised: While long known competitive strategies for unsupervised representation learning indeed rely on WTA dynamics (Fukushima, 1980; Rumelhart and Zipser, 1985), deep learning approaches incorporating these increasingly apparent dendritic dimensions of learning processes have yet to be proposed (Poirazi and Mel, 2001; Kastellakis et al., 2015).

Original: Unlike CNNs, the neocortex also has a prominent feedback pathway down the hierarchy, wherebytop-down input from upper layers innervate the apical tufts of pyramidal cells in layer 1 of a givencortical region (Felleman and Van, 1991).
Revised: Unlike CNNs, the neocortex also has a prominent feedback pathway down the hierarchy, whereby topdown input from upper layers innervate the apical tufts of pyramidal cells in layer 1 of a given cortical region (Felleman and Van, 1991).

Original: Associations between top-down and feed-forward (bottomup) activation are known to trigger dendritic calcium spikes and dendritic bursting (Larkum et al. ,1999), which again speciﬁcally activates the WTA dynamics of the Martinotti neurons (Murayamaet al., 2009).
Revised: Associations between top-down and feed-forward (bottom-up) activation are known to trigger dendritic calcium spikes and dendritic bursting (Larkum et al., 1999), which again speciﬁcally activates the WTA dynamics of the Martinotti neurons (Murayama et al., 2009), but disinhibitory VIP neurons can also modulate their impact (Karnani et al., 2016).

Original: These feed-back pathways have been proposed to implement predictive coding (Raoand Ballard, 1999), and error back-propagation for supervised learning algorithms (Guerguiev et al. ,2017; Sacramento et al., 2018).
Revised: These feed-back pathways have been proposed to implement predictive coding (Rao and Ballard, 1999), and error back-propagation for supervised learning algorithms (Guerguiev et al., 2017; Sacramento et al., 2018).

Original: While their importance for rapid object recognition has been recentlydemonstrated, their computational role remained inconclusive (Kar et al., 2019).
Revised: While their importance for rapid object recognition has been recently demonstrated, their computational role remained inconclusive (Kar et al., 2019).

Original: Cake but no free lunch
Revised: Cake but no free lunch

Original: With the demonstrated applicability of supervised learning for a broad range of problems and datadistributions, and an ever expanding toolbox of optimized software libraries, it is unlikely thatsupervised learning, back-propagation and gradient descent will be dethroned as the work horses of
Revised: With the demonstrated applicability of supervised learning for a broad range of problems and data distributions, and an ever expanding toolbox of optimized software libraries, it is unlikely that supervised learning, back-propagation and gradient descent will be dethroned as the work horses of

Original: AI for many years to come.
Revised: AI for many years to come.

Original: Nonetheless, as applications of deep networks are moving into regions where sparse data, generalization and transfer are increasingly important, unsupervised approaches designed with the explicitgoal of learning good representations from mere observation may ﬁnd an important place in the AIecosystem.
Revised: Nonetheless, as applications of deep networks are moving into regions where sparse data, generalization and transfer are increasingly important, unsupervised approaches designed with the explicit goal of learning good representations from mere observation may ﬁnd an important place in the AI ecosystem.

Original: Quoting Yann LeCun 2“If intelligence is a cake, the bulk of the cake is unsupervised learning, the icingon the cake is supervised learning, and the cherry on the cake is reinforcementlearning.”
Revised: Quoting Yann LeCun“If intelligence is a cake, the bulk of the cake is unsupervised learning, the icing on the cake is supervised learning, and the cherry on the cake is reinforcement learning.”

Original: A promising strategy would be to assume learning with sparse labels, overcoming adversarialexamples, transfer learning, and few-shot learning together as the success criteria for the furtherdevelopment of the powerful unsupervised approaches we seek.
Revised: A promising strategy would be to assume learning with sparse labels, overcoming adversarial examples, transfer learning, and few-shot learning together as the success criteria for the further development of the powerful unsupervised approaches we seek.

Original: Recent advances in our understanding of the processes of neocortical plasticity may well offer usefulinspiration, but let’s close with some words of moderation.
Revised: Recent advances in our understanding of the processes of neocortical plasticity may well offer useful inspiration, but let’s close with some words of moderation.

Original: Biology’s solutions also shows us therewill be no free lunch, i.e. neocortical unsupervised learning algorithms will be less general thansupervised learning by gradient descent.
Revised: Biology’s solutions also show us there will be no free lunch, i.e. neocortical unsupervised learning algorithms will be less general than supervised learning by gradient descent.

Original: Neocortex relies on structure at speciﬁc spatial and temporalscales in its input streams to learn representations.
Revised: Neocortex relies on structure at speciﬁc spatial and temporal scales in its input streams to learn representations.

Original: Evolution has had millions of years to conﬁgurethe sensory organs to provide signals to the neocortex in ways that it can make sense of them, andthat serve the animal’s ecological niche.
Revised: Evolution has had millions of years to conﬁgure the sensory organs to provide signals to the neocortex in ways that it can make sense of them, and that serve the animal’s ecological niche.

Original: We should not expect, for example, cortical unsupervisedlearning algorithms to cluster frozen white noise images.
Revised: We should not expect, for example, cortical unsupervised learning algorithms to cluster frozen white noise images.

Original: If we are to successfully take inspirationfrom biology, we must also work within its limitations. 
Revised: A neocortical solution requires a neocortical problem (e g. from the so-called “Brain set” (Richards et al., 2019)), so if we are to successfully take inspiration from it, we must also work within its limitations. Acknowledgments

Original: A neocortical solution requires a neocorticalproblem!
Revised: Thanks to Giuseppe Chindemi, Perouz Taslakian, Pau Rodriguez, Isabeau Prémont-Schwarz, Hector Palacios Verdes, Pierre-André Noël, Nicolas Chapados, Blake Richards and Guillaume Lajoie for helpful discussions.