Source: https://www.nature.com/articles/nrn.2018.6?error=cookies_not_supported&code=422eba42-0b65-4b48-9563-a653538bae84
Timestamp: 2019-04-24 02:38:47+00:00

Document:
Humans and many other animals have an enormous capacity to learn about sensory stimuli and to master new skills. However, many of the mechanisms that enable us to learn remain to be understood. One of the greatest challenges of systems neuroscience is to explain how synaptic connections change to support maximally adaptive behaviour. Here, we provide an overview of factors that determine the change in the strength of synapses, with a focus on synaptic plasticity in sensory cortices. We review the influence of neuromodulators and feedback connections in synaptic plasticity and suggest a specific framework in which these factors can interact to improve the functioning of the entire network.
Sutton, R. S. & Barto, A. G. Reinforcement Learning (MIT Press, 1998).
Littman, M. L. Reinforcement learning improves behaviour from evaluative feedback . Nature 521, 445–451 (2015).
Rumelhart, D. E., Hinton, G. E. & Williams, R. J. in Parallel Distributed Processing: Explorations in the Microstructure of Cognition (eds Rumelhart, D. E. & McClelland, J. L.) 318–364 (MIT Press, 1986).
Hebb, D. O. The Organization of Behavior. A Neuropsychological Theory (John Wiley & Sons, 1949).
Martin, S. J., Grimwood, P. D. & Morris, R. G. M. Synaptic plasticity and memory: an evaluation of the hypothesis . Annu. Rev. Neurosci. 23, 649–711 (2000).
Schultz, W. Getting formal with dopamine and reward . Neuron 36, 241–263 (2002).
Niv, Y. & Schoenbaum, G. Dialogues on prediction errors . Trends Cogn. Sci. 12, 265–272 (2008).
Baxter, J. & Bartlett, P. L. Infinite-horizon policy-gradient estimation . J. Artif. Intell. Res. 15, 319–350 (2001).
Frémaux, N., Sprekeler, H. & Gerstner, W. Reinforcement learning using a continuous time actor-critic framework with spiking neurons . PLoS Comput. Biol. 9, e1003024 (2013).
Izhikevich, E. M. Solving the distal reward problem through linkage of STDP and dopamine signalling . Cereb. Cortex 17, 2443–2452 (2007).
Legenstein, R., Pecevski, D. & Maass, W. A learning theory for reward-modulated spike-timing-dependent plasticity with application in biofeedback . PLoS Comput. Biol. 4, e1000180 (2008).
Rombouts, J. O., Bohte, S. M., Martinez-trujillo, J., Roelfsema, P. R. & Pieter, R. A learning rule that explains how rewards teach attention . Vis. Cogn. 23, 179–205 (2015).
Yagishita, S. et al. A critical time window for dopamine actions on the structural plasticity of dendritic spines . Science 345, 1616–1620 (2014).
Frey, U. & Morris, R. G. M. Synaptic tagging and long-term potentiation . Nature 385, 533–536 (1997).
Montague, P. R., Dayan, P., Person, C. & Sejnowski, T. J. Bee foraging in uncertain environments using predictive Hebbian learning . Nature 377, 725–728 (1995).
Lisman, J., Grace, A. A. & Duzel, E. A neoHebbian framework for episodic memory; role of dopamine-dependent late LTP . Trends Neurosci. 34, 536–547 (2011).
Frémaux, N. & Gerstner, W. Neuromodulated spike-timing-dependent plasticity, and theory of three-factor learning rules . Front. Neural Circuits 9, 85 (2016).
Pennartz, C. A. M. The ascending neuromodulatory systems in learning by reinforcement: comparing computational conjectures with experimental findings . Brain Res. Rev. 21, 219–245 (1995).
Trabasso, T. & Bower, G. H. Attention in Learning: Theory and Research (Krieger Pub. Co., 1968).
Ahissar, M. & Hochstein, S. Attentional control of early perceptual learning . Proc. Natl Acad. Sci. USA 90, 5718–5722 (1993).
Jiang, Y. & Chun, M. M. Selective attention modulates implicit learning . Q. J. Exp. Psychol. 54, 1105–1124 (2001).
Vartak, D., Jeurissen, D., Self, M. W. & Roelfsema, P. R. The influence of attention and reward on the learning of stimulus-response associations . Sci. Rep. 7, 9036 (2017).
Seitz, A. R., Kim, D. & Watanabe, T. Rewards evoked learning of unconsciously processed visual stimuli in adult humans . Neuron 61, 700–707 (2009).
Roelfsema, P. R., van Ooyen, A. & Watanabe, T. Perceptual learning rules based on reinforcers and attention . Trends Cogn. Sci. 14, 64–71 (2010).
Jonikaitis, D. & Deubel, H. Independent allocation of attention to eye and hand targets in coordinated eye-hand movements . Psychol. Sci. 22, 339–347 (2011).
Moore, T. Shape representations and visual guidance of saccadic eye movements . Science 285, 1914–1917 (1999).
Roelfsema, P. R. & van Ooyen, A. Attention-gated reinforcement learning of internal representations for classification . Neural Comp. 17, 2176–2214 (2005).
Lamme, V. A. F. & Roelfsema, P. R. The distinct modes of vision offered by feedforward and recurrent processing . Trends Neurosci. 23, 571–579 (2000).
Moore, T. & Armstrong, K. M. Selective gating of visual signals by microstimulation of frontal cortex . Nature 421, 370–373 (2003).
Lillicrap, T. P., Cownden, D., Tweed, D. B. & Akerman, C. J. Random synaptic feedback weights support error backpropagation for deep learning . Nat. Commun. 7, 13276 (2016).
Rombouts, J. O., Bohte, S. M. & Roelfsema, P. R. How attention can create synaptic tags for the learning of working memories in sequential tasks . PLoS Comput. Biol. 11, e1004060 (2015).
Mnih, V. et al. Human-level control through deep reinforcement learning . Nature 518, 529–533 (2015).
Crick, F. The recent excitement about neural networks . Nature 337, 129–132 (1989).
Braitenberg, V. & Schütz, A. Anatomy of the Cortex (Springer-Verlag, 1991).
Mountcastle, V. B. in The Mindful Brain (eds Edelman, G. M. & Mountcastle, V. B.) (MIT Press, 1978).
Felleman, D. J. & Van Essen, D. C. Distributed hierarchical processing in the primate cerebral cortex . Cereb. Cortex 1, 1–47 (1991).
Yamins, D. L. K. & DiCarlo, J. J. Using goal-driven deep learning models to understand sensory cortex . Nat. Neurosci. 19, 356–365 (2016).
Sherman, S. M. Thalamus plays a central role in ongoing cortical functioning . Nat. Neurosci. 16, 533–541 (2016).
Callaway, E. M. Feedforward, feedback and inhibitory connections in primate visual cortex . Neural Networks 17, 625–632 (2004).
Harris, K. D. & Mrsic-Flogel, T. D. Cortical connectivity and sensory coding . Nature 503, 51–58 (2013).
Douglas, R. & Martin, K. A. C. Neuronal circuits of the neocortex . Annu. Rev. Neurosci. 27, 419–451 (2004).
Markov, N. T. et al. Cortical high-density counterstream architectures . Science 342, 1238406 (2013).
Ullman, S. Sequence seeking and counterstreams: a computational model for bidirectional information flow in the visual cortex . Cereb. Cortex 5, 1–11 (1995).
Harris, K. D. & Shepherd, G. M. G. The neocortical circuit: themes and variations . Nat. Neurosci. 18, 170–181 (2015).
Feldmeyer, D. Excitatory neuronal connectivity in the barrel cortex . Front. Neuroanat. 6, 24 (2012).
Maunsell, J. H. R. & Gibson, J. R. Visual response latencies in striate cortex of the macaque monkey . J. Neurophysiol. 68, 1332–1344 (1992).
Nowak, L. G., Munk, M. H. J., Girard, P. & Bullier, J. Visual latencies in areas V1 and V2 of the macaque monkey . Visual Neurosci. 12, 371–384 (1995).
van Kerkoerle, T., Self, M. W. & Roelfsema, P. R. Effects of attention and working memory in the different layers of monkey primary visual cortex . Nat. Commun. 8, 13804 (2017).
Self, M. W., van Kerkoerle, T., Supèr, H. & Roelfsema, P. R. Distinct roles of the cortical layers of area V1 in figure-ground segregation . Curr. Biol. 23, 2121–2129 (2013).
Constantinople, C. M. & Bruno, R. M. Deep cortical layers are activated by thalamus . Science 340, 1591–1594 (2013).
Morgenstern, N. A., Bourg, J. & Petreanu, L. Multilaminar networks of cortical neurons integrate common inputs from sensory thalamus . Nat. Neurosci. 19, 1034–1040 (2016).
Bolz, J. & Gilbert, C. D. Generation of end-inhibition in the visual cortex via interlaminar connections . Nature 320, 362–365 (1986).
Olsen, S. R., Bortone, D. S., Adesnik, H. & Scanziani, M. Gain control by layer six in cortical circuits of vision . Nature 483, 47–52 (2012).
Bortone, D. S., Olsen, S. R. & Scanziani, M. Translaminar inhibitory cells recruited by layer 6 corticothalamic neurons suppress visual cortex . Neuron 82, 474–485 (2014).
Rockland, K. S. & Virga, A. Terminal arbors of individual 'feedback' axons projecting from area V2 to V1 in the macaque monkey: a study using immunohistochochemistry of anterogradely transported Phaseolus vulgaris-leucoagglutinin . J. Comp. Neurol. 285, 54–72 (1989).
Larkum, M. E. A cellular mechanism for cortical associations: an organizing principle for the cerebral cortex . Trends Neurosci. 36, 141–149 (2013).
Schneider, D. M., Nelson, A. & Mooney, R. A synaptic and circuit basis for corollary discharge in the auditory cortex . Nature 513, 189–194 (2014).
van Versendaal, D. & Levelt, C. N. Inhibitory interneurons in visual cortical plasticity . Cell. Mol. Life Sci. 73, 3677–3691 (2016).
Deschênes, M., Veinante, P. & Zhang, Z. W. The organization of corticothalamic projections: reciprocity versus parity . Brain Res. Rev. 28, 286–308 (1998).
Veinante, P., Lavallée, P. & Deschênes, M. Corticothalamic projections from layer 5 of the vibrissal barrel cortex in the rat . J. Comp. Neurol. 424, 197–204 (2000).
Jones, E. G. Thalamus (Cambridge Univ. Press, 2007).
Roth, M. M. et al. Thalamic nuclei convey diverse contextual information to layer 1 of visual cortex . Nat. Neurosci. 19, 299–307 (2016).
Meyer, H. S. et al. Cell type-specific thalamic innervation in a column of rat vibrissal cortex . Cereb. Cortex 20, 2287–2303 (2010).
Ohno, S. et al. A morphological analysis of thalamocortical axon fibers of rat posterior thalamic nuclei: a single neuron tracing study with viral vectors . Cereb. Cortex 22, 2840–2857 (2012).
Lu, S. M. & Lin, R. C. Thalamic afferents of the rat barrel cortex: a light- and electron-microscopic study using Phaseolus vulgaris leucoagglutinin as an anterograde tracer . Somatosens. Mot. Res. 10, 1–16 (1993).
Mease, R. A., Metz, M. & Groh, A. Cortical sensory responses are enhanced by the higher-order thalamus . Cell Rep. 14, 208–215 (2016).
Groh, A. et al. Convergence of cortical and sensory driver inputs on single thalamocortical cells . Cereb. Cortex 24, 3167–3179 (2014).
Ahissar, E., Sosnik, R. & Haidarliu, S. Transformation from temporal to rate coding in a somatosensory thalamocortical pathway . Nature 406, 302–306 (2000).
Moore, J. D., Mercer Lindsay, N., Deschênes, M. & Kleinfeld, D. Vibrissa self-motion and touch are reliably encoded along the same somatosensory pathway from brainstem through thalamus . PLoS Biol. 13, e1002253 (2015).
Guo, Z. V. et al. Maintenance of persistent activity in a frontal thalamocortical loop . Nature 545, 181–186 (2017).
Kwon, S. E., Yang, H., Minamisawa, G. & O'Connor, D. H. Sensory and decision-related activity propagate in a cortical feedback loop during touch perception . Nat. Neurosci. 19, 1243–1249 (2016).
Self, M., Kooijmans, R. N., Supèr, H., Lamme, V. A. F. & Roelfsema, P. R. Different glutamate receptors convey feedforward and recurrent processing in macaque V1 . Proc. Natl Acad. Sci. USA 109, 11031–11036 (2012).
Daw, N. W., Stein, P. S. G. & Fox, K. The role of NMDA receptors in information processing . Annu. Rev. Neurosci. 16, 207–222 (1993).
Rivadulla, C., Martinez, L. M., Varela, C. & Cudeiro, J. Completing the corticofugal loop: a visual role for the corticogeniculate type 1 metabotropic glutamate receptor . J. Neurosci. 22, 2956–2962 (2002).
Gambino, F. et al. Sensory-evoked LTP driven by dendritic plateau potentials in vivo . Nature 515, 116–119 (2014).
Klink, P. C., Dagnino, B., Gariel-Mathis, M. A. & Roelfsema, P. R. Distinct feedforward and feedback effects of microstimulation in visual cortex reveal neural mechanisms of texture segregation . Neuron 95, 209–220 (2017).
Tritsch, N. X. & Sabatini, B. L. Dopaminergic modulation of synaptic transmission in cortex and striatum . Neuron 76, 33–50 (2012).
Picciotto, M. R., Higley, M. J. & Mineur, Y. S. Acetylcholine as a neuromodulator: cholinergic signaling shapes nervous system function and behavior . Neuron 76, 116–129 (2012).
Eggermann, E., Kremer, Y., Crochet, S. & Petersen, C. C. H. Cholinergic signals in mouse barrel cortex during active whisker sensing . Cell Rep. 9, 1654–1660 (2014).
Gu, Q. Neuromodulatory transmitter system in the cortex and their role in cortical plasticity . Neuroscience 111, 814–835 (2002).
Lesch, K. P. & Waider, J. Serotonin in the modulation of neural plasticity and networks: implications for neurodevelopmental disorders . Neuron 76, 175–191 (2012).
Froemke, R. C. Plasticity of cortical excitatory-inhibitory balance . Annu. Rev. Neurosci. 38, 195–219 (2015).
Hayashi-takagi, A. et al. Labelling and optical erasure of synaptic memory traces in the motor cortex . Nature 525, 333–338 (2015).
Xiong, Q., Znamenskiy, P. & Zador, A. M. Selective corticostriatal plasticity during acquisition of an auditory discrimination task . Nature 521, 348–351 (2015).
Jin, X. & Costa, R. M. Start/stop signals emerge in nigrostriatal circuits during sequence learning . Nature 466, 457–462 (2010).
Liu, B., Huberman, A. D. & Scanziani, M. Cortico-fugal output from visual cortex promotes plasticity of innate motor behaviour . Nature 538, 383–387 (2016).
Schoups, A., Vogels, R., Qian, N. & Orban, G. A. Practising orientation identification improves orientation coding in V1 neurons . Nature 412, 549–553 (2001).
Poort, J. et al. Learning enhances sensory and multiple non-sensory representations in primary visual cortex . Neuron 86, 1478–1490 (2015).
Goltstein, P. M., Coffey, E. B. J., Roelfsema, P. R. & Pennartz, C. M. A. In vivo two-photon Ca2+ imaging reveals selective reward effects on stimulus-specific assemblies in mouse visual cortex . J. Neurosci. 33, 11540–11555 (2013).
Freedman, D. J. & Assad, J. A. Experience dependent representation of visual categories in parietal cortex . Nature 443, 85–88 (2006).
Makino, H. & Komiyama, T. Learning enhances the relative impact of top-down processing in the visual cortex . Nat. Neurosci. 18, 1116–1122 (2015).
Bajo, V. M., Nodal, F. R., Moore, D. R. & King, A. J. The descending corticocollicular pathway mediates learning-induced auditory plasticity . Nat. Neurosci. 13, 253–260 (2010).
Brosch, T., Neumann, H. & Roelfsema, P. R. Reinforcement learning of linking and tracing contours in recurrent neural networks . PLoS Comput. Biol. 11, e1004489 (2015).
Pooresmaeili, A., Poort, J. & Roelfsema, P. R. Simultaneous selection by object-based attention in visual and frontal cortex . Proc. Natl Acad. Sci. USA 111, 6467–6472 (2014).
Self, M. W. et al. The effects of context and attention on spiking activity in human early visual cortex . PLoS Biol. 14, e1002420 (2016).
Houtkamp, R., Spekreijse, H. & Roelfsema, P. R. A gradual spread of attention during mental curve tracing . Percept. Psychophys. 65, 1136–1144 (2003).
Roelfsema, P. R. & Spekreijse, H. The representation of erroneously perceived stimuli in the primary visual cortex . Neuron 31, 853–863 (2001).
Khayat, P. S., Pooresmaeili, A. & Roelfsema, P. R. Time course of attentional modulation in the frontal eye field during curve tracing . J. Neurophysiol. 101, 1813–1822 (2009).
Roelfsema, P. R. Cortical algorithms for perceptual grouping . Annu. Rev. Neurosci. 29, 203–227 (2006).
Reynolds, J. H. & Chelazzi, L. Attentional modulation of visual processing . Annu. Rev. Neurosci. 27, 611–647 (2004).
Zhou, H., Schafer, R. J. & Desimone, R. Pulvinar–cortex interactions in vision and attention . Neuron 89, 209–220 (2016).
Purushothaman, G., Marion, R., Li, K. & Casagrande, V. A. Gating and control of primary visual cortex by pulvinar . Nat. Neurosci. 15, 905–912 (2012).
Robinson, D. L. & Petersen, S. E. The pulvinar and visual salience . Trends Neurosci. 15, 127–132 (1992).
Chalupa, L. M., Coyle, R. S. & Lindsley, D. B. Effect of pulvinar lesions on visual pattern discrimination in monkeys . J. Neurophysiol. 39, 354–369 (1976).
Manita, S. et al. A top-down cortical circuit for accurate sensory perception . Neuron 86, 1304–1316 (2015).
Xu, N. et al. Nonlinear dendritic integration of sensory and motor input during an active sensing task . Nature 492, 247–251 (2012).
Sachidhanandam, S., Sreenivasan, V., Kyriakatos, A., Kremer, Y. & Petersen, C. C. H. Membrane potential correlates of sensory perception in mouse barrel cortex . Nat. Neurosci. 16, 1671–1677 (2013).
Takahashi, N., Oertner, T. G., Hegemann, P. & Larkum, M. E. Active cortical dendrites modulate perception . Science 354, 1587–1590 (2016).
Gambino, F. & Holtmaat, A. Spike-timing-dependent potentiation of sensory surround in the somatosensory cortex is facilitated by deprivation-mediated disinhibition . Neuron 75, 490–502 (2012).
Letzkus, J. J. et al. A disinhibitory microcircuit for associative learning in the auditory cortex . Nature 480, 331–335 (2011).
Letzkus, J. J., Wolff, S. B. E. & Lüthi, A. Disinhibition, a circuit mechanism for associative learning and memory . Neuron 88, 264–276 (2015).
Lee, S., Hjerling-Leffler, J., Zagha, E., Fishell, G. & Rudy, B. The largest group of superficial neocortical GABAergic interneurons expresses ionotropic serotonin receptors . J. Neurosci. 30, 16796–16808 (2010).
Wall, N. R. et al. Brain-wide maps of synaptic input to cortical interneurons . J. Neurosci. 36, 4000–4009 (2016).
Audette, N. J., Urban-Ciecko, J., Matsushita, M. & Barth, A. L. POm thalamocortical input drives layer-specific microcircuits in somatosensory cortex . Cereb. Cortex https://doi.org/10.1093/cercor/bhx044 (2017).
Pi, H.-J. et al. Cortical interneurons that specialize in disinhibitory control . Nature 503, 521–524 (2013).
Fu, Y. et al. A cortical circuit for gain control by behavioral state . Cell 156, 1139–1152 (2014).
Pfeffer, C. K., Xue, M., He, M., Huang, Z. J. & Scanziani, M. Inhibition of inhibition in visual cortex: the logic of connections between molecularly distinct interneurons . Nat. Neurosci. 16, 1068–1076 (2013).
Wang, Y. et al. Anatomical, physiological and molecular properties of Martinotti cells in the somatosensory cortex of the juvenile rat . J. Physiol. 561, 65–90 (2004).
van Versendaal, D. et al. Elimination of inhibitory synapses is a major component of adult ocular dominance plasticity . Neuron 74, 374–383 (2012).
Kubota, Y., Hatada, S., Kondo, S., Karube, F. & Kawaguchi, Y. Neocortical inhibitory terminals innervate dendritic spines targeted by thalamocortical afferents . J. Neurosci. 27, 1139–1150 (2007).
Palmer, L., Murayama, M. & Larkum, M. Inhibitory regulation of dendritic activity in vivo . Front. Neural Circuits 6, 26 (2012).
Chen, S. X., Kim, A. N., Peters, A. J. & Komiyama, T. Subtype-specific plasticity of inhibitory circuits in motor cortex during motor learning . Nat. Neurosci. 18, 1109–1115 (2015).
Cichon, J. & Gan, W. Branch-specific dendritic Ca2+ spikes cause persistent synaptic plasticity . Nature 520, 180–185 (2015).
Fu, Y., Kaneko, M. K., Tang, Y., Alvarez-Buylla, A. & Stryker, M. P. A cortical disinhibitory circuit for enhancing adult plasticity . eLife 4, e05558 (2015).
Higley, M. J. Localized GABAergic inhibition of dendritic Ca2+ signalling . Nat. Rev. Neurosci. 15, 567–572 (2014).
Bittner, K. C., Milstein, A. D., Grienberger, C., Romani, S. & Magee, J. C. Behavioral time scale synaptic plasticity underlies CA1 place fields . Science 357, 1033–1036 (2017).
Basu, J. et al. Gating of hippocampal activity, plasticity, and memory by entorhinal cortex long-range inhibition . Science 351, aaa5694 (2016).
Smith, S. L., Smith, I. T., Branco, T. & Häusser, M. Dendritic spikes enhance stimulus selectivity in cortical neurons in vivo . Nature 503, 115–120 (2013).
Dahmen, J. C., Hartley, D. E. H. & King, A. J. Stimulus-timing-dependent plasticity of cortical frequency representation . J. Neurosci. 28, 13629–13639 (2008).
Pawlak, V. & Kerr, J. N. D. Dopamine receptor activation is required for corticostriatal spike-timing-dependent plasticity . J. Neurosci. 28, 2435–2446 (2008).
Pawlak, V., Greenberg, D. S., Sprekeler, H., Gerstner, W. & Kerr, J. N. D. Changing the responses of cortical neurons from sub- to suprathreshold using single spikes in vivo . eLife 2, e00012 (2013).
Haas, H. L., Sergeeva, O. A. & Selbach, O. Histamine in the central nervous system . Physiol. Rev. 88, 1183–1241 (2008).
Montague, P. R., Hyman, S. E. & Cohen, J. D. Computational roles for dopamine in behavioral control . Nature 431, 760–767 (2004).
Schultz, W. Multiple dopamine functions at different time courses . Annu. Rev. Neurosci. 30, 259–288 (2007).
Bromberg-Martin, E. S., Matsumoto, M. & Hikosaka, O. Dopamine in motivational control: rewarding, aversive, and alerting . Neuron 68, 815–834 (2010).
Bao, S., Chan, V. T. & Merzenich, M. M. Cortical remodelling induced by activity of ventral tegmental dopamine neurons . Nature 412, 79–81 (2001).
Zaborszky, L. et al. Neurons in the basal forebrain project to the cortex in a complex topographic organization that reflects corticocortical connectivity patterns: an experimental study based on retrograde tracing and 3D reconstruction . Cereb. Cortex 25, 118–137 (2015).
Kim, J.-H. et al. Selectivity of neuromodulatory projections from the basal forebrain and locus ceruleus to primary sensory cortices . J. Neurosci. 36, 5314–5327 (2016).
Kawai, H., Lazar, R. & Metherate, R. Nicotinic control of axon excitability regulates thalamocortical transmission . Nat. Neurosci. 10, 1168–1175 (2007).
Férézou, I. et al. 5-HT3 receptors mediate serotonergic fast synaptic excitation of neocortical vasoactive intestinal peptide/cholecystokinin interneurons . J. Neurosci. 22, 7389–7397 (2002).
Pinto, L. et al. Fast modulation of visual perception by basal forebrain cholinergic neurons . Nat. Neurosci. 16, 1857–1863 (2013).
Hangya, B., Ranade, S. P., Lorenc, M. & Kepecs, A. Central cholinergic neurons are rapidly recruited by reinforcement feedback . Cell 162, 1155–1168 (2015).
Richardson, R. T. & DeLong, M. R. Nucleus basalis of Meynert neuronal activity during a delayed response task in monkey . Brain Res. 399, 364–368 (1986).
Chubykin, A. A., Roach, E. B., Bear, M. F. & Shuler, M. G. H. A cholinergic mechanism for reward timing within primary visual cortex . Neuron 77, 723–735 (2013).
Kilgard, M. P. & Merzenich, M. M. Cortical map reorganization enabled by nucleus basalis activity . Science 279, 1714–1718 (1998).
Froemke, R. C. et al. Long-term modification of cortical synapses improves sensory perception . Nat. Neurosci. 16, 79–88 (2013).
Bakin, J. S. & Weinberger, N. M. Induction of a physiological memory in the cerebral cortex by stimulation of the nucleus basalis . Proc. Natl Acad. Sci. USA 93, 11219–11224 (1996).
Froemke, R. C., Merzenich, M. M. & Schreiner, C. E. A synaptic memory trace for cortical receptive field plasticity . Nature 450, 425–429 (2007).
Juliano, S. L., Ma, W. & Eslin, D. Cholinergic depletion prevents expansion of topographic maps in somatosensory cortex . Proc. Natl Acad. Sci. USA 88, 780–784 (1991).
Warburton, E. C. et al. Cholinergic neurotransmission is essential for perirhinal cortical plasticity and recognition memory . Neuron 38, 987–996 (2003).
Easton, A., Ridley, R. M., Baker, H. F. & Gaffan, D. Unilateral lesions of the cholinergic basal forebrain and fornix in one hemisphere and inferior temporal cortex in the opposite hemisphere produce severe learning impairements in rhesus monkeys . Cereb. Cortex 12, 729–736 (2002).
Winkler, J., Suhr, S. T., Gage, F. H., Thal, L. J. & Fisher, L. J. Essential role of neocortical acetylcholine in spatial memory . Nature 375, 484–487 (1995).
Jacobs, B. L. & Azmitia, E. C. Structure and function of the brain serotonin system . Physiol. Rev. 72, 165–229 (1992).
Celada, P., Puig, M. V. & Artigas, F. Serotonin modulation of cortical neurons and networks . Front. Integr. Neurosci. 7, 25 (2013).
Nakamura, K., Matsumoto, M. & Hikosaka, O. Reward-dependent modulation of neuronal activity in the primate dorsal raphe nucleus . J. Neurosci. 28, 5331–5343 (2008).
Ranade, S. P. & Mainen, Z. F. Transient firing of dorsal raphe neurons encodes diverse and specific sensory, motor, and reward events . J. Neurophysiol. 102, 3026–3037 (2009).
Cohen, J. Y., Amoroso, M. W. & Uchida, N. Serotonergic neurons signal reward and punishment on multiple timescales . eLife 4, e06346 (2015).
Bromberg-martin, E. S., Hikosaka, O. & Nakamura, K. Coding of task reward value in the dorsal raphe nucleus . J. Neurosci. 30, 6262–6272 (2010).
Liu, Z. et al. Dorsal raphe neurons signal reward through 5-HT and glutamate . Neuron 81, 1360–1374 (2014).
Nakamura, K. The role of the dorsal raphé nucleus in reward-seeking behavior . Front. Integr. Neurosci. 7, 60 (2013).
Jitsuki, S. et al. Serotonin mediates cross-modal reorganization of cortical circuits . Neuron 69, 780–792 (2011).
Sara, S. J. & Bouret, S. Orienting and reorienting: the locus coeruleus mediates cognition through arousal . Neuron 76, 130–141 (2012).
Bouret, S. & Richmond, B. J. Sensitivity of locus ceruleus neurons to reward value for goal-directed actions . J. Neurosci. 35, 4005–4014 (2015).
Martins, A. R. O. & Froemke, R. C. Coordinated forms of noradrenergic plasticity in the locus coeruleus and primary auditory cortex . Nat. Neurosci. 18, 1483–1492 (2015).
Devilbiss, D. M. & Waterhouse, B. D. Phasic and tonic patterns of locus coeruleus output differentially modulate sensory network function in the awake rat . J. Neurophysiol. 105, 69–87 (2011).
Pawlak, V., Wickens, J. R., Kirkwood, A. & Kerr, J. N. D. Timing is not everything: neuromodulation opens the STDP gate . Front. Synapt. Neurosci. 2, 146 (2010).
Hu, H. et al. Emotion enhances learning via norepinephrine regulation of AMPA-receptor trafficking . Cell 131, 160–173 (2007).
Johansen, J. P. et al. Hebbian and neuromodulatory mechanisms interact to trigger associative memory formation . Proc. Natl Acad. Sci. 111, E5584–E5592 (2014).
Urbanczik, R. & Senn, W. Reinforcement learning in populations of spiking neurons . Nat. Neurosci. 12, 250–252 (2009).
Seol, G. H. et al. Neuromodulators control the polarity of spike-timing-dependent plasticity . Neuron 55, 919–929 (2007).
He, K. et al. Distinct eligibility traces for LTP and LTD in cortical synapses . Neuron 88, 528–538 (2015).
Cassenaer, S. & Laurent, G. Conditional modulation of spike-timing-dependent plasticity for olfactory learning . Nature 482, 47–52 (2012).
Redondo, R. L. & Morris, R. G. M. Making memories last: the synaptic tagging and capture hypothesis . Nat. Rev. Neurosci. 12, 17–30 (2011).
Clopath, C., Ziegler, L., Vasilaki, E., Büsing, L. & Gerstner, W. Tag-trigger-consolidation: a model of early and late long-term-potentiation and depression . PLoS Comput. Biol. 4, e1000248 (2008).
Nadim, F. & Bucher, D. Neuromodulation of neurons and synapses . Curr. Opin. Neurobiol. 29, 48–56 (2014).
Blundon, J. A., Bayazitov, I. T. & Zakharenko, S. S. Presynaptic gating of postsynaptically expressed plasticity at mature thalamocortical synapses . J. Neurosci. 31, 16012–16025 (2011).
Brzosko, Z., Schultz, W. & Paulsen, O. Retroactive modulation of spike timing-dependent plasticity by dopamine . eLife 4, e09685 (2015).
Güçlü, U. & van Gerven, M. A. J. Deep neural networks reveal a gradient in the complexity of neural representations across the ventral stream . J. Neurosci. 35, 10005–10014 (2015).
Urbanczik, R. & Senn, W. Learning by the dendritic prediction of somatic spiking . Neuron 81, 521–528 (2014).
Schiess, M., Urbanczik, R. & Senn, W. Somato-dendritic synaptic plasticity and error-backpropagation in active dendrites . PLoS Comput. Biol. 12, e1004638 (2016).
Scellier, B. & Bengio, Y. Equilibrium propagation: bridging the gap between energy-based models and backpropagation . Front. Comput. Neurosci. 11, 24 (2017).
Marblestone, A., Wayne, G. & Kording, K. Towards an integration of deep learning and neuroscience . Front. Comput. Neurosci. 10, 94 (2016).
Laramée, M.-E. & Boire, D. Visual cortical areas of the mouse: comparison of parcellation and network structure with primates . Front. Neural Circuits 8, 149 (2015).
Berezovskii, V. K., Nassi, J. J. & Born, R. T. Segregation of feedforward and feedback projections in mouse visual cortex . J. Comp. Neurol. 519, 3672–3683 (2011).
Agnati, L. F., Guidolin, D., Guescini, M., Genedani, S. & Fuxe, K. Understanding wiring and volume transmission . Brain Res. Rev. 64, 137–159 (2010).
Knott, G. W., Quairiaux, C., Genoud, C. & Welker, E. Formation of dendritic spines with GABAergic synapses by whisker stimulation in induced adult mice . Neuron 34, 265–273 (2002).
The authors thank S. Bohte, C. Pennartz, M. Sherman, V. Kehayas and H. Kennedy for helpful input and comments. The work was supported by the Netherlands Organisation for Scientific Research (NWO; ALW grant 823-02-010 to P.R.R.), the European Union Seventh Framework Programme (grant agreement 7202070 'Human Brain Project' to P.R.R. and European Research Council (ERC) grant agreement 339490 'Cortic_al_gorithms' to P.R.R.), the Swiss National Science Foundation (SNF; research grants 31003A-153448 and CRSII3-154453 to A.H. and the National Centre of Competence in Research (NCCR) SYNAPSY grant 51NF40-158776 to A.H.) and the International Foundation for Research in Paraplegia (to A.H.).
Department of Vision and Cognition, Netherlands Institute for Neuroscience, Royal Netherlands Academy of Arts and Sciences, Amsterdam, Netherlands.
Department of Integrative Neurophysiology, Center for Neurogenomics and Cognitive Research, VU University, Amsterdam, Netherlands.
Psychiatry Department, Academic Medical Center, Amsterdam, Netherlands.
Department of Basic Neurosciences, Geneva Neuroscience Center, Faculty of Medicine, University of Geneva, Geneva, Switzerland.
P.R.R. and A.H. researched data for the article, made substantial contributions to discussions of the content, wrote the article and reviewed and/or edited the manuscript before submission.
Correspondence to Pieter R. Roelfsema.
(RPEs). Differences between the amount of reward that was expected and the amount that was obtained.
Trial-and-error learning when interacting with an environment and experiencing rewards and punishments as consequences of the chosen actions.
Local parameters at the synapses of a network that determine whether they undergo plasticity upon reward-prediction errors during reinforcement learning.
Biochemical signals at synapses that determine whether they will undergo plasticity.
A mathematical method used to calculate the contribution of connections to the error of a network with multiple layers between input and output.
The derivative of the error function to a synaptic weight is the rate of change of the error when changing the strength of a particular synapse.
A mathematical optimization method that determines the direction of the vector of changes in all synaptic weights that causes the largest decrease in the error of the network.
A property of an image processing system whereby the recognition of the object is independent of the object's location relative to the viewer.
A process in which, if the feedforward and feedback weights of a neural network are not reciprocal, error backpropagation causes feedforward weights to align; that is, to become more symmetrical.
The innate reflexive smooth eye movements elicited by large moving visual stimuli.
Area of the frontal cortex involved in the planning of eye movements.
Somatostatin-expressing inhibitory interneurons with a characteristic morphology that target the dendritic tufts of pyramidal cells in various cortical layers.
A type of learning in which the structure of unlabelled data is inferred as information about desired categorization is not provided.
(STDP). A plasticity rule whereby the change in the strength of synapses depends on the relative timing of presynaptic and postsynaptic action potentials.
Can neocortical feedback alter the sign of plasticity?

References: V. 
 V. 
 V. 
 V. 
 V. 
 V. 
 V. 
 V. 
 V. 
 V. 
 V. 
 V. 
 V. 
 V.