The rst model of the basal ganglia (BG) was conceived almost half a century ago. Since then, extensive research e orts have been carried out to further re ne and understand the physiological and pathological BG behaviour and role. Currently, it is well-known that the BG are crucial in motor learning and motor diseases are associated to dysfunction of the nuclei, such as the parkinsonian syndrome, dystonia, chorea, etc. We are still a long way from giving an answer to all the questions, but advances in technology are making research advance signi cantly in recent years. Computational modeling is one of these methodologies and allows to evaluate the interactions within and among multiple neural systems. The development and analysis of the behaviour of computational models, in concert with multimodal analysis and in vivo experiments, leads to new scienti c results. This review provides a critical synopsis of the evolution of thought regarding the physiological model of the BG with respect to motor learning, revisiting past theories and summarizing the main recent ndings to this eld of research, to highlight their innovative contribution to knowledge of the functioning of the nuclei and formulate a state-of-art hypothesis of BG modeling. Introduction Many are the yet unanswered questions in neuroscience. Some of these unsolved problems have (today, now!) evidence supporting a hypothesized solution and the eld is rapidly evolving, albeit it is a very tall order. For others, solutions are decades away. And still others may never be solved. The current pace of advances in methods forces us to stop and consider, to ask where we are going and what we should research next. From a biological point of view, intelligent neural processes require the interaction between di erent levels of the nervous system (NS) and other systems of human body; for example, voluntary movements require the interaction between nervous and musculoskeletal systems. The interactions between these system are challenging and they still are not de nitely clear. The eld of research is left open and wide- ranging but advances in technology and the discovery of new biomedical techniques are making research advance signi cantly in recent years. Many researchers have used di erent approaches and methods, such as genetic analysis, neuroimaging techniques, animal models and clinical treatments, in seeking to understand the functioning of the NS, from di erent perspectives and levels of analysis. These studies have led to a large body of knowledge and the development of several theories related to the role of the entral nervous system (CNS) in controlling most functions of the body and mind, such as simple and complex movements. These theories consider the interaction among multiple brain regions, whose cooperation leads to the execution, for example, of skilled movements. For the purpose of evaluating a theory, can the interactions between brain regions be investigated, tested? There exist diverse techniques to explore the activity of the CNS at di erent spatial scale. Indeed, it is necessary to identify and understand the relationship between the activity of micro and macro cortical circuits in the transmission of neural information in order to estimate micro areas activity from less invasive measurements of macro areas activity. These relationships allow to understand (dys)functions of the CNS and to investigate the phenomena behind the dysfunction; they allow to discover and analyse smaller areas of the neural system which cause the dysfunction of the entire neural area with their own dysfunction. Among other methodologies, invasive methods, by means of microelectrodes or arrays of microelec- trodes, remain the most accurate at measuring local eld potentials (LFP) and spiking activity (SPK), 1 but they are prone to a variety of complications including infection, hemorrhage and neurological de cits and are not used on humans; though, intracranial recording techniques give insight about the behaviour of small neural populations, therefore are useful to study neuronal activity at microscopic scale, but do not allow to analyze entire areas assigned to a function. Less invasive techniques are EEG, MEG, PET, fMRI and so on, which allow to study the activation of macrocircuits, but not their functioning in detail, and measure brain activity in a very indirect manner; fMRI, for example, is based on the BOLD (Blood Oxygenation Level Dependent) e ect as the method to determine which parts of the brain are most active, but this signal derives from variations in the magnetic susceptibility of blood due to variations in the concentration of deoxyhemoglobin and it is particularly sensitive to noise. Computational modeling is a di erent methodology and provides for a valuable accessory to the experimental brain research, especially in evaluating the interactions within and among multiple neural systems. The development and analysis of the behaviour of neurocomputational models can be a valid support to provide guidelines for further investigations, helping to clarify mechanisms which are source of much debate and confusion in the specialized literature. They are developed on the basis of the ndings made through the other methodologies and allow to analyze the behaviour of the NS at di erent levels of detail. At microscopic scale, neurons and cells are investigated, in terms of morphology and behaviour; mesoscopic scale deals with population of neurons and macroscopic scale with cortical areas and neural subsystems. In other words, neurocomputational models can be developed at di erent level of detail to analyze and understand the SN at di erent spatial scales (micro-, meso-, macro-). The di erent levels of abstraction are required for two reasons. The organization of the CNS extends over a wide range of spatial scales, from the speci city of single synapses to hierarchically organized networks of entire cortical areas. The properties of neuronal networks in the transmission of neural information only emerge from the interplay of di erent scales. On the other hand, di erent spatial scales mean di erent levels of detail and computational complexity. The more detailed the description of the model (in terms of coupled di erential equations) the more complex it is in computational terms. Instead, simpli ed models are useful for studying large interconnected networks. No technique is better than the others. It is rather essential that there is a dialogue between those who develop models and those who use other techniques for measuring nervous activity in order to understand how the CNS works. Neuroanatomy The progression through di erent learning stages has been studied with imaging and it emerges from these studies an important role to cortico-striatothalamo-cortical loops, in particular to the visual and motor loops [1]. According to the current knowledge, a motor plan is executed through interactions between parietal and premotor areas. Depending on the sensory and proprioceptive information received from the posterior parietal cortex, the premotor cortex speci es the characteristics of a motor plan and sends this information to the motor circuits in the spinal cord, which control the movements of the limbs [2, 3]. A primary role in modulating the information that the premotor cortex sends to the spinal cord is played by the BG (and the cerebellum). The aim of this section is to investigate why this group of nuclei is a key structure in motor learning. Basal ganglia Physiologically, the BG are part of a basic feedback circuit, receiving information from several sources, including the cerebral cortex, eventually feeding this information back to the cortex, via the thalamus. In doing so, the BG act to modulate and re ne cortical activity, such as that of controlling descending motor pathways. In other words, the BG are regulators of cortical function via their in uence on thalamocortical projections. The core components of the BG sit in the telencephalon and include the globus pallidus (GP) and the striatum (or caudate-putamen), a term used to refer to the combination of the caudate and putamen. The related nuclei of the BG reside in the diencephalon (subthalamic nucleus (STN)), mesencephalon (substantia nigra (SN)), and pons (pedunculopontine nucleus) [4, 5]. The BG and related nuclei can be broadly categorized as input nuclei, output nuclei and intrinsic nuclei. Input nuclei are those structures receiving incoming information from di erent sources, mainly cortical, thalamic and nigral. The striatum is considered input nuclei. The output nuclei are those structures that send BG information to the thalamus and consist of the internal segment of the globus pallidus (GPi) and substantia nigra pars 2 reticulata (SNr). Finally, intrinsic nuclei, that are the external segment of globus pallidus (GPe), STN and substantia nigra pars compacta (SNc) are located between the input and output nuclei in the relay of information. In summary, cortical and thalamic e erent information enters the striatum to be processed further within the BG system. The output nuclei (GPi and SNr) project mainly to the thalamus, which, in turn, project back to the cerebral cortex. The appropriate functioning of the BG system requires DA to be released at the input nuclei. Input nuclei: the striatum The striatum, the input nucleus to the BG, is heterogeneous in both its anatomy and function. The major a erents to BG (from the cerebral cortex, thalamus and brain stem nuclei) end in the striatum. They consist of the excitory cortico-striatal pathway, originated from M1 and PMC and a projection from thalamus to striatum (thalamo-striatal pathway). Its neurons project to the GP and SN [5, 3]. The striatum contains two di erent types of neurons: projection neurons and interneurons. Projection or striatofugal neurons are also called Medium-Sized Spiny Neurons (MSNs) because these multipolar neurons have small to medium cellular somata (20 µm in diameter). All striatal MSNs are inhibitory neurons and use -aminobutyric acid (GABA) as their principal neurotransmitter [3]. These cells are both major targets of cortical input [5] and the sole source of the striatal output. The MSNs have a very low spontaneous ring rate and they re in episodes that only last for about 0.1-3 s. In order to re, they must be stimulated by the cortex [6]. The striatal MSNs have been divided further, according to di erent criteria: their projection targets, DA receptor or neuroactive peptide content. In short, MSNs innervating the GPe nucleus express the dopamine receptor subtype 2 (D2R), whereas those projecting directly to GPi and SNr contain dopamine receptor subtype 1 (D1R). In addition, the striatum also contains several di erent classes of local-circuit neurons (interneurons). All the classes of inhibitory interneurons have extensive axon collaterals that reduce the activity of the striatal output neurons. Although few in number, they are responsible for most of the tonic activity in the striatum. It is demonstrated [7] that this variety of GABAergic interneurons controls striatal MSNs activity state, playing a crucial role for synaptic plasticity and motor skill release. Output nuclei: GPi and SNr The GPi and the SNr are the output nuclei of the BG and share similar immunohistochemical structures. Both are mainly composed of tonically active GABAergic inhibitory projection neurons that exert a constant inhibition on their output nuclei, namely thalamic and brainstem nuclei [4]. The primary inhibitory input to GPi and SNr is from GABAergic D1R-containing striatal MSNs, that give rise to the direct pathway, which decreases the ring of the BG output pathways, in other words, keeping the thalamic neurons in a permanent state of inhibition. Instead, the STN provides an excitatory glutamatergic projection to the GPi and SNr, as part of the indirect pathway as well as the hyperdirect pathway and increases the inhibitory output of the BG. Both types of a erent systems (GABAergic and glutamatergic) converge onto BG output neurons, which, in turn, innervate thalamic targets and brainstem. Intrinsic nuclei: GPe, STN and SNc The GPe shares with the GPi several cyto- and chemoarchi- tectural features, because both nuclei are made of sparsely distributed GABAergic neurons with large somas [4]. Two principal a erent systems innervate GPe neurons. The GPe receives a GABAergic pro- jection from D2R-containing striatal MSNs, an inhibitory projection that represents the rst synaptic relay station of the indirect pathway. GPe neurons are also reciprocally connected with STN neurons, from which they receive strong glutamatergic excitatory projections. The STN is closely connected anatomically with both segments of the GP and the SN. The subtha- lamic neurons are excitatory and use glutamate as their neurotransmitter. Extensive research has been performed in the last few decades focused on the Luys' body because it is the best surgical target for electrode placement during deep brain stimulation (STN-DBS) [4, 3], though impairs cognitive functions [8]. Deep in the ventral midbrain, several groups of tyrosine-positive neurons supplying the BG nuclei with DA can be found in the SNc. In humans, dopaminergic neurons located in the SNc are coloured black by the pigment neuromelanin. This pigmentation is visible as a distinctive black stripe in brain sections and represents the origin of the name given to this area. Di erent dopaminergic neurons target di erent brain structures: those in the ventral tegmental area target the nucleus accumbens and are involved in reward processing, whereas those in the SNc target the striatum and are involved in control of movement. Indeed, dopaminergic neurons in the SNc degenerate progressively in Parkinson's disease, 3 leading to severe DA de ciency and the appearance of the cardinal clinical symptoms of the disease, mainly motor. Cellular Mechanisms of DA in the BG: Synaptic Plasticity What is the role of DA in the BG in modulating frontal processes and how is it involved in motor skill learning? Neural signals travelling from the cerebral cortex to the striatum are modulated by dopaminergic bres originating from the SNc. The regulation of DA release by dopaminergic neurons into subcortical regions is currently thought to occur via two independent mechanisms: phasic and tonic transmission. In short, phasic release is caused by burst spike ring, usually in response to a sensory input, while tonic release is caused by slow spontaneous baseline spike activity. Dopaminergic bers are broadly said to be critically involved in learning; in fact, phasic bursts and dips in SNc ring (and therefore DA release) ensue from correct and incorrect responses, respectively; that is they have been inferred to occur during positive and negative feedback of trial-and-error tasks. These phasic changes drive learning by preferentially activating the direct pathway after a correct response and the indirect pathway after an incorrect response [9, 10]. This is feasible because two main populations of cells in the striatum respond di erentially to phasic changes in DA and it causes the two groups of striatal cells to independently learn positive and negative reinforcement values of responses, and ultimately acts to facilitate or suppress the execution of commands in the frontal cortex. In other words, dopaminergic bres provide rewarding signals related to environmental and internal cues in consequence of a particular response selected through the BG. Direct and indirect pathways are a ected di erently by the dopaminergic projections, because changes of extracellular levels of DA, emitted by neurons of SNc, modify synaptic plasticity via D1R and D2R [9, 11]. The broadly accepted principle is the Hebbian one: cells that re together wire together. In other words, the synapses between two neurons that are activated simultaneously are long-term strengthened. This phenomenon is called Long-Term Potentiation (LTP). Since DA enhances activity in the direct pathway, DA bursts may induce LTP in the direct pathway cells. Thus, the ecacy of recently active synapses may be reinforced by a burst of DA acting as a teaching signal , leading to the learning of rewarding behaviours. Conversely, since DA has an inhibitory e ect on the indirect pathway, it may induce Long-Term Depression in the indirect pathway cells, which means that the synaptic connection is weakened since depression decreases the ecacy of a synapse. In summary, phasic bursts and dips of DA occur di erentially during positive and negative feedback, result in modi cation of synaptic plasticity, and, therefore, may be critical for habit learning. Thus, synaptic plasticity in BG pathways is modulated by DA: while DA facilitates LTP in cortico-striatal synapses of the direct pathway via D1R, it facilitates LTD in cortico-striatal synapses of the indirect pathways via D2R [12]. Overview of motor learning Although the term `motor learning' is widely used in the literature, the exact meaning is unclear. It is of general agreement that the learning of a motor skill should result in performance improvement beyond baseline levels. In other terms, a shift of the speed-accuracy trade-o of performance of that skill [13, 14]. This form of learning is a relatively slow process that requires a series of steps, including the selection of a particular action by comparing the expected value of possible actions, executing the chosen action and evaluating the result of the decision. According to the daily experience, a coordinated sequence of elementary movements is acquired and executed faster and more accurately the more it is practiced. After repeated practice, indeed, sensory cues are learnt to be associated with rewarding or discouraging events in order to optimize and automatize motor skills. Namely, motor learning occurs through the acquisition of a sequence of elementary actions (de ned as `chunks'), motor primitives, combined in the correct order into a single complex executable program to achieve a particular goal [15, 13, 16]. Such behaviours include learning to ride a bicycle or to play the guitar or to write a cursive word [17, 18, 19, 14]. However, the improvements in performance of a motor skill could be made in multiple ways. Krakauer et al. found that improved performance largely consisted of reduction in trial-to-trial variability and increase in movement smoothness [20]. Since in this review we sought to investigate motor skill learning from a computational point of view, we are not discussing all the theories and facets of last decades of researches on motor control, but giving general principles on which neurocomputational modeling is based. In 1967, the psychologists Paul Fitts and Michael Posner proposed three main stages of learning motor skills: a cognitive phase, an associative phase, and an autonomous phase [21]. They paved the way for the 4 classic view on the development of motor skills [22, 23, 24]. In the rst phase, the learner is new to a task, performance is usually tentative, slow and inaccurate, but improvements are dramatic; in the second phase performance becomes more accurate and automatic with small changes in the motor patterns, whilst in the third, after a long time of practice, the skill becomes largely automatic, performance is faster, e ortless and precise. Later experimental studies on motor learning, based on computational modeling combined with brain imaging techniques (fMRI, PET, EEG), not only corroborated the psychological theory, but also showed that learning takes place in di erent brain regions at di erent timescales [25, 26, 27, 16, 28, 29, 30], but their activation and cooperation depend both on the kind of movements that are being made (the model system used) and on the e ector that is being used [31]. Kawato put motor skill (control and) learning on a formal footing. Each motor primitive is an element of computation in the sensorimotor map that transforms desired limb trajectories into motor commands. The internal model principle, originally proposed by Francis & Wonham [32] in the eld of robust control theory, was then introduced in the eld of human motor control theory by Kawato et al. [33]. In 1987, they proposed that the CNS to precisely control rapid movements would learn and use di erent kinds of generalized representations of the relations between motor commands and the sensory signals (position, velocity, acceleration, forces or torques). These learned models are stored in the brain and are called internal models. Speci cally, the integration between sensory information and e erent copies of motor commands are linked by inverse and forward models [34, 14, 33], which are the two kinds of internal models. In other words, Kawato posits that each elementary motor action is executed following a sensorimotor transformation process in which the location of the target, encoded in trajectory coordinates, is converted into a motor command [31]. Information provided by the visuo-proprioceptive feedback guides learning. According to Kawato et al. [33], in the rst stage of the learning process of a skilled motor task, movements are executed slowly, therefore the CNS adjusts and correct the sequence of motor commands on the basis of visual and proprioceptive feedback and the motor goals speci ed by the CNS. This process involves a large amount of computation, in other words it is extremely demanding for the brain. They [33] suggest that the CNS would use the rst stage to learn a form of internal model, the forward model. In the second stage, the forward model is continuously adapted, on the basis of the biological feedback. When the forward model is acquired, since it simulates the true motor plant precisely, the movement actually produced is very close to the desired trajectory and there is no more need for the visuo-proprioceptive feedback and the sensorimotor transformation. In other words, when a skill is acquired, the sequence of movements comes to be executed as a single behavior and is performed automatically. In the third stage, the CNS learns an internal model, called the inverse model, which allows inference of the sequence of motor commands from desired trajectories. In the last stage, the inverse model has been learned and it is used to infer directly the sequence of motor commands from the desired trajectories. Following these results, it can be suggested the hypothesis that learning novel motor skills requires two phases, in which two di erent processes occur. Speci cally, early in learning the execution of a motor task is more based upon the trajectory plan and this process is performed by the BG, whereas late in learning upon the sequence of motor commands, maintained by the cortex-cerebellar system. Neural models of BG and DA The BG nuclei are the principal subcortical components of a family of circuits linking the thalamus and cerebral cortex. Many studies have shown that the BG facilitate motor learning and that DA plays an important role in this process. Recent studies place emphasis on the interactions between cortico-striatal and cortico-subthalamic projections on one hand, and internal feedback circuits modulating BG output through the GPi, on the other hand, the so-called cortico-striato-pallido-thalamo-cortical or extrapyra- midal circuit. Also, there is solid evidence that the initial learning of many skills depends critically on the BG, especially on the striatum, because striatal plasticity alters the transfer of information throughout BG circuit. Eventually, the role of striatal interneurons is beginning to be investigated. The Rate Model Movement control The rst coherent model of the BG was developed in the mid-1980s [35, 36, 37, 38, 39, 40], as indicated also by both Obeso and Lanciego [41] and Nelson and Kreitzer [42] in their reviews. This model was called "Rate Model", incorporating a growing body of literature describing the connectivity, neurochemistry and physiology of the BG. It was derived from early clinico-pathological observations of BG lesions associated with movement disorders and, subsequently, by the profound impact 5 that striatal DA depletion caused both in animals and in patients with Parkinson's disease (PD). This is the reason why, the rst idea of BG function has been the control of movement. The Rate Model was based on anatomical and physiological studies in monkeys and the main contribution is the description of the existence of segregated cortico-BG-thalamo-cortical circuits mediating di erent functions, as determined by the cortical area of origin. These circuits comprised the (skeleto)motor, oculomotor, associative and limbic circuits, whose general anatomical organization was similar, but whose cortical and sub-cortical component regions were distinct [43]. Direct and indirect pathways Within the loops, the model posited that di erent populations of striatal MSNs project to the output of the BG (GPi and SNr) via \direct" and \indirect" pathways. The \direct" pathway is a monosynaptic projection , as Obeso and Lanciego [41] denoted, arising from MSNs that express D1Rs and exerting a phasic and robust inhibitory e ect on GPi/SNr neurons. The \indirect" pathway stems from MSNs expressing D2Rs that project to the GPi/SNr through a polysynaptic (\indirect") pathway, polysynaptic because involves relays in the GPe and the STN. This rst concept is based on extensive anatomical and neurochemical data available at the time. In 1986, Alexander [40] claimed that the number of independent loops is hard to estimate. In addition, the model only considered GABAergic input from the GPe and STN e erents targeting BG output nuclei; in other words, the STN is seen as the second relay station, after GPe, of the indirect pathway. In short, the Rate Model postulated that the BG process cortical input through parallel pathways from striatum through to the output nuclei and feed it back to the cortex via a thalamic relay. Increases or decreases in ring rate of di erent BG nuclei are postulated to regulate BG output and behaviour. Multiple actions The striatum must orchestrate action selection in the face of numerous candidate actions to select among. Indeed, GPi/SNr neurons re in a mostly tonic manner, keeping targeted structures in the thalamus and brainstem under tonic inhibitory control. Brief pauses in neuronal activity in the output of the BG facilitate movement, otherwise movement is inhibited. Dopamine About the role of DA within the BG, the authors agree it appears to be complex. They had evidence that the nigrostriatal DA projections exert contrasting e ects on the direct and indirect pathways. In the speci c, dopaminergic inputs appear to have a net excitatory e ect on striatal neurons that send GABA to GPi (via the direct pathway), and a net inhibitory e ect on those that send GABA projections to GPe (via the indirect pathway). We must say that these considerations followed histo- chemical and immunocytochemical studies of neurotransmitters and neuromodulators. It goes without saying that they could make assumptions but further research and evidence was required. Also, it must be considered that DA has been discussed in relation to the clinical implications of the treatments which alter levels of DA neurotransmission, since they a ect motor behaviour. Indeed, evidence on dopaminer- gic depletion showed that DA loss reduces the activity of MSNs belonging to the \direct" projection and enhances the activation of \indirect" pathway neurons, leading to increased activity of the STN which in turns over-activates inhibitory output neurons in the GPi/SNr. Then, the increased activity in the GPi/SNr ultimately reduces the likelihood of phasic inhibitory activity in these output neurons, thereby impeding movement initiation and execution. It is clear that the Rate Model is greatly oversimpli ed, as the authors themselves highlighted. It must be noted that it intended to provide a new framework in which to interpret the pathophysiology of movement disorder, more than the BG role in general; actually, BG role was then emerging. Therefore, the interpretation of the biochemical, anatomical, and electrophysiological ndings by the authors is only related to movement disorders (especially the neuropharmacology of movement disorders). In other words, their reading key of direct and indirect pathways is whether shifts in the balance between activity in these pathways, resulting in alterations in GPi/SNr output, accounted for hypo- and hyper-kinetic features of BG disorders. The main cap the model sets is that the BG are relegated to a functional role in producing motor output. In other words, the BG have been thought to function only in voluntary movement as a \go through" station that receives, processes and conveys information via closed and parallel loops. However, the Rate Model has helped propel a large body of hypothesis-driven research, which ultimately promoted greater knowledge of BG function. In this regard, a number of experimental observations con icted with the Rate Model, which led to two well-recognized paradoxes [41]. What is remarkable is how this evidence has driven the evolution of the model. 6 Evolution of the classic model A new perspective on the computational role of BG arose in the mid-1990s [44, 45], shortly after the model was rst introduced: now the BG have been hypothesised not to generate movements; rather, when voluntary movement is generated by cerebral cortical and cerebellar mechanisms, the BG would act broadly to inhibit competing motor mechanisms that would otherwise interfere with the desired movement. Moreover, the nding of the existence of a large projection to the BG from cortical areas that subserve primarily cognitive, rather than sensorimotor functions, indicated that the role of the BG might not be con ned to the selection of movements and could play a comparable role in cognition to that of action selection in motor control. Other anatomical and physiological studies of this age [46, 47] showed that the BG contribute to cognitive and even emotional functions. Dopamine In their article, Redgrave et al. [44] provide a section of its own for DA and deal with the `dopaminergic regulation of switching' capability of BG. Experimental data on altered levels of DA neuro- transmission have shown that tonic levels of DA neurotransmission play an important role in regulating behavioural selections. Indeed, mild to moderate increases in dopaminergic activity tend to facilitate switching while comparable reductions in transmission may retard switching. Then, the authors note that behavioural switching should occur when changing circumstances result in higher salience for a competing command system. In fact, they report accumulating data suggesting that the most e ective stimuli for inducing behavioural switches include novel events, primary reinforcers and previously neutral stimuli that have become associated with reward or punishment. In addition, they also report 100-150 ms as approximate response time of BG to these stimuli, gained on the basis of the duration of the response observed in SN and VTA DA neurones following such stimuli. Reward processing Dopaminergic neurons uctuations are associated to the function of reward. There is evidence [48] that supports the proposal that DA neurons in the VTA and SN are reported to be appar- ently signalling ongoing prediction errors for reward. Furthermore, some authors [49, 50, 51] reviewing electrophysiological evidence on DA and synaptic plasticity suggested that dopaminergic input to the striatum could place the system into a state where corticostriatal synapses are eligible for modi cation. In other words, in addition to facilitating switching, the early DA response could also place the BG into a `state of readiness' to modify selections on the basis of subsequent experience. The role of STN Furthermore, much experimental evidence superseded the simplistic view of STN role in BG. Besides the classical GABAergic projection from GPe neurons, the STN also receives glutamatergic projections from the cerebral cortex (known as the `hyperdirect' pathway) and from other nuclei. The cortico{subthalamo{pallidal hyperdirect pathway conveys powerful excitatory e ects from the motor- related cortical areas directly onto BG output, bypassing the striatum, with shorter conduction time than e ects conveyed through the striatum [52, 53]. Direct and indirect pathways Many authors [9, 54, 55] agree on the hypothesis that one of the roles of the BG within the frontal lobe - striatum system is to contribute in selecting, among a set of behaviours processed by the cortex, the appropriate behaviour so that it becomes active and can be expressed. In other words, the BG does not encode the details of motor responses, but simply modulates their execution by signaling GoorNo-Go . Shortly, the BG encode stimulus-response (SR) associations. Thus, they are thought to act as a brake on competing motor actions: only the most appropriate motor command is able to release the brake and get executed [9]. This process is referred as action selection and is performed through the direct and indirect pathways that, operating in concert, release the (only one) desired response and inhibit the (remaining) unwanted ones. In line with these hypotheses, patients with BG disorders (i.e., PD and Huntington's disease) are impaired in response selection [56, 9]. A larger functional complexity for BG organization has been recognized then, displacing two main concepts of the classic model: that the striatofugal system is composed of two separate (direct and indirect) pathways originating from distinct cell populations in the striatum and that the neuronal ring rate in the output of the BG governs and predicts the motor state [57]. In fact, evidence showed that striato-pallidal pathways are not so distinctly separated. Instead, the same striatofugal neurons frequently establish connections with GPi, GPe and SNr neurons [58, 59]. Recording local eld potentials, it was assessed oscillatory activity from STN and GPi [60] in patients with PD which could be generated by the BG but also by the cortex and thalamus. Then, more than neuronal ring rate in the output of the BG, also neuronal synchronization and discharge pattern must be considerated in BG disorders. 7 The role of GPe The view of the motor circuit changes. Cortical activity arrives the BG through the projection to striatum and STN. The STN is thus now considered as another input station of the BG that receives a erents from the cerebral cortex, thalamus and brainstem. Many studies [53, 61, 62] report that the STN also receives glutamatergic projections from the cerebral cortex (known as the hyperdirect pathway) as well as other nuclei [63]. In other words, the close interrelationship between the GPe and STN has reversed the classic theory of considering the GPe as simply a relay station between the striatum and the STN. The cortical projection exerts an opposite (inhibitory/excitatory) disynaptic e ect onto the GPe and GPi. Activity of the GPe modulates back the excitability of the striatum and STN by means of the reciprocal inhibitory connections [64]. Thus, the GPe seems as a critically important nucleus to regulate BG neuronal output toward the thalamocortical projection within the motor loop [65]. Accordingly, the exclusive feed-forward nature of information processing within the BG is not the only major feature of BG functional organization. Also, reciprocal connections between the striatum and DA neurons in the SNc [66] have been recognized. The role of STN Frank [60] reports STN is a key structure in BG nuclei that participates in both motor and cognitive processes. His model shows that by modulating when a response is executed, the STN reduces premature responding and therefore has substantial e ects on which response is ultimately selected, particularly as there are multiple competing responses. Speci cally, since the STN gets in- creasingly excited with increasing cortical activity, if several competing responses are activated (in the cortex), the STN sends a stronger \Global NoGo" signal which allows the BG system to fully consider all possible options before sending a Go signal to facilitate the most adaptive one. In other words, the STN excitation allows extra time to consider the best option when confronted with con icting choices. Frank's model provides support for this hyphothesis, since models with STN lesions showed impairment at making high con ict decisions. This result is consistent with the notion that the STN is critical for preventing premature responding; furthermore, the strength of this Global NoGo signal is modulated by the degree of response con ict present in premotor cortex. Thus, if multiple competing responses are active, the model may take more time to select a given one. The role of DAergic system Data regarding how DA acts, particularly in the striatum, has changed considerably since the model was created. SNc DAergic neurons discharge tonically at a low frequency but exhibit robust and highly synchronous burst ring under special circumstances such as reward or anticipation of movement. The available data suggest that DA regulates the excitability of MSNs from the cortex and thalamus. In particular, presynaptical DA depresses striatal glutamatergic release by activation of D2R [67]. On the other hand, postsynaptic activation of D1R increases MSN excitability [68]. These phasic changes in DA have been interpreted as an error signal that informs the corticostriatal system of the discrepancy between the prediction of a reward and its actual occurrence. This hypothesis sustains a major role for the DAergic system in reinforcement learning. Indeed, they dynamically modulate the BG threshold for facilitating/suppressing a cortical command in response to particular stimuli [9]. DA also plays a fundamental role in the mediation of plastic synaptic changes in the striatum. Thus, striatal long-term potentiation (LTP) and depression (LTD) are critically dependent on D1R and D2R activation respectively, and therefore mediated by MSN in the direct and indirect circuits [57]. In addition, the DAergic system is hypothesized to be capable of selectively inhibiting and exciting di erent MSNs pools. Such di erential e ect of DA on D1R and D2R and its e ects on interneurons, seem designed to facilitate a particular signal while inhibiting all others. Besides nigro-striatal projections, accumulating evidence showed that DAergic neurons also innervate and modulate neuronal activity of key BG nuclei other than the striatum, such as the GPe, STN, GPi and also various thalamic nuclei [69]. Also, more projections from outside nuclei with the BG starts to be in- vestigated since evidence regarding complex projection from the thalamus and from the pedunculopontine nucleus (PPN) are found [57, 41]. Interneurons The classic model did not take into account the role of striatal interneurons. The striatum contains the cholinergic, tonically active neurons (TANs) and the GABAergic fast-spiking in- terneurons (FSIs). TANs modulate MSN excitability, by inhibiting cortical glutamatergic input, and mediate DAergic mechanisms leading to LTD. FSIs mediate feed-forward inhibition which, together with axon collaterals from MSNs, provide intrastriatal inhibition. These inhibitory e ects were not considered at all in the original model, whereas they serve as a substrate for the selection process [41]. 8 The present Over the last ten years, great strides have been made in exploring and understanding the activity of the nervous system at di erent spatial scale. Progress has been stunning in the neuroscience community, as reported on the 50th anniversary of the Society for Neuroscience. There exist several di erent techniques to analyse the activity of brain circuits which turned neu- roscience research upside down. The classic approach based on anatomical and histochemical data or invasive (microelectrodes) methods was inherently limited. In particular, the imperfections and inaccu- racies of the Rate Model are not minor and many issues remain open to study. Now, there exist less invasive techniques as EEG, MEG, PET, fMRI, arti cial intelligence approaches as neurocomputational modeling, virtual reality environments; also, optogenetics, chemogenetics, new methods for visualizing biomarkers or imaging and further innovative techniques are currently being developed. Importantly, human studies are now possible thanks to the development of these powerful neuroimaging and neurophysiological techniques, which provide far better resolution of BG activation and distortions under normal and pathological conditions. Arti cial intelligence remarkably sparked a boom in the eld. Since research has included multimodal neural recordings and arti cial neural networks, results have steadily improved. Technological advances will have profound impacts on how we understand normal and abnormal function and treat neurological disorders. In neurocomputational modeling, recent research divides in investigating BG neuronal activity at microscopic scale, to understand the exact morphology of neurons, how they communicate and learn, and at mesoscopic scale, focusing on the behavioral aspect, dynamics, size and structure of neural areas. In this review, we focus on the second class. Striatal learning Most models of BG developed in last decade do not model the indirect pathway. Yet they are important for motor learning modeling, since they prove that direct pathway is the only functional architecture needed to learn to respond appropriately to environmental cues through alterations of striatal plasticity. Indeed, the development and execution of appropriate responses to environmental stimuli require continual updating, learning and adjustments of the response and this process of response learning is the combination of di erent areas (motor, cognitive, motivational, etc) forming a smoothly executed, goal-directed behavior [70]. There are now major points that argue against a strict parallel processing system. These pathways are not as segregated as once thought [71, 72]. Schroll et al. [73] did not model neither the indirect and the hyperdirect pathway, with the aim of showing that they are not implicated in the function targeted in their paper, that is the ability to learn working memory (WM) and response selection. The authors show that the model is able to learn stim- ulus{response mapping from past experience thanks to the cortico-BG-thalamic loops, thus constituting the enough architecture for both WM control and response selection. Indeed, the striatum and its direct pathway allows for WM maintenance in prefrontal loops and for stimulus{response associations in motor loops. Other authors [74, 75, 76] support the hypothesis that stimulation of direct pathway is sucient to induce learning by means of approaches other than computational modeling. Ashby and Crossley [77] developed a model to analyse the role of cholinergic interneurons in the striatum when reward is not available. Then, they oversimpli ed the neuroanatomy of the BG, omitting the indirect and hyperdirect pathways, to focus on the e ects of the TANs on MSNs. They proved that the TANs exert a tonic inhibitory in uence to striatal MSNs over cortical input, but they learn to pause in rewarding environments, releasing the striatal output neurons from inhibition, thus facilitating learning and expression of striatal-dependent behaviors. Although, when rewards are no longer available, the TANs cease to pause and protect striatal learning from decay. Interneurons All neurons in the striatum can be considered pacemakers for their rhythmic changes in membrane potential. Cholinergic interneurons are also autonomous pacemakers, but they act on the other striatal neurons. The role of the cholinergic interneurons (a ected by DA themselves), despite being still debated in the literature, is becoming increasingly critical, supported by new research ndings [78, 79, 80]. Baston and Ursino [81] presented a a novel parsimonious mathematical model of BG in action selec- tion developed in Matlab. The model is biologically inspired and reproduces the main BG structures and pathways, modeling explicitly both the dopaminergic and the cholinergic system. The innovation of the model is represented by the introduction of cholinergic interneurons in BG mechanism and synapse learn- ing and the explicit formulation of all equations and parameters. Their model may represent a decisive tool to discover novel BG functionalities in behavioral experiments thanks to the explicit representation of the cholinergic interneurons. 9 In the computational model of BG circuits for motor learning by Franklin and Frank [82], cholinergic interneurons (TANs) have been considered, by building on their existing computational model of the BG. This model the rate of learning does not remain constant, but learning occurs more quickly when the outcome of a behavior is uncertain. Indeed, TANs enable the BG to take account of uncertainty. They proposed that the pause in TAN activity is formed by local striatal inhibition to code the uncertainty and regulate learning rates through cholinergic projections to MSNs. With the experiments, they found that by varying the length of the TAN pause, the BG can adjust learning rates based on the degree of uncertainty over behavioral outcomes. Genetic algorithms It is remarkable the work of Bahuguna et al. [83], as they decided to use a genetic algorithm to search for the unknown connectivity parameters in a ring rate model in order to get through the main hurdle of developing a computational model of the multiple interconnected nuclei of BG, that is the parameters of the connections. They forged a binary cost function derived from empirical ring rate and phase data for the physiological and Parkinsonian conditions. Their approach generates ensembles of over a thousand valid solutions, or homologies, for each condition. It is to be noted that providing a transient cortical stimulus, the networks classi ed as physiological were not susceptible to oscillations, whereas parkinsonian networks showed the opposite tendency. Their groundbreaking approach of generating and analyzing an ensemble of multiple solutions provides greater con dence in predictions of the model than those derived from a unique solution. Integrative frameworks Rosa Senatore [19] proposed a neural scheme for motor learning, validated through a neurocomputational model developed in Emergent [84]. In the system, the BG, interacting with the parietal cortex, select the next target point in the sequence of motor commands. This model ts the hypothesis according to which motor learning follows two distinct phases. Therefore, the BG are involved during the early phase of learning, in which the model learns the spatial sequence in visual coordinates (i.e. the sequence of points to reach in order to realize the motor task) through the interactions between the BG and the parietal cortex. The novelty of this model is the simulation of motor learning including BG, Cerebellum and cortex; the BG model is the one developed by Frank [9, 60]. Wu and Faisal [85] developed an integrative spiking neuron framework to study motor learning from synaptic learning rules of neurons to an arm's movements. Certainly, it is simplistic in the description of the sensorimotor control loop, but captures many elements of the perception-action loop including muscle dynamics and arm mechanics typically neglected in computational neuroscience. The ability they o er to simulate and explain motor learning across the levels of neural activity as well as psychophysics experiments is both fascinating and provides a rst step towards better understanding of cortical activity and neural interfacing. Discussion Developing a neurocomputational model of the basal ganglia is highly challenging. The analysis conducted on all the contributions provided by the models allows to formulate the following hypothesis. The primary function of BG nuclei remains that of learning and regulating the activities of the motor and premotor cortical areas for executing skilled smooth movements in response to cortical inputs. Despite direct and indirect pathways are proved to be not segregated anatomically, separating them functionally in a model still ensures great expressive power for SR association learning. Rather, striatal learning mechanism should be massively tested, from di erent perspectives and spatial scales, as DAergic system was in the previous decade. Indeed, the role of interneurons in understanding BG functions and dysfunctions may be crucial, as the models reviewed suggest; developing a model without considering interneurons today would be obsolete and highly restrictive with respect to the functionality of BG. Also, the adoption of genetic algorithms for parameters estimation is attractive. In general, the adoption of arti cial intelligence allows to overcome one of the main criticisms of model-based approaches, that is the speci city of the solution and the lack of generalisation. Thus, to bring a particular subject in a highly dimensional space is a burning issue. As a general comment, ndings obtained from the experiments conducted through neurocomputational models give an indication to formulate new tests for the rehabilitation of patients su ering from damage to BG, such as PD or cerebral ischemia patients. Speci cally, it is de nitely more fruitful to study PD and other neurodegenerative diseases in terms of its dynamical features rather than its structural changes. Dynamical features provide symptomatic targets for therapy without presupposing speci c underlying structures which may hold only in the mean, and not be applicable to a given patient. 10 Funding This research received no external funding. Con icts of Interest The authors declare no con ict of interest.