A spiking neural model of adaptive arm control - Supplemen-tary material

There are several previous models of the motor control system that overlap with the work presented here. For instance, [30] presents a model based on an attractor network implementation that captures the stereotyped sudden transitions (STs) in the dorsal pre-motor cortex (PMd) predicting forthcoming actions on a single-trial basis. These researchers suggest that there is a cascade of STs throughout the areas of the brain involved in motor control starting with those areas that plan the movement (PMd) and down to areas executing the movement, i.e. the primary motor cortex. While this model captures some of the neurophysiological phenomena of the motor system, it offers little in the way of mechanistic explanation of the function of these areas. It also does not demonstrate complex paths, or adaptation to online changes in dynamics or kinematics of the arm. In [27], the authors present a model able to achieve repeatable activation patterns by training networks of rate neurons to converge to a specific path through the state space. The authors suggest that a similar training mechanism and network could serve as a basis for trajectory generation in the brain, by decoding the neural activity at different parts of the path into a sequence of target positions. While this model is able to perform some of the functionality required by trajectory planning areas, it was implemented in rate neurons as opposed to spiking neurons (the authors noted explicitly that how to implement their network in spiking neurons was unclear), and lacks the generalizability of a trajectory generation system based on dynamic movement primitives. As well, it does not control a nonlinear 3-link arm, or demonstrate online dynamic or kinematic adaptation. Recently, in [31], the authors implemented a spiking neuron system that performs the transformation between task-space and low-level control signals, similar to the primary motor cortex portion of our model. The focus of that work was the implementation on neuro-morphic hardware and the controlling of a physical robotic arm, so no comparisons were made to the motor control system of the brain. We have gone beyond this work by implementing a secondary controller that operates in the null space of the first [26, 45], applying nonlinear adaptive control methods to account for unmodelled dynamics and changes to the kinematics online, and providing a comparison to cortical activity by analyzing spiking data. Similarly, in [11] the authors explore the control of a 2 DOF muskuloskeletal model using spiking neurons, that is linked to a robotic arm. The model focuses on employing reinforcement learning in the network, which consists several hundred neurons divided into three layers. They demonstrate an improvement in the smoothness of the movement after learning, compared to a simpler arm model. Unfortunately, the neural activity and behavioural profile of this model is not compared to empirical studies. In contrast, the REACH model is mapped in greater anatomical detail, incorporates rapid dynamics learning, includes complex path following, and demonstrates a good match to behavioural and physiological data collected from a variety of empirical experiments. Motivated by work presented in [38], the authors of [37] present a behavioral model of human adaptation in the context of reaching under force fields. The authors show that the nonlinear adaptation methods of [36] give rise to the same trajectory profiles post-learning as seen in humans, as well as similar after-effects when the force field is removed. The scope of this model is narrow, however, focusing on a behavioral comparison of adaptation only. No other behavioral measures, such as velocity profiles or total squared jerk, are used, and no mapping to biological substrates or discussion of neural implementation are provided. The model presented in [40] is a partially-neural model that encompasses the motor cortex and cerebellum. The authors employ internal models of the dynamics of the system, learned in an anatomically grounded cerebellar circuit during fast movements to allow the arm to move more precisely during ballistic movements. The authors acknowledge several limitations of their model, however, including the lack of a biologically plausible learning rule, lack of ability to model non-linear