Journal of Robotics, Networking and Artificial Life, Vol. 4, No. 1 (June 2017) 62–66___________________________________________________________________________________________________________A Combination of Machine Learning and Cerebellar-like Neural Networks for the Motor
Control and Motor Learning of the Fable Modular RobotIsmael Baira Ojeda

Center for Playware, Department of Electrical Engineering, Technical University of Denmark, Elektrovej Building 326
Kongens Lyngby, 2800, Denmark
Silvia Tolu, Moisés Pacheco, David Johan Christensen, and Henrik Hautop Lund

Center for Playware, Department of Electrical Engineering, Technical University of Denmark, Elektrovej Building 326
Kongens Lyngby, 2800, DenmarkE-mail: iboj@elektro.dtu.dk, {stolu, mpac, djchr, hhl}@elektro.dtu.dk
www.dtu.dk

Abstract
We scaled up a bio-inspired control architecture for the motor control and motor learning of a real modular robot.
In our approach, the Locally Weighted Projection Regression algorithm (LWPR) and a cerebellar microcircuitcoexist, in the form of a Unit Learning Machine. The LWPR algorithm optimizes the input space and learns theinternal model of a single robot module to command the robot to follow a desired trajectory with its end-effector.
The cerebellar-like microcircuit refines the LWPR output delivering corrective commands. We contrasted distinct
cerebellar-like circuits including analytical models and spiking models implemented on the SpiNNaker platform,showing promising performance and robustness results.
Keywords: Motor control, cerebellum, machine learning, modular robot, internal model, adaptive behavior.

consumption. As a result, studying how we control ourbodies in uncertain or unknown environments, how wecoordinate smooth movements and the mechanisms of
motor control and motor learning of the central nervoussystem (CNS) has become of interest towards thedevelopment of bio-inspired autonomous roboticsystems.

1. Introduction
Understanding the human brain is one of the greatest
and most appealing challenges facing scientificresearch. Therefore, initiatives such as the Human Brain
Project1 (HBP) were conceived to encourage thedelivery of beneficial breakthroughs for society andindustry. The HBP unites the effort of numerous
research centers and universities involving multiple
disciplines and goals in the form of 12 subprojects. In
particular, our group is framed within the subproject 10
focused on neurorobotics.
Robots lack the adaptability and precision of humanbeings towards uncertain or unknown environments. In
contrast, the brain accomplishes tasks in an admirable
way allowing smooth movements with a low power

1.1. The cerebellum
Among the distinct parts of the brain, the cerebellum
stands out due to its key role in modulating accurate,complex and coordinated movements, acting as auniversal learning machine2. Its contributions include
the neural control of bodily functions, such as postural
positioning, balance or coordination of movements overtime3. Thus, understanding and mimicking theCopyright © 2017, the Authors. Published by Atlantis Press.
This is an open access article under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).
62

Journal of Robotics, Networking and Artificial Life, Vol. 4, No. 1 (June 2017) 62–66___________________________________________________________________________________________________________cerebellar
mechanisms
throughbio-inspiredarchitectures are interesting processes in thedevelopment of innovative robotic systems capable ofcarrying out complex and accurate tasks in varying
situations (See Refs. 4-6).
Following this motivation, we took inspiration from thecerebellum for the motor control and learning of a real
modular robot. For this purpose, we used a bio-inspired
control architecture which combines machine learningand a cerebellar microcircuit. The cerebellar models
were simplifications of the real biological microcircuit
including only the Purkinje, granule and deep cerebellar
nuclei cells, and the parallel, mossy and climbing fibers.
Moreover, we considered three cerebellar models,
including spiking and non-spiking approaches, aiming
at enlightening when to use which model andestablishing the grounding for our future research. The
result was a compliant robot module that by means of a
bio-inspired approach was able to learn how to trace out
a circular trajectory with its end-effector.

Fig. 1. The robot Fable. An example of a modular
configuration comprising four actuated modules and a head
module endowed with ultrasound sensors.

2.2. The modular approach
Scientific research studies on the cerebellum such as
Refs. 8-9, describe it as a set of adaptive modules,
called microcomplexes, which represent the minimalfunctional unit and show a uniform almost crystallinemicrocircuitry10. Thus, we decided to use its structure tocontrol a robot module in a generic manner.
Two microcomplexes were used to command the jointsof a 2-DoF module. Each cerebellar output was linked
to one joint as it happens in our body, where one motor
cell commands one motor unit.

2. Material and Methods
In this section, we address the modular robot, ourmodular control approach and the Machine Learningtechnique utilized.
2.1. The modular robot: Fable
The target robotic platform is the modular robot called
Fable7. Fable is based on the combination of selfcontained modules which can work independently or
collaborate in modular configurations. Due to a low lag
radio communication link to the modules the user canprogram the distributed robot modules at different levelsof abstraction as if they were centralized and connected
directly to the computer. To do so, the communication
is done via a radio dongle addressing each module withan ID and a radio channel.

Fig. 2. Modular scheme of the connections between thecomputer, the modular robot Fable and the neuromorphic
SpiNNaker platform, which was used for the implementationof the spiking cerebellar model.

2.3. The bio-inspired modular control architecture
In order to perform the motor control and learning of a
Fable module, we chose the Adaptive Feedback Error
Learning (AFEL) architecture4 shown in Fig. 3.

63

Journal of Robotics, Networking and Artificial Life, Vol. 4, No. 1 (June 2017) 62–66___________________________________________________________________________________________________________The trajectory generation block generates the desired
joint angles and velocities (Qd, Q̇d) by inverse
kinematics.
On the one hand, the AFEL scheme guarantees thestability of the system by means of a control loop in
which a Learning Feedback (LF) controller4 isimplemented. The LF overcomes the lack of a precise
robot morphology dynamic model ensuring stability andadjusting its output torque through a learning rule after
consecutive iterations of the same task. Its gains were
heuristically tunes to Kp = 7.5, Kv = 6.4 and Ki = 1 for
the Fable module.
On the other hand, the AFEL architecture is endowedwith a ULM, comprised by the LWPR algorithm and acerebellar circuit. The ULM performs a feedforward
control of the robot module. The LWPR is in charge ofabstracting the internal model of the robot, while thecerebellar microcircuit refines the output deliveringcorrective torques.

In case 3 we implemented a simplified spiking
cerebellar model using the neuromorphic platform
SpiNNaker13, consisting of Purkinje cells and DCN cells
but without considering recurrent or inhibitory
synapses.
2.4. The LWPR algorithm
The LWPR algorithm14 creates and combines N linear
local models and feeds the sensorimotor inputs (Qd, Q̇d,Q, Q̇) of the robot including desired and real values tothem. Thereafter, the LWPR incrementally divides thissensorimotor input space into a set of receptive fields(RFs) performing an optimal function approximation.
The RFs are represented by a Gaussian weighting kernel(Eq. 1) which computes a weight in each k-th local unit,
for each xi data point according to its distance to the ck
center of the kernel.
−1

𝑝𝑘 = 𝑒 2 �(𝑥𝑖 −𝑐𝑘)𝑇 𝐷𝑘 (𝑥𝑖 −𝑐𝑘 )�

(1)

The weight measures how often an item xi of the data
falls into the validity region of each linear model,
characterized by a positive definite matrix Dk, called
distance matrix.
The LWPR conveys the weights to the cerebellar circuit
and at the same time, it delivers a torque output
computed as the weighted mean average of the linear
local model's contributions.
We chose the LWPR algorithm for three reasons: tooptimize the input space to enhance learning speed andaccuracy; since it can substitute and optimize the role ofa certain group of cells of the cerebellum, called granulecells; and due to its capability of learning incrementallyon-line.
Fig. 3. The AFEL control architecture. The AFEL controlarchitecture embeds a ULM which acts as a feed-forward
controller while it abstracts the internal model of the robot
module.

3. Results
We tested the control architecture using the threecerebellar model cases described in Section 2.3 bycommanding the robot to trace out a circular trajectorywith its end-effector as depicted in Fig. 4. The tests
considered the normalized mean square error (nMSE) of
the position of the joints with respect to the desired
positions. First, the performance test consisted infollowing a circular trajectory with constant amplitude
and spin frequency (see Fig. 4).

The cerebellar microcircuitry considered three cases.
Case 1, based on Ref. 4, included only the Purkinje cells
whose learning rule is based on the heterosynaptic
covariance learning rule11 in the continuous form andadjusted by an error signal τfb. In Case 2 we included theDeep Cerebellar Nuclei (DCN) cells, adding two extra
synaptic plasticities whose learning rules were inspiredby Ref. 12.

64

Journal of Robotics, Networking and Artificial Life, Vol. 4, No. 1 (June 2017) 62–66___________________________________________________________________________________________________________Fig. 4. Examples of circular trajectories.

Fig. 7. Robustness test: Varying spin frequency (0.5-1Hz).

4. Conclusions
We combined the LWPR algorithm and a cerebellar
circuit for the motor control and learning of a real robot
module. Furthermore, we implemented three distinct
cerebellar models: two non-spiking and one spiking
using the neuromorphic SpiNNaker platform.
Compared to Case 1, Case 2 showed better results in theperformance test while keeping similar results in bothrobustness tests. Case 3 did not show improvementswith respect to the non-spiking models, but since its
circuitry was quite simple there is much room forpromising further research. Future research will exploit
the potential of more detailed spiking models whereinhibitory and recurrent synapses will take place andexplore the control of several robot modules using
SpiNNaker.

Fig. 5. Performance test: Circular trajectory with constant
amplitude and spin frequency.

Secondly, we carried out two robustness tests (see Figs.
5 and 6) where we considered trajectories that varied
they amplitude keeping the spin frequency constant, andthereafter, trajectories that kept the amplitude constantbut varied their spin frequency (0.5-1Hz).

Acknowledgements
This scientific work has received funding from theEuropean Union Horizon 2020 Programme under thegrant n. 720270 (Human Brain Project SGA1). A
special thank is expressed to the University ofManchester for the SpiNNaker board.
References
1. Ailamaki, A., A. Alvandpour, K. Amunts, W. Andreoni,J. Ashburner, M. Axer, M. Baaden, and R. Badia. "TheHuman Brain Project: A Report to the EuropeanCommission”. Lausanne: The HBP-PS Consortium(2012).
2. Ito, Masao. “Control of mental activities by internal
models in the cerebellum”. Nature Reviews Neuroscience9, no. 4 (2008): 304-313.

Fig. 6. Robustness test: Using three distinct amplitude values.

65

Journal of Robotics, Networking and Artificial Life, Vol. 4, No. 1 (June 2017) 62–66___________________________________________________________________________________________________________

3. Ito, Masao. “Cerebellar circuitry as a neuronalmachine.” Progress in neurobiology 78, no. 3 (2006):272-303.
4. Tolu, Silvia, Mauricio Vanegas, Niceto R. Luque, Jesús
A. Garrido, and Eduardo Ros. “Bio-inspired adaptivefeedback error learning architecture for motor control.”
Biological cybernetics 106, no. 8-9 (2012): 507-522.
5. Lepora, Nathan F., Paul Verschure, and Tony J. Prescott.
“The state of the art in biomimetics.” Bioinspiration &biomimetics 8, no. 1 (2013): 013001.
6. Bologna, L. L., J. Pinoteau, J. B. Passot, J. A. Garrido,Jörn Vogel, E. Ros Vidal, and A. Arleo. “A closed-loop
neurobotic system for fine touch sensing.” Journal ofneural engineering 10, no. 4 (2013): 046019.
7. Pacheco, Moises, Rune Fogh, Henrik Hautop Lund, andDavid Johan Christensen. “Fable: A modular robot forstudents, makers and researchers.” In Proceedings of theIROS workshop on Modular and Swarm Systems: fromNature to Robotics. 2014.
8. Dean, Paul, John Porrill, Carl-Fredrik Ekerot, and Henrik
Jörntell. “The cerebellar microcircuit as an adaptive
filter: experimental and computational evidence.” NatureReviews Neuroscience 11, no. 1 (2010): 30-43.

9. Verduzco-Flores, Sergio O., and Randall C. O'Reilly.
“How the credit assignment problems in motor control
could be solved after the cerebellum predicts increases inerror.” Frontiers in computational neuroscience 9 (2015).
10. Ruigrok, Tom JH. “Ins and outs of cerebellar modules.”The cerebellum 10, no. 3 (2011): 464-474.
11. Sejnowski, Terrence J. “Storing covariance withnonlinearlyinteractingneurons.”
Journalofmathematical biology 4, no. 4(1997): 303-321.
12. D’Angelo, Egidio, Lisa Mapelli, Claudia Casellato, Jesus
A. Garrido, Niceto Luque, Jessica Monaco, Francesca
Prestori, Alessandra Pedrocchi, and Eduardo Ros.
“Distributed circuit plasticity: new clues for thecerebellar mechanisms of learning.” The Cerebellum 15,no. 2 (2016): 139-151.
13. Furber, Steve B., Francesco Galluppi, Steve Temple, andLuis A. Plana. “The spinnaker project.” Proceedings ofthe IEEE 102, no. 5 (2014): 652-665.
14. Vijayakumar, Sethu, Aaron D’souza, and Stefan Schaal.
“Incremental online learning in high dimensions.” Neuralcomputation 17, no. 12 (2005): 2602-2634.

66