From “Understanding the Brain by Creating theBrain” toward Manipulative Neuroscience
Mitsuo Kawato
ATR Computational Neuroscience Laboratories
Abstract
Ten years have passed since the Japanese “Century of the Brain” waspromoted, and its most notable objective, the unique “Creating the Brain”
approach, has led us to apply a humanoid robot as a neuroscience tool. Here,
we aim to understand the brain to the extent that we can make humanoid
robots solve tasks typically solved by the human brain by using essentiallythe same principles. I postulate that this “Understanding the Brain by
Creating the Brain” approach is the only way to fully understand neuralmechanisms in a rigorous sense. Several humanoid robots and theirdemonstrations are introduced. A theory of cerebellar internal models and asystems-biology model of cerebellar synaptic plasticity are discussed. Bothmodels are experimentally supported, but the latter is more easily verifiablewhile the former is still controversial. I argue that the major reason for thisdifference is that essential information can be experimentally manipulated
in molecular and cellular neuroscience while it cannot be manipulated at thesystem level. I propose a new experimental paradigm, manipulative
neuroscience,toovercome

thisdifficulty

and

allow

us

to

prove

cause-and-effect relationships even at the system level.
(main text 8123 words)

1. “Creating the Brain” in the Japanese Century of theBrain
In April 1996, an ad hoc committee on brain science of the Science Council ofJapan, the nation’s most prestigious association of scientists, issued anadvisory called “Promotion of Brain Science” to the Japanese government. In
August of the same year, a committee for brain science promotion in the
Science and Technology Agency of the Japanese government proposed a hugenational brain research project funded at two trillion yen (~$20 billion) for
twenty years within a blueprint for the “Century of the Brain”. In May 1997,a brain science committee of the prime minister’s Council of Science andTechnology, the nation’s highest science advisory body, established threemajor research areas for its long-term plan: Understanding the Brain,
Protecting the Brain, and Creating the Brain. The Brain Science Institute of
RIKEN (the Institute of Physical and Chemical Research) was started in
1997, closely following this long-term plan (Normile 1997). Ten years havepassed since the Japanese Century of Brain was launched, but, surprisingly,
according to Shun-ichi Amari, president of RIKEN’s Brain Science Institute,the 2005 budget for Japanese brain science was only 18 billion yen, and it
has been decreasing over the past 7 years after peaking in 2000 (Amari 2006),
in sharp contrast to the magnitude of the original proposal.
Among the three ways of exploring the brain, Creating the Brain is unique toJapan. Computers and robots have improved dramatically, but they are still
greatly inferior to humans in most brain functions such as natural language
processing, visual scene understanding, smooth and dexterous manipulationof objects, and common-sense reasoning, all of which can be executed almost
effortlessly by humans, at least at the conscious level. This situation
indicates that computer science and robotics did not make as much progress
as expected 50 years ago and that neuroscience has not provided sufficient
information on these areas. The “Creating the Brain” was motivated by thisreflection, with the aim to develop brain-style information processing andcommunication technologies. In Particular, the development of “brain-style”
computer systems, with human-like intellectual and emotional capabilities,
was set as this work’s long-term goal.
Over the past 10 years, millions of artificial retinal chips were sold forseveral billion yen per year, and many kinds of “neuro-” electric home
appliances with learning capabilities based on artificial neural networks
have been popular in the Japanese market. However, the current status ofresearch is still far from the development of “brain-style” computer systems.
The main difficulty is that conventional medical and biological approaches inneuroscience do not provide the type of knowledge that is essential forcomputer development. The accumulation of a vast amount of data on brain
locations (which function is lost when a specific brain region is destroyed), on
substances (a specific molecule is critical for some brain functions), and on
correlations (firing rates of neurons in a specific brain area correlated to ahypothetical element in cognition) is unfortunately nearly useless inunraveling neural mechanisms, at least to the extent that principles of
information processing can be learnt from the brain. On the other hand,these kinds of knowledge are useful for Protecting the Brain (e.g. avoiding
the speech area in neurosurgical ablation) or at some levels ofUnderstanding the Brain.
We cannot state that we fully understand brain functions by simply
accumulating the above types of data on locations, substances andcorrelations. In the deepest sense of “Understanding the Brain”, we should
reveal the secrets of neural mechanisms of brain functions to the extent that
we can “Create the Brain”. Furthermore, definitions of computational
neuroscience vary depending on the researchers involved in this area. For
some, approaches that depend on computers, mathematics, physics,
engineering or David Marr’s computational theory could be called“Computational Neuroscience”. From the mid-80s, we initiated our own
approach to computational neuroscience with the following definition: “We
elucidate information processing of the brain to the extent that artificial
machines, either computer programs or robots, can be built to solve the same
computational problems that are solved by the human brain, using
essentially the same principles”. Many researchers, including our group,
started to believe that the only possible methodology to fully understand how
the brain works is to build or reconstruct artificial systems that can realizebrain functions. We call this approach “Understanding the Brain by Creating
the Brain”, and it is currently the major approach taken in the Japanese“Creating the Brain” area; furthermore, this might be regarded as theJapanese definition of computational neuroscience. Generally speaking, thisis the new direction of “analysis by synthesis” in biological disciplines, and it
has common ground with synthetic biology in systems biology orbioinformatics of molecular and cellular biology, where the goal is to create
artificial organisms in silico or in wet biology.

2. Humanoid Robots as a Neuroscience Tool
Even if we could create an artificial brain, we could not investigate its
functions, such as vision or motor control, if we just let it float in incubation
fluid in a jar. The brain must be connected to sensors and a motor apparatus
so that it can interact with its environment. A humanoid robot controlled byan artificial brain, which is implemented as software based on computationalmodels of brain functions, seems to be the most plausible candidate for thispurpose, given currently available technology. With the slogan of“Understanding the Brain by Creating the Brain”, in the mid-80s we started
to use robots for brain research (Miyamoto & Kawato 1988), and about 10
different kinds of robots have been used by our group at Osaka University’s
Department of Biophysical Engineering, ATR Laboratories, ERATO Kawato
Dynamic

Brain

Project(ERATO

1996-2001),

and

ICORP

KawatoComputational Brain Project (ICOPR 2004-2009).
An optimal computational theory for one type of body may not be optimal forother types of bodies. Thus, if a humanoid robot is used for exploring andexamining neuroscience theories rather than for engineering, it should be as
close as possible to a human body. Within the ERATO project, in
collaboration with the SARCOS research company led by Professor Stephen
C. Jacobsen of the University of Utah, Dr. Stefan Schaal as a robot groupleader and his colleagues developed a humanoid robot called DB (Dynamic

Brain) (Figure 1) with the aim of most closely replicating a human body,
given the robotics technology of 1996. DB possessed 30 degrees-of-freedom
and human-like size and weight. DB is mechanically compliant like a human
body, and unlike most electric-motor-driven and highly-geared humanoid
robots, because the SARCOS’ hydraulic actuators are powerful enough to
avoid the necessity of using reduction mechanisms at the joints. Within its
head, DB is equipped with an artificial vestibular organ (gyro sensor), whichmeasures head velocity, and four cameras with vertical and horizontal
degrees-of-freedom. Two of the cameras have telescopic lenses corresponding
to foveal vision, while the other two have wide-angle lenses corresponding toperipheral vision. SARCOS developed the hardware and low-level analog
feedback-loops, while the ERATO project developed high-level digital
feedback-loops and all of the sensory-motor coordination software.
The photographs in Fig. 1 introduce 14 of the more than 30 different tasks
that can be performed by DB (Atkeson et al. 2000). Most of the algorithms
used for these task demonstrations are based roughly on principles of
information processing in the brain, and many of them contain some or all of
the three learning elements: imitation learning (Miyamoto et al. 1996;
Schaal 1999; Ude & Atkeson 2003; Ude et al. 2004; Nakanishi et al. 2004),
reinforcement learning, and supervised learning. Imitation learning(“Learning by Watching”, “Learning by Mimicking” or “Teaching by
Demonstration”) was involved in Okinawan folk dance “Katya-shi” (Riley etal. 2000) (A), three-ball juggling (Atkeson et al. 2000) (B), devil-sticking (C),air-hockey (Bentivegna et al. 2004a; Bentivegna et al. 2004b) (D), polebalancing (E), sticky-hands interaction with a human (Hale & Pollick 2005)(L), tumbling a box (Pollard et al. 2002) (M), and a tennis swing (Ijspeert etal. 2002) (N). The air-hockey demonstration (Bentivegna et al. 2004a;
Bentivegna et al 2004b) (D) utilizes not only imitation learning but also a
reinforcement-learning algorithm with reward (a puck enters the opponent’sgoal) and penalty (a puck enters the robot’s goal) and skill learning (a kind of
supervised learning). Demonstrations of pole-balancing (E) and visually
guided arm reaching toward a target (F) utilized a supervised learningscheme (Schaal & Atkeson 1998), which was motivated by our approach tocerebellar internal model learning, introduced in the next section.
Demonstrations of adaptation of the vestibulo-ocular reflex (Shibata &Schaal 2001) (G), adaptation of smooth pursuit eye movement (H), andsimultaneous realization of these two kinds of eye movements together withsaccadic eye movements (I) were based on computational models of eye
movements and their learning (Shibata et al. 2005). Demonstrations ofdrumming (J), paddling a ball (K), and a tennis swing (N) were based on
central pattern generators.
The ICORP Computational Brain Project (2004-2009), which is aninternational collaboration project with Prof. Chris Atkeson of the Carnegie
Mellon University, follows the ERATO Dynamic Brain Project in its slogan“Understanding the Brain by Creating the Brain” and “Humanoid Robots asa Tool for Neuroscience”. Again in collaboration with SARCOS, at the
beginning of 2007 Dr. Gordon Cheng as a group leader with his colleagues
developed a new humanoid robot called CB (Computational Brain), shown in
Fig. 2. CB is even closer to a human body than DB. To improve themechanical compliance of the body, CB also used hydraulic actuators rather
than electric motors. The biggest improvement of CB over DB is its
autonomy. DB was mounted at the pelvis because it needs to be powered by
an external hydraulic pump, through oil hoses arranged around the mount.
A computer system for DB was also connected to DB by wires. Thus, DB
could not function autonomously. In contrast, CB carries both onboard power
supplies (electric and hydraulic) and a computing system on its back, andthus it can function fully autonomously. CB was designed for full-bodyautonomous interaction, for walking and simple manipulations. It isequipped with a total of 51 degrees-of-freedom (DOF): 2x7 DOF legs, 2x7
DOF arms, 2x2 DOF eyes, 3 DOF neck/head, 1 DOF mouth, 3 DOF torso, and
2x6 DOF hands. CB is designed to have similar configurations, range of
motion, power, and strength to a human body, allowing it to better reproduce
natural human-like movements, in particular for locomotion and object
manipulation.

Locomotion and posture control are the most challenging research topics fora human-like and autonomous humanoid robot such as CB, because of its
mechanically compliant and 51-DOF body. Within the ICORP CB Project,biologically inspired control algorithms for locomotion have been studied
using the three different humanoid robots shown in Fig. 3 (a: DB-chan(Nakanishi et al. 2004), b: Fujitsu Automation HOAP-2 (Matsubara et al.
2006), c: CB (Morimoto et al. 2006)) as well as the SONY small-size
humanoid robot QRIO (Endo et al. 2005) as test beds. Successful locomotion
algorithms utilize various aspects of biological control systems, such asneural networks for central pattern generators, phase resetting by various
sensory feedbacks including adaptive gains, and hierarchical reinforcement
learning algorithms. In the demonstration of robust locomotion shown in Fig.
3a, three biologically important aspects of control algorithms are utilized:
imitation learning, a nonlinear dynamical system as a central patterngenerator, and phase resetting by a foot-ground contact signal (Nakanishi etal. 2004). First, a neural network model developed by Schaal et al. (2003)
quickly learns correctly demonstrated locomotion trajectories by humans orother robots. In order to synchronize this limit-cycle oscillator (central
pattern generator) with a mechanical oscillator functioning through therobot body and the environment, the neural oscillator is phase-reset byfoot-ground contact. This guarantees stable synchronization of neural andmechanical oscillators with respect to phase and frequency. The achieved
locomotion is quite robust against different surfaces with various frictions
and slopes, and it is human-like in the sense that the robot body’s center ofgravity is high while the knee is nearly fully extended at foot contact. This is
in sharp contrast to locomotion engineered by zero-moment point control,
which usually induces a low center of gravity and bent knees.

3. Cerebellar Internal Models
Many of the DB demonstrations mentioned in the previous section are based

on the cerebellar internal model theory, which we have been developing overthe past 25 years (Kawato et al. 1987; Kawato & Gomi 1992; Kawato 1999):
each micro-zone of the cerebellar cortex learns to acquire an internal modelof some specific object in the external world, such as an arm or eye of one’s
own body, manipulative tools, or another person’s brain. In this section, we
briefly describe the background of this theory, the theory itself and someexperimental support of this theory.
Among the many different models of the cerebellum, learning models havealways been the most attractive, versatile and influential. The learningmodels are based on unique structures of the cerebellar cortex. The Purkinje
cell, the only output neuron in the cerebellar cortex, receives two major
excitatory inputs from climbing fibers and parallel fibers. A single climbing
fiber axon makes multiple synapses on a single Purkinje cell and inducesvery strong excitatory postsynaptic potentials, which lead to complex spikes
that consist of a few Ca2+ spikes. By contrast, 200,000 synapses from parallel
fibers, which are the axons of 1010~11 granule cells, generate simple spikes,
each of which is a single Na+ spike. Although early cerebellar learningmodels (Marr 1969; Ito 1970; Albus 1971) proposed that the cerebellum
performs pattern recognition, probably influenced by a “perceptron”, they are
not consistent with physiological data showing that simple spike firing ratesof Purkinje cells temporally change and encode dynamic and kinematicfeatures of movements. Recent studies suggest that the human cerebellum isimportant for sensory and cognitive functions as well as for motor control.
Might there be a general computation realized by the cerebellar cortex thatcovers the sensory, cognitive and motor domains and also conforms toexisting hypotheses on such matters as timing, coordination, and error
correction? Internal models seem to provide the most versatile andhypothesis-compatible computational entity satisfying these requirements.
Internal models are neural networks inside the brain (“internal”) that can
mimic (“model”) the input-output characteristics of some dynamical process
outside the brain. In a motor control context, a forward model of a body can
predict sensory consequences of movement from the efference copy of a motor

command. This is because the controlled objects in movements generate a
trajectory from the motor commands. By contrast, an inverse model of a bodyand the external world can compute the necessary motor commands from adesired movement pattern. Internal models are useful for predictive
computations in sensory, cognitive and motor functions. If an inverse
dynamics model for a controlled object possessing dynamics and kinematics
with multiple degrees of freedom is destroyed, the movement becomes
clumsy, slow and not well corrected. This is because neither feedforward
control (which is dependent on the inverse model) nor sophisticated feedback
control (which is dependent on the forward model) is available. Thus, control
must rely on poor and crude feedback control alone.
About 25 years ago, my colleagues and I proposed that different parts of thecerebellum contain either forward or inverse models (Tsukahara & Kawato
1982; Kawato et al. 1987) and also that a computational scheme called
feedback-error-learning is used to acquire an inverse model in the supervised
learning scheme as shown in Fig. 4 (Kawato et al. 1987, Kawato 1990,
Kawato & Gomi 1992). If supervised learning takes place in the brain formotor control, the following difficult computational problem called “distalteacher”, formulated by Jordan & Rumelhart (1992), should be resolved. The
error between desired and actual movement patterns can be measured by
sensory organs, but these movement errors could be very different in spatial
coordinates as well as in temporal dynamics from the necessary error signal
for the motor command. Furthermore, it should be emphasized that there
exists no teaching signal for motor commands in supervised motor learning,since if it did exist, it would be directly utilized for motor control and there
would be no need for learning.
In the feedback-error-learning model depicted in Figure 4, the inverse model
first transforms a desired trajectory into a feedforward motor command.
Second, a crude feedback controller generates a feedback motor commandfrom the sensory error. Third, the summation of the feedforward andfeedback motor commands is sent to the controlled object. Finally, and most

importantly, the feedback motor command is used as the error signal for
motor commands in the supervised learning to acquire the inverse model.
Stability and convergence of the feedback-error-learning algorithm havebeen mathematically proven (Nakanishi & Schaal 2004) as well as
successfully applied to several robotic demonstrations (e.g. Miyamoto et al.
1988). Here, simple spikes represent feedforward motor commands and theparallel fiber inputs represent the desired trajectory as well as the sensory
feedback of the current status of the controlled object. A microzone of thecerebellar cortex constitutes part of an inverse model of a specific controlled
object, such as the eye or arm. Most importantly, climbing fiber inputs are
assumed to carry a copy of the feedback motor commands generated by the
feedback control circuit. Thus, the complex spikes are assumed to be
trajectory error signals already expressed in motor command coordinates.
Climbing fiber inputs are delayed by 100 msec with respect to the
responsible parallel fiber inputs that caused the movement and the resulting
error, but the spike-timing dependency of parallel-fiber Purkinje-cell longterm depression (LTD) selectively decreases the efficacy of only the
responsible parallel fiber synapses, explained in the next section.
Kenji

Kawanoandhiscolleagues

supportedthecerebellar

feedback-error-learning model with neurophysiological studies in the ventral
paraflocculus of the monkey cerebellum during ocular-following responses(Kawano 1999; Kawato 1999; Shidara et al. 1993; Gomi et al. 1998;
Kobayashi et al. 1998; Takemura et al. 2001; Yamamoto et al. 2002).
Ocular-following responses are tracking movements of the eyes evoked by
movements in a large visual scene that are thought to be important forvisual stabilization of gaze. In the neural circuit controlling this response,
the phylogenetically older and cruder feedback circuit is comprised of theretina, the accessory optic system and the brain stem. The phylogeneticallynewer and more sophisticated feedforward pathway and the inverse
dynamics model correspond to the cerebral and cerebellar cortical pathway
and the cerebellum, respectively. The sensory error signal is computed as theimage motion on the retina (retinal slip). This is different from the scheme of

Fig. 4, where some neural mechanisms explicitly compute the difference(error) between the desired and actual states. However, in the case of eye
movement control such as ocular-following responses or vestibulo-ocularreflexes, the desired state is eye movement in which the retinal image does
not move, and thus the optical computation (retinal slip detection) can
replace subtraction of the desired and actual states. On the other hand, thedesired trajectory information transmitted to the flocculus for thevestibulo-ocular reflex is the head motion signal detected by semi-circular
canals, whereas for the ocular following responses this is the image motion
signal conveyed by the visual areas in the cerebral cortex.
During ocular following responses, the time-courses of simple spike-firing
frequency show complicated patterns. However, they were quite accurately
reconstructed by using an inverse-dynamics representation of eye movement(Shidara et al. 1993; Gomi et al. 1998). The model fit was good for the
majority of the neurons studied under a wide range of visual stimulus
conditions, different eye movements, and different firing patterns. The same
inverse dynamics analysis for the medial superior temporal area (MST) of
the cerebral cortex and dorsolateral pontine nucleus, which provide visualmossy fiber inputs, revealed that their firing time-courses were not wellreconstructed from motor commands but were well reconstructed from visual
retinal slip (Takemura et al. 2001). Taken together, these data suggest thatthe ventral paraflocculus is the major site of the inverse dynamics model ofthe eye for ocular-following responses.
Motor commands, conveyed by simple spikes, should be directly modified
and acquired by LTD while guided by motor-command errors, which areconveyed by climbing fiber inputs. For this to work, the climbing fiber inputs
need to convey comparable details of temporal and spatial information to themotor commands, but the ultra-low discharge rates of the latter would
appear to rule this out. This apparently discrete and sporadic nature ofclimbing fiber inputs was characterized as an unexpected event detector, andit suggested a reinforcement-learning-type theory. However, if thousands oftrials are averaged, the firing rates actually conveyed very accurate andreliable time-courses of motor command error within a few hundreds ofmilliseconds (Kobayashi et al. 1998). More specifically, complex spiketime-courses are better reconstructed by retinal slip than eye movements,
but in the motor command coordinates rather than in visual coordinates asdetailed below. Because LTD has recently been shown to temporally
leaky-integrate a postsynaptic Ca2+ signal (Tanaka et al. 2007) as introducedin the next section, it can perform temporal averaging. Consequently, thefiring probability of climbing fiber inputs can convey high-frequency
temporal information that matches information of the dynamic commandsignal.
The preferred directions of MST and pontine neurons were evenly
distributed over 360 degrees. Thus, the visual coordinates are uniformly
distributed over all possible directions. On the other hand, projections of the
3-D spatial coordinates of the extraocular muscles were in either a horizontal
or vertical direction, and they were entirely different from the visual
coordinates. Preferred directions of simple spikes were either downward or
ipsiversive (i.e. toward the same side), and at the site of each recording,electrical stimulation of a Purkinje cell elicited eye movement toward thepreferred direction of the simple spike (Kawano 1999). These data indicatethat the simple spike coordinate framework is already that of the motor
commands. Thus, at the parallel fiber-Purkinje cell synapse, a drastic
visuomotor coordinate transformation occurs. The neural representation
dramatically changes from population coding in the MST and pontine
nucleus to firing-rate coding of Purkinje cells at the parallel fiber-Purkinje
cell synapse. What, then, is the origin of this drastic transformation? The
theory proposes that the complex spikes, and ultimately the accessory optic
system, are the source of this motor command spatial framework. The
preferred directions of pretectum neurons are upward, and those of thecontralateral

nucleus of optic tract neurons are ipsiversive (thuscontraversive for the cerebellum). Their signals are conveyed to the inferior
olive and further to the cerebellum to produce the complex spikes. Insummary, the coordinate frame of the climbing fibers was vertical andhorizontal, forming the motor coordinates, not the visual coordinates of MSTor pontine nucleus. A realistic neural network model successfully simulated
all of these experimental findings (Yamamoto et al. 2002). The coordinates ofsemicircular canals and the extraocular muscles are very similar, and it israther difficult to evaluate coordinate transformation in the vestibulo-ocularreflex and flocculus compared with ocular following responses. However,recent physiological studies indicate that climbing fiber inputs encode motor
error rather than sensory error (Winkelman & Frens 2006). This argues
against a class of model that assumes sensorimotor transformation
downstream of the Purkinje cells, including de-correlation models (Dean etal. 2002).
For arm movements under multiple force fields, firings of many Purkinjecells correlate with dynamics (Yamamoto et al. 2007). fMRI studies mappedforward and inverse models of tools and manipulated objects in the
cerebellar cortex (Imamizu et al. 2000; Kawato et al. 2003). Thus, we havebeen accumulating experimental support for the cerebellar internal model
theory for arm movements, the human cerebellum for tool use, and forward
models in addition to inverse models. Having said this, I must admit thatexperimental support outside the inverse dynamics model for the ocularfollowing responses in the ventral paraflocculus is rather indirect andcircumstantial, and these issues are still controversial (for example, compare
Yamamoto et al. 2007 and Pasalar et al. 2006). However, because the neuralcircuit of the cerebellar cortex is so uniform for different regions anddifferent functions, unlike the cerebral cortex, I predict that a theory fully
supported even for only a single function in a small region of the cerebellum
should possess some computational principles that can be generalized to allfunctions and regions of the cerebellum.

4. Systems-Biology Models of Cerebellar LTD

If the long-term depression (LTD) of parallel fiber-Purkinje cell synapses isthe elementary cellular process of supervised learning, as postulated in
cerebellar learning theory, it must satisfy the following three requirements.
First, LTD should be associative. That is, LTD should be induced only whenboth parallel fiber inputs and climbing fiber inputs are conjunctively
activated. Second, LTD should be maximally induced when climbing fiber
activation is delayed by 50 to 200 msec with respect to parallel fiber
activation. This is because the time window of LTD should compensate forthe delay of the reafferent climbing fiber inputs to feed back theconsequences of the parallel-fiber-simple-spike-induced movements. Third,
LTD should be synapse-specific; only a parallel fiber synapse that isconjointly activated with a climbing fiber input should be depressed and notother synapses. However, a considerable number of studies have reported
violations of these three requirements under some experimental conditions;consequently, LTD has been questioned several times as the elementary
process of cerebellar supervised learning (Llinas et al. 1997) and has been
postulated for other functions, such as the normalization of total synaptic
efficacies on a single Purkinje cell (de Schutter 1995). The kinetic simulation
model of LTD signal-transduction pathways in a Purkinje-cell dendritic
spine (Fig. 5), which has been developed in a series of publications from ourgroup (Kuroda et al. 2001; Doi et al. 2005; Ogasawara et al. 2007) and wasrecently supported experimentally (Tanaka et al. 2007), provided a coherent
perspective on these confusing experimental data and competing theoretical
hypotheses, and it is introduced in this section.
Climbing fiber inputs activate AMPA receptors (AMPA-Rs) on dendritic
shafts and induce depolarization, which opens the voltage-gated calcium
channels (VGCCs) on a spine as indicated by the broken line in Fig. 5,
leading to Ca2+ influx from the extracellular space into the spine head,reaching a concentration of about 1 µM . Parallel fiber inputs also induce
AMPA-R activation on dendritic spines, just as a climbing fiber input does
but with a smaller depolarization and Ca2+ influx. In addition, they
simultaneously activate metabotropic glutamate receptors (mGluRs).

Furthermore, they presynaptically synthesize nitric oxide (NO), which
diffuses into the spine. In the metabotropic pathway, activated mGluRs
induce production of inositol 1,4,5-triphosphate (IP3) and diacylglycerol(DAG) via G-proteins (Gq) and phospholipase C " (PLC " ) at thepostsynaptic density. IP3 then diffuses into the cytosol and binds to IP3
receptors (IP3Rs), which are IP3-gated Ca2+ channels on the intracellular!
!
Ca2+ stores, such as the endoplasmic reticulum (ER). Cytosolic Ca2+ abovesome threshold triggers a regenerative cycle of IP3-dependent andCa2+-induced Ca2+ release (depicted as a red positive feedback loop between
Ca2+ and IP3Rs in Fig. 5) from the ER via IP3Rs opened by binding of both IP3
and Ca2+. This means that the open probability of IP3Rs increases with Ca2+
concentration and the Ca2+ threshold of this regenerative cycle decreases
with IP3 concentration. An increase in cytosolic IP3 concentration was
simulated to be maximal at 3 µ M about 100 msec after the parallel fiber
input. This slow rate of increase in IP3 concentration is due to the slow
dynamics of the metabotropic pathway. Because Ca2+ influx via VGCCs due!
to climbing fiber activation is an immediate event, when the climbing fiber
input is delayed about 100 msec with respect to the parallel fiber input, Ca2+
increase coincides with IP3 increase, and together they trigger the
Ca2+-induced Ca2+ release of 10 µ M . If either climbing fiber input or parallel
fiber input alone is activated, this large Ca2+ release does not occur because
either IP3 or Ca2+, respectively, does not reach the threshold for the!
regenerative cycle. Consequently, the Ca2+ nonlinear excitable dynamics
model of Doi et al. (2005) (right 1/3 of Fig. 5) reproduced the associative
nature and the spike-timing-dependent plasticity (STDP) of cerebellar LTD.
In the model proposed by Kuroda et al. (2001), the large Ca2+ increase, whichis caused by the conjunctive parallel fiber input and the delayed climbing
fiber input, induces the activation of linear cascades of phosphorylation ofprotein kinase C (PKC) followed by phosphorylation and then internalizationof AMPA-R, which explains the early phase of LTD up to about 10 minutes.
On the other hand, the intermediate phase of LTD up to several tens ofminutes is mediated by a mitogen-activated protein (MAP) kinase-dependentpositive feedback loop, which consists of PKC, Raf, MAP/extracellularsignal-regulated kinase (MEK), MAP kinase, cytosolic phosopholipase A2(PLA2), arachidonic acid (AA), and back to PKC (the magenta loop in the
middle of Fig. 5). This positive feedback loop has two stable equilibrium
points separated by a saddle point that determines the threshold of LTD. At
the low equilibrium point, the MAP-kinase positive feedback loop is not
activated, but if strong or long-lasting Ca2+ elevation occurs, the state jumps
to the higher equilibrium over the saddle point and AMPA-Rs continue to bephosphorylated and internalized during this prolonged activation of thepositive feedback loop. Consequently, this MAP-kinase positive feedback
model (middle 1/3 of Fig. 5) explains the long-term nature of LTD.
In the NO model (left 1/3 of Fig. 5), which was first proposed by Kuroda et al.
(2001) and then extensively studied in combination with electrical propertiesof dendrites by Ogasawara et al. (2007), NO activates soluble guanylyl
cyclase (sGC), and the activated sGC catalyzes the conversion of GTP into
cGMP, which activates cGMP-dependent protein kinase (PKG). PKG
phosphorylates
G-substrate

its

substrate,

preferentiallyG-substrate,

inhibits protein

and

the

phosphorylated

phosphatase

2A (PP2A).

Activated PP2A dephosphorylates MEK and AMPA-R. Inhibition of thedephosphorylation of MEK was required for the activation of the
MAP-kinase positive feedback loop. Therefore, by ultimately inhibiting PP2A
and activating the positive feedback loop, NO plays a permissive role in the
induction of LTD.
The Ca2+ dynamics model coherently reproduced both the associative andnon-associative natures of LTD, and the spike-timing dependency of
associative LTD, as follows. Within the physiological ranges of input
strength, the conjunctive stimulation of the parallel fiber input and thedelayed climbing fiber input is essential to induce a large, regenerative Ca2+
increase,

which

is

necessaryforcrossingtheMAP-kinase

positive-feedback-loop threshold and inducing LTD. However, if stimulationof parallel fiber bundles is strong enough, or if uncaged Ca2+ or IP3 is large

enough, these stimuli alone can increase the Ca2+ concentration so that it
crosses the Ca2+ dynamics threshold, resulting in a large enoughCa2+-induced Ca2+ release from the ER to reproduce non-associative LTD. In
the hippocampal and cerebral pyramidal neurons, backpropagation ofsomatic action potentials removes the maganesium block of NMDA receptors,
and when coincided with glutamate-release by pre-spiking, NMDA receptors
are fully activated and lead to supralinear Ca2+ influx, resulting in the
induction

of

LTP.

The

coincidenceofglutamate-releasewithbackpropagation in this order within a ten-millisecond order is essential for
NMDA receptor-dependent STDP there. Backpropagation of somatic action
potentials require firing of postsynaptic cells, which is the essential conditionof Hebbian learning. Neither backpropagation nor the functional NMDA
receptors exist in Purkinje cells, suggesting completely different cellularmechanisms of Purkinje-cell supervised learning with 100-msec STDP from
cerebral Hebbian learning with STDP of tens of msec. These systems-biology
models explain diverse LTD experiments and clearly demonstrate that LTD
is a supervised learning rule, and not anti-Hebbian as characterizedelsewhere (Dayan & Abbott 1999).
The excitable dynamics of the Ca2+ model and bistable dynamics of the
MAP-kinase positive feedback loop model may provide a possible molecularmechanism to resolve the “plasticity-stability” dilemma, maintaining
memory stably while still being sensitive to delicate environmental changes.
Each spine is equipped with a surprisingly small number of molecules (e.g.
only 40 AMPA-Rs). With a cascade of excitable and bistable dynamics
starting from Ca2+ dynamics, followed by the MAP-kinase positive feedback
loop, internalization of AMPA receptors, a possible change in cytoskeleton
and membrane proteins, and finally a morphological change in the spine,
LTD could be induced by very delicate and low-energy inputs but still
maintain acquired memory over a prolonged period of days, even understochasticity due to the small numbers of molecules.
The synapse-specificity of LTD is also well explained by the NO modelcombined with an electrical cable model of Purkinje-cell dendrite and spines,
the Ca2+ dynamics model, and the MAP-kinase positive feedback loop model(Ogasawara et al. 2007). When no nearby parallel fiber is activated and NO
is low, not even a conjunction of the parallel fiber and climbing fiber inputs
can induce LTD. If too many surrounding parallel fibers are stimulated andNO is very high, even the synapse, which is not stimulated, exhibits LTD
only by climbing fiber stimulation. In this case, a spread of LTD occurs andthe synapse-specificity of LTD is lost. Only when a modest number ofparallel fibers are activated and NO is within an intermediate range cansynapse-specific LTD occur; the contextual information for storing different
motor skills might be conveyed by this gating mechanism. This strongly
predicts the sparseness of parallel fiber coding in vivo, which has recently
gained much experimental support (e.g. Chadderton et al. 2004).
Taken together, the LTD kinetic simulation models provide a comprehensive
account of several seemingly diverse, conflicting and unrelated experimental
data within the framework of complicated nonlinear dynamics regulated by
Ca2+, IP3, and NO concentrations. We must note that because of thenonlinear summation effects of these signaling molecules, non-physiologicalstimuli in vitro, such as parallel fiber bundle stimulation and uncaging (i.e.
direct and instant application of interested molecules by photolysis of its
caged-compounds) of Ca2+ or IP3, might induce qualitatively different
characteristics from in vivo LTD in terms of its associative nature,
spike-timing dependency, and synapse-specificity.
The systems-biology models of LTD recently gained strong experimental
support. First, in the same Journal of Neuroscience that published our paperon the Ca2+ dynamics model (Doi et al. 2005), IP3 was found to increase
slowly with a 100-msec time scale in Purkinje cell dendrites after parallel
fiber stimulation (Okubo et al. 2004). Slow IP3 increase, the key prediction of
our model, was thus confirmed. Yet, we must note that the measurement
was not for a single spine because of technical difficulties, and futurehigh-spatial resolution experiments are expected to examine this prediction

more precisely. Second, very recently, George Augustine and his colleagues
examined with us Ca2+ dose-response curves of Purkinje-cell LTD in a sliceby systematically varying magnitudes and durations of the Ca2+ elevation,
which was controlled by uncaging and simultaneously measuring Ca2+
(Tanaka et al. 2007). They demonstrated that LTD is a highly cooperative
phenomena with large Hill coefficients, and suggested that the MAPK
positive feedback loop leaky-integrates the incoming Ca2+ time course anddetermines whether to trigger the entire process of LTD in an all-or-nothingmanner by comparing the leaky-integrated Ca2+ with its threshold for thebistable dynamics. Because the measurements of the postsynaptic currents
were summed over roughly 30 spines and not shown for a single spine, theabove suggestion is not yet conclusive, but one enigma regarding thestability of our memory might be nearly resolved in the following scenario.
Bistability of nonlinear dynamics within the MAPK positive feedback loop
guarantees memory maintenance in the first place, and cerebellar memory is

discrete in its nature at the single-spine level.

5. Toward Manipulative Neuroscience
Ten years have not yet elapsed since the cerebellar-LTD MAPK
positive-feedback-loop model was first published (Kuroda et al. 2001), before
experiments firmly supported its essential proposition that the bistability of
nonlinear dynamics is the elementary process of memory (Tanaka et al.
2007) (see section 4). This is in sharp contrast to the cerebellar internal
model theory, for which two completely opposing experimental papers were
published (Pasalar et al. 2006; Yamamoto et al. 2007), although 25 years
have passed since its first proposal in a preliminary form (Tsukahara &Kawato 1982) (see section 3). One possible reason for this marked difference
could be that LTD within a single spine is a much more small-scale andsimpler phenomenon than learning of internal models in sensory-motor
coordination as well as in cognitive function, which utilizes not only the
cerebellum but also all of its loops with the cerebral cortex and other brain

regions. Or, some might even argue that the former theory is correct but thelatter theory is wrong. I of course do not agree with this view but believe thatthe biggest difference between the two concepts is in the availability ofexperimental

methodology

thatpermits

manipulationofessential

information within a given system. In the LTD example within the molecular
cellular biology domain, the time-course of Ca2+ concentration, the mostimportant intracellular signal, can be directly manipulated by uncaging anddirectly

measuredbybioimaging

techniques.

In

contrast,

systems

neuroscience does not possess any experimental technique for direct
manipulation or control of information at the system level; information suchas an error signal in motor command coordinates is carried by climbing
fibers, or feedforward motor commands represented by simple spikes, in the
case

of

cerebellarinternal

model

theory.

Anatomical

ablationorelectrocoagulation can destroy neural substrates but cannot manipulate
information. Electrical stimulation may excite or suppress neurons
indefinitely but cannot manipulate information such as the error signal or
motor commands. Furthermore, in most cases at the system level, we do noteven know how neural firing and its coherence represent information.
Consequently, computational theories are essential to make the initialassumptions about what is represented in the brain and how it is neuronally
represented.
Several decades ago, systems neuroscientists could get excited by finding amere temporal correlation between neural firing rates or an fMRI signal,with some experimental parameters (e.g. attributes of sensory stimuli,
movement parameters), or with some hypothetical cognitive aspects of thetask performed by subjects. However, as explained in section 1, mere
accumulation of this kind of correlation data does not lead to understanding
the neural mechanism of any brain function, in other words, it does not leadto a rigorous science that explains cause and effect relationships. I recently
heard the criticism from scientists in other disciplines that systems
neuroscientists do not know what should be revealed before they can statethat they ultimately understand the brain. Answering this criticism, I definethe ultimate goal of system neuroscience as follows. We can simultaneously
measure the activities of all neurons in the brain, and we can freely changethese activities experimentally. Thereby we can control the arbitrary
information in the brain based on computational theories that explain how
information is represented by neural activities, and the theories can predictthe resulting changes in the organisms regarding their sensation,
movements, emotions and thoughts. The current status of systems
neuroscience is far from reaching this ultimate goal in all aspects ofmeasurement, control, manipulation of information, and theory. However,regarding measurement of neuron and brain activities, much progress has
been made, such as optical imaging of Ca2+ or voltage and many non-invasive
brain activity measurement techniques. Regarding control of neuron
activities, promising work in development includes the juxta-cellular clamp,
the voltage clamp for simulating synaptic currents (dynamic clamp), andgenetic

engineering

methodsofusingchannelrhodopsin-2

and

its

photostimulation (Boyden et al. 2005). For simpler organisms such as
C-elegans, simultaneous measurement and control of all neural activities
might be achieved within ten years. However, at least for several decades, itwill be very difficult for mammals in vivo, and it seems almost impossible forhumans unless new physical principles for observation and control areinvented.
Much progress has been made with respect to close interactions between
computational

theoriesandneurophysiologicalandneuroimaging

experiments. Many experiments are nowadays designed for proving or
disproving predictions of computational models, or at least motivated by
them.

In

the

new

experimental“computational-model-based

paradigms,

whichneurophysiology

are

named
and

computational-model-based neuroimaging”, interactions between theories
and experimental designs are even more intimate. Here, animal or humansubjects are given sensory stimuli and a series of reward/penalty signals andasked to execute some tasks and conduct movements. Computational models
such as the cerebellar internal model theory or basal-ganglia reinforcement

learning theory are employed to quantitatively simulate subjects’ behaviors
while the models are given exactly the same sensory stimuli and series ofreward/penalty signals for the subjects. First of all, we expect thatcomputational models fairly well reproduce subjects’ behaviors. Then, withineach computational model, there exist some hypothetical computationalvariables such as the error signal or the feedforward motor command in thecase of the cerebellar internal model theory, or the reward prediction error orvalue functions in the basal-ganglia reinforcement learning theory example.
Each of these hypothetical variables can be used as an explanatory variablein correlation analysis with an fMRI signal or firing frequency of neurons. By
using this correlation analysis, researchers can make statements such as“climbing fiber inputs may represent error signals in motor command
coordinates” (Kobayashi et al. 1998) or “putamen neurons may encode action
and state dependent reward prediction…” (Samejima et al. 2005, Haruno &Kawato 2006a) “while the caudate nucleus may encode reward-prediction
error” (Haruno & Kawato 2006b). Our group as well as other groups haverecently published a fair amount along this experimental paradigm(Imamizu et al. 2000; Imamizu et al. 2003; Haruno et al. 2004; Tanaka et al.
2004;

Samejima

et

al.

2005;

Haruno&Kawato2006a,

2006b).

Computational-model-based neuroscience is a major advance over theclassical approach of simply correlating neural or brain activities withstimulus attributes, movement parameters or some conceptual aspects of thetask. If only the former two had been utilized as explanatory variables,
cognitive functions remote from sensory and motor interfaces of the brain
would never have been directly studied. An ad hoc conceptual aspect of thetask could be informative in giving some clues to brain mechanisms, but thisis far from quantitative understanding or revealing cause and effectrelationships. A computational-model-based approach is attractive first
because it is a quantitative theory, which can be disproved because it makes
testable predictions. Second, it tries to reproduce behaviors as well as neuralor brain activities. Third, computational models try to explain the whole
sequence of information processing necessary for some task, while ad hoc
cognitive hypotheses tend to explain just a fragmentary aspect within thewhole processing sequence. However, we tend to be frustrated by thedifficulties of computational models and this approach’s weakness indemonstrating cause and effect relationships, even after advocating thisapproach for 10 years.
The first difficulty is in computational models. We still have not yet
developed a powerful enough computational model to simultaneously explaina large variety of behaviors and whole-brain activities. Thus, in many cases,
only a small part of the behavioral data can be reproduced by a single model.
In this case, many models could equally well reproduce experimental data,
and thus it is difficult to select one best model. The second difficulty is thelikely complicated relationship between information and its carrier, such asneural firing rates and coherence or fMRI signals, due to possible population
coding, sparse coding, or temporal coding. The third and most criticaldifficulty is that what we can demonstrate in computational-model-basedneuroscience is still just the temporal correlation between hypotheticalcomputational variables and neural/brain activities. We can never provecause and effect relationships, no matter how high the correlation coefficientis, how much of the behavioral data the model explains, or how deeply thetheory goes into the core of cognition.
Consequently, among the four key factors necessary for ultimate systems
neuroscience—(1) measurement and (2) control of neural/brain activity, (3)manipulation of information, and (4) computational theories—we have madesignificant progress or are about to make major improvements on all thefactors except the manipulation of information. The manipulation of
information is theoretically and technically the most difficult first because
we do not know how information is represented, second because there exist
complicated relationships between neural/brain activities and represented
information, and third because simultaneous control of neural/brain
activities from many neurons and in many areas is necessary but extremely
difficult. However, without the manipulation of information, we can neverprove cause and effect relationships at the system level.

Based on these reflections on the computational-model-based approach, wehave started to advocate and promote a new approach named “manipulative
neuroscience”, where we aim to manipulate the information even for humansubjects.

Technically,

this

is

based

on

computational

theories,abrain-network interface, and non-invasive decoding algorithms. Masa-aki
Sato and his colleagues at ATR Computational Neuroscience Laboratories(ATR-CNS) have been developing a “brain-network interface” based on ahierarchical, variational Bayesian technique to combine information fromfMRI and magnetoencephalography (Sato et al. 2004). They succeeded inestimating brain activities with spatial resolution of a few millimeters andmillisecond-level temporal resolution for various domains such as visual
perception, visual feature attention, and voluntary finger movements. In
collaboration with the Shimazu company, we aim to develop within 10 years
a portable and wireless combined EEG/NIRS (electroencephalography/near
infrared spectroscopy)-based Bayesian estimator for millimeter andmillisecond accuracy. “Brain-network interface” is a term we have created
for this project, and it is like a brain-machine interface or a brain-computerinterface. A brain-network interface non-invasively estimates brain activity
by solving the inverse problem, and it also estimates neural activities andreconstructs represented information. Accordingly, it is not a brain-machineinterface because it is non-invasive, and it is not a brain-computer interface
because it does not require extensive user training since it decodes
information. Researchers at ATR-CNS have already succeeded, for example,
in estimating the velocity of wrist movements from single-trial data without
subject training. Furthermore, Honda Research Institute of Japan incollaboration with ATR-CNS demonstrated real-time control of a robot handby decoding three motor primitives (rock-paper-scissors, as in the children’sgame) from the fMRI data of a subject’s primary motor cortex activity (Press
Release 2006). This was based on the machine learning algorithms
previously developed by Kamitani and Tong (2005, 2006) for decoding theattributes of visual stimuli from fMRI data.

Manipulative neuroscience can be achieved by combining computational
theories, brain-network interfaces, and non-invasive decoding algorithms asfollows. Human subjects execute a task. We assume that we have alreadydeveloped a computational theory that postulates that specified neuralnetworks

solve

necessarycomputational

problems

and

that

some

information is represented in a specified way by brain activities within acertain area. During task execution, this information is reconstructed by abrain-network interface and decoding algorithm in real time, and it is thenfed back to the subject in any sensory modality or even by transcranial
magnetic or electrical stimulation. Then, based on a prediction by thecomputational model, some manipulation is applied to the extracted
information. Predicted changes in brain activities and behaviors are
examined experimentally. Although we discussed a human example with
non-invasive measurements, the same idea can be applied to animals withneural recordings. In that case, information is decoded from multiple neuralactivities and then manipulated and fed back to the animal by any available
method.
Having defined manipulative neuroscience in this way, it becomes apparent
that almost all systems-neuroscience experiments try to change or at least
influence information representations in the brain by giving sensory stimuli,
assigning reward/penalty, imposing tasks, requiring movements, and so on,
and then they examine changes in behavior and cognition as well as
neural/brain activities. Computational-model-based neuroscience further
employs a computational model to reproduce behaviors and correlate its
hypothetical variable with neural/brain activity, which should somehow
represent the relevant information. The new approach also utilizes acomputational model but in prediction rather than correlation, and it
directly manipulates information rather than attempts to indirectly
influence information. If subjects’ behavior and neural/brain activity change
as predicted by a computational model when the extracted information is
manipulated while guided by the computational theory, we are in a muchbetter position to state that the model is experimentally supported and thatthe hypothetical information has a causal relationship with the alteredbehavior and neural/brain activity. This is because the information and theneural/brain activities that carry it are directly manipulated in the newparadigm, and this manipulation’s effects are compared with model
predictions. This level of understanding has been very rare in traditional
systems neuroscience or computational-model-based neuroscience. We arevery aware of the difficulty and risks of the new approach but at the sametime believe that this is probably the only persuasive way to fully
understand brain functions.

Refere nces
Albus, J.S. 1971 A theory of cerebellar function. Mathematical Biosciences. 10, 25-61.
Amari, S. 2006 Crisis of Japanese brain science (in Japanese). Kagaku 76, 433-437.
Atkeson, C.G., Hale, J., Pollick, F., Riley, M., Kotosaka, S., Schaal, S., Shibata, T.,
Tevatia, G., Vijayakumar, S., Ude, A., Kawato, M. 2000 Using humanoid robots tostudy human behavior. IEEE Intelligent Systems: Special Issue on Humanoid Robotics,
15, 46-56.
Bentivegna, D.C., Atkeson, C.G., Cheng, G. 2004a. Learning tasks from observationand practice. Robotics and Autonomous Systems, 47, 163-169.
Bentivegna, D.C., Atkeson, C.G., Ude, A., Cheng, G. 2004b Learning to act fromobservation and practice. International Journal of Humanoid Robotics, 1, 585-611.
Boyden,E.S., Zhang,F.,

Bamberg,E., Nagel,G.,

Deisseroth,

K.

2005

Millisecond-timescale, genetically targeted optical control of neural activity. Nature
Neuroscience. 8 1263-1268.
Chadderton, P., Margrie, T.W., Hausser, M. 2004 Integration of quanta in cerebellar
granule cells during sensory processing. Nature 428 856-860.
Dayan, P., Abbott, L.F. 1999 Theoretical Neuroscience, Cambridge, MIT Press
Dean, P., Porrill, J., Stone, J.V., 2002. Decorrelation control by the cerebellum achieves
oculomotor implant compensation in simulated vestibulo-ocular reflex. Proceedings of
the Royal Society of London. Series B: Biological Sciences 269, 1895-1904.
De Schutter, E. 1995 Cerebellar long-term depression might normalize excitation of
Purkinje cells: a hypothesis. Trends in Neurosciences 18, 291-295.

Doi,

T.,

Kuroda,S.,

Michikawa,T.,

Kawato,M.

2005

Inositol

1,4,5-

trisphosphate-dependent Ca2+ threshold dynamics detect spike timing in cerebellar
Purkinje cells. Journal of Neuroscience 25, 950-961.
Endo, G., Morimoto, J., Matsubara, T., Nakanishi, J., Cheng, G. 2005 Learning CPG
sensory feedback with policy gradient for biped locomotion for a full body humanoid.
The Twentieth National Conference on Artificial Intelligence (AAAI-05 Proceedings,
1237-1273, Pittsburgh, USA, July 9–13, 2005
EARTO (1996-2001) web page, http://www.kawato.jst.go.jp/
Gomi, H., Shidara M., Takemura A. et al. 1998 Temporal firing patterns of Purkinjecells in the cerebellar ventral paraflocculus during ocular following responses inmonkeys. I. Simple spikes. Journal of Neurophysiology 80, 818-831.
Hale, J.G., Pollick, F.E. 2005 ‘Sticky hands’: Learning and generalisation forcooperative physical interactions with a humanoid robot. IEEE Transactions on Systems,
Man, and Cybernetics 35, 512-521.
Haruno, M., Kawato, M. 2006a Different neural correlates of reward expectation andrewardexpectation

error

in

the

putamenandcaudatenucleusduringstimulus-action-reward association learning, Journal of Neurophysiology 95, 948-959.
Haruno, M., Kawato, M. 2006b Heterarchical reinforcement-learning model for
integration of multiple cortico-striatal loops; fMRI examination in stimulus-actionreward association learning. Neural Networks 19, 1242–1254.
Haruno, M., Kuroda, T., Doya, K., Toyama, K., Kimura, M., Samejima, K., Imamizu,
H., Kawato, M 2004 A neural correlate of reward-based behavioral learning in caudate
nucleus: a functional magnetic resonance imaging study of a stochastic decision task.
Journal of Neuroscience, 24, 1660-1665.
ICORP (2004-2009) web page, http://www.cns.atr.jp/hrcn/ICORP/project.html

Ijspeert, A.J., Nakanishi J., Schaal, S. 2002. Movement imitation with nonlinear
dynamical systems in humanoid robots. Proceedings of the IEEE International
Conference on Robotics and Automation (ICRA2002), 1398-1403, Washington, USA,May 11-15, 2002
Imamizu, H., Kuroda, T., Miyauchi, S., Yoshioka, T., Kawato, M 2003 Modularorganization of internal models of tools in the human cerebellum. Proc Natl Acad Sci
USA., 100, 5461-5466.
Imamizu, H., Miyauchi, S., Tamada, T., Sasaki, Y., Takino, R., Puetz, B., Yoshioka, T.,
Kawato, M. 2000 Human cerebellar activity reflecting an acquired internal model of a
new tool. Nature, 403, 192-195.
Ito, M. 1970 Neurophysiological basis of the cerebellar motor control system.
International Journal of Neurology 7, 162-176.
Jordan, M.I., Rumelhart, D.E. 1992 Forward models: supervised learning with a distalteacher. Cognitive Science, 16, 307-354.
Kamitani, Y., Tong, F. 2005 Decoding the visual and subjective contents of the humanbrain. Nature Neuroscience 8 679-685.
Kamitani, Y., Tong, F. 2006 Decoding seen and attended motion directions from
activity in the human visual cortex. Current Biology 16, 1096-1102.
Kawano, K. 1999 Ocular tracking: behavior and neurophysiology. Current Opinion in
Neurobiology 9, 467-73.
Kawato, M. 1990 Feedback-error-learning neural network for supervised motor learning.
(ed. R. Eckmiller) Advanced Neural Computers. pp.365-372.

North-Holland: Elsevier

Kawato, M. 1999 Internal models for motor control and trajectory planning. Current

Opinion in Neurobiology, 9, 718-727.
Kawato, M., Furukawa, K., Suzuki, R. 1987 A hierarchical neural-network model forcontrol and learning of voluntary movement. Biological Cybernetics 57, 169-185
Kawato, M., Gomi, H. 1992 A computational model of four regions of the cerebellum
based on feedback-error-learning. Biological Cybernetics 68, 95-103.
Kawato, M., Kuroda, T., Imamizu, H., Nakano, E., Miyauchi, S., Yoshioka, T. 2003
Internal forward models in the cerebellum: fMRI study on grip force and load force
coupling. Progress in Brain Research, 142, 171-188.
Kobayashi, Y., Kawano, K., Takemura, A. et al. (1998) Temporal firing patterns of
Purkinje cells in the cerebellar ventral paraflocculus during ocular following responses
in Monkeys. II. complex spikes. Journal of Neurophysiology 80, 832-48.
Kuroda, S., Schweighofer, N., Kawato, M. 2001 Exploration of signal transduction
pathways in cerebellar long-term depression by kinetic simulation. Journal of
Neuroscience 21, 5693-5702.
Llinas, R., Lang, E.J., Welsh, J.P. 1997 The cerebellum, LTD, and memory: alternative
views. Learning & Memory 3, 445-455.
Marr, D. 1969 A theory of cerebellar cortex. Journal of Physiology (London) 202,437-470.
Matsubara, T., Morimoto, J., Nakanishi, J., Sato, M., Doya, K. 2006 Learning
CPG-based biped locomotion with a policy gradient method. Robotics and Autonomous
Systems, 54, 911-920.
Miyamoto, H., Kawato, M., Setoyama, T., Suzuki, R. 1988 Feedback-error-learningneural network for trajectory control of a robotic manipulator. Neural Networks 1,
251-265.

Miyamoto, H., Schaal, S., Gandolfo, F., Gomi, H., Koike, Y., Osu, R., Nakano, E.,
Wada, Y., Kawato, M. 1996 A Kendama learning robot based on dynamic optimization
theory. Neural Networks 9, 1281-1302.
Morimoto, J., Endo, G., Nakanishi, J., Hyon, S., Cheng, G., Bentivegna, D.C., Atkeson,
C.G. 2006 Modulation of simple sinusoidal patterns by a coupled oscillator model for
biped walking. IEEE International Conference on Robotics and Automation(ICRA2006) Proceedings, 1579-1584, Orlando, USA, May 15-19, 2006.
Nakanishi, J., Morimoto, J., Endo, G., Cheng, G., Schaal, S., Kawato, M. 2004 Learningfrom demonstration and adaptation of biped locomotion. Robotics and Autonomous
Systems, 47, 79-91.
Nakanishi, J., Schaal, S. 2004 Feedback error learning and nonlinear learning adaptive
control. Neural Networks, 17, 1453-1465.
Normile, D. 1997 Japanese neuroscience: new institute seen as brains behind big bootsin spending. Science 275, 1562-1563. (DOI 10.1126/science.275.5306.1562)
Ogasawara, H., Doi, T., Doya, K., Kawato, M 2007 Nitric Oxide Regulates Input
Specificity of Long-Term Depression and Context Dependence of Cerebellar Learning.
PLoS Computational Biology 3, e179.
Okubo, Y., Kakizawa, S., Hirose, K., Iino, M. 2004 Cross Talk between
Metabotropic and Ionotropic Glutamate Receptor-Mediated Signaling in
Parallel Fiber-Induced Inositol 1,4,5-Trisphosphate Production in Cerebellar
Purkinje Cells. The Journal of Neuroscience, 24 9513-9520.
Pasalar, S., Roitman, A.V., Durfee, W.K., Ebner, T.J. 2006 Force field effects on
cerebellar Purkinje cell discharge with implications for internal models. Nature
Neuroscience 9, 1404-1411.

Pollard, N.S., Hodgins, J.K., Riley, M.J., Atkeson, C.G. 2002 Adapting human motion
for the control of a humanoid robot. Proceedings of the IEEE International Conferenceon Robotics and Automation (ICRA2002), 1390-1397, Washington, USA, May 11-15,
2002
Press Release 2006 (http://www.atr.jp/html/topics/press_060526_e.html
Riley, M., Ude A., Atkeson, C.G. Methods for motion generation andinteraction with a humanoid robot: case studies of dancing and catching,
Proceeding of 2000 Workshop on Interactive Robotics and Entertainment(WIRE-2000), 35-42, Pittsburgh, USA, April 30 – May 1, 2000
Samejima, K., Ueda, Y., Doya, K. et al. 2005 Representation of action-specific reward
values in the striatum. Science 310, 1337-1340.
Sato, M., Yoshioka, T., Kajiwara, S., Toyama, K., Goda, N., Doya, K., Kawato, M.
2004 Hierarchical Bayesian estimation for MEG inverse problem. NeuroImage, 23,806-826.
Schaal, S. 1999 Is imitation learning the route to humanoid robots? Trends in Cognitive
Science 3, 233-242.
Schaal, S., Atkeson, C.G. 1998. Constructive incremental learning from only local
information, Neural Computation, 10, 2047-2084.
Schaal, S., Peters, J., Nakanishi, J., Ijspeert, A. 2003 Control, planning, learning andimitation with dynamic movement primitives. IEEE/RSJ International Conference onIntelligent Robots and Systems (IROS2003) Workshop on Bilateral Paradigms of
Human and Humanoid. ), 39-58, Las Vegas, USA, October 27-31, 2003
Shibata, T., Schaal, S. 2001 Biomimetic gaze stabilization based on feedback-error
learning with nonparametric regression networks, Neural Networks, 14, 201-216.

Shibata, T., Tabata, H., Schaal, S., Kawato, M. 2005 A model of smooth pursuit basedon learning of the target dynamics using only retinal signals, Neural Networks, 18,
213-225
Shidara, M., Kawano, K., Gomi, H., Kawato, M. 1993 Inverse-dynamics model eye
movement control by Purkinje cells in the cerebellum. Nature 365, 50-52.
Takemura, A., Inoue, Y., Gomi, H., Kawato, M., Kawano, K. 2001. Change in neuronalfiring patterns in the process of motor command generation for the ocular following
response. Journal of Neurophysiology 86, 1750-1763.
Tanaka, K., Khiroug, L., Santamaria, F., Doi, T., Ogasawara, H., Ellis-Davies, G.,
Kawato, M., Augustine, G.J. 2007 Ca2+ requirements for cerebellar long-term synaptic
depression: role for a postsynaptic leaky integrator. Neuron, to appear.
Tanaka, S.C., Doya, K., Okada, G., Ueda, K., Okamoto, Y., Yamawaki, S., Doya, K.
2004 Prediction of immediate and future rewards differentially recruits cortico-basal
ganglia loops. Nature Neuroscience 7, 887-893.
Tsukahara, N., Kawato, M. 1982 Dynamic and plastic properties of the brain stemneuronal networks as the possible neuronal basis of learning and memory. (eds. S.
Amari, M.A. Arbib) Lecture Notes in Biomathematics 45. Competition and Cooperationin Neural Nets., pp. 430-441. New York: Springer-Verlag.
Ude, A., Atkeson, C.G. 2003. Online tracking and mimicking of human movements by
a humanoid robot. Journal of Advanced Robotics, 17, 165-178.
Ude, A., Atkeson, C.G., Riley, M. 2004 Programming full-body movements forhumanoid robots by observation. Robotics and Autonomous Systems, 47, 93-108.
Winkelman, B., Frens, M. 2006 Motor coding in floccular climbing fibers. J
Neurophysiol. 95, 2342-2351.

Yamamoto, K., Kawato, M., Kotosaka, S., Kitazawa, S. 2007 Encoding of movement
dynamics by Purkinje cell simple spike activity during fast arm movements underresistive and assistive force fields. Journal of Neurophysiology. In press.
Yamamoto, K., Kobayashi, Y., Takemura, A., Kawano, K., Kawato, M., 2002.
Computational studies on acquisition and adaptation of ocular following responses
based on cerebellar synaptic plasticity. Journal of Neurophysiology 87, 1554-1571.

Figure Le gen ds
Figure 1
Demonstrations of 14 different tasks by the ERATO humanoid robot DB
Figure 2
New humanoid robot called CB (Computational Brain)
Figure 3
Biologically motivated biped locomotion of three humanoid robots(a) DB-chan with learning from demonstration and a nonlinear dynamicalsystem for motor primitives (Nakanishi et al. 2004)(b) HOAP2 with central pattern generators and reinforcement learningalgorithms (Matsubara et al. 2006)(c) CB with coupled oscillators that are phase regulated by reaction forcesfrom the floor (Morimoto et al. 2006)
Figure 4(a) General feedback-error-learning model.
(b) Cerebellar feedback-error-learning model (Kawato 1999). The ‘controlled
object’ is a physical entity that needs to be controlled by the central nervous
system, such as the eyes, hands, legs or torso.
Figure 5
Signal transduction model of cerebellar LTD(AA: arachidonic acid; CF: climbing fibers; ER: endoplasmic reticulum; Glu:glutamate; Gq: G-proteins; IP3: inositol 1,4,5-triphosphate; IP3Rs: IP3
receptors;

MAP:

mitogen-activatedprotein;

MEK:

MAP/extracellularsignal-regulated kinase; mGluRs: metabotropic glutamate receptors; NO:nitricoxide;

PF:

parallelfibers;

PKC:

proteinkinaseC;

PKG:

cGMP-dependent protein kinase; PP2A: protein phosphatase 2A; PLA2:
cytosolic phosopholipase A2; PLC " : phospholipase C " ; sGC: soluble
guanylyl cyclase; VGCCs: voltage-gated calcium channels)

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5