Prospective coding in motor
learning and motor decision making
・I will succeed
・I will fail

Where to aim?
The keeper will
move leftward.

Ken Takiyama
Tokyo University of Agriculture and Technology(TUAT)

Prospective coding in motor
learning and motor decision making
2. motor decision
making・I will succeed
・I will fail

Where to aim?
1. motor learning

The keeper will
move leftward.

Ken Takiyama
Tokyo University of Agriculture and Technology(TUAT)

Prospective errors determine
motor learning- a step towards a uniﬁed model of motor learning -ARTICLE
Received 27 Oct 2014 | Accepted 21 Nov 2014 | Published 30 Jan 2015

DOI: 10.1038/ncomms6925

OPEN

Prospective errors determine motor learning
Ken Takiyama1, Masaya Hirashima2 & Daichi Nozaki3

Takiyama, Hirashima, Nozaki, Nature Comm, 2015

Diverse features of motor learning have been reported by numerous studies, but nosingle theoretical framework concurrently accounts for these features. Here, we propose aOur hypothesis

・I will succeed
・I will failThe predicted movement error, prospective error, determines
neural activity and motor command in motor learning.

Prospective error modelMulti-timescale model(Smith, 2006, PLoS Biol)①Savings、②Interference、③Spontaneous recovery⑦Transfer⑥Random learning ④Uncertainty ⑤Error modiﬁcation

No modelBayes model

Bayes model(Kording, 2004, Nature)(Wei, 2009, Jnp)

1. Introduction
2. Results - mathematics
3. Results - behavioral experiment
4. Results - ﬁtting to conventional data
5. Results - simulation
6. Conclusion
7. Prospective error in a competitive gameExperiment: reaching movements (unimanual) + Perturbation

These videos wereoﬀered by Yokoi-sensei.

Reaching movement

30° visuomotor rotation
TargetInitial position
Task:“Please move the cursor towards
target as straight as possible witha moderate movement speed.”Compatibility of simplicity in learning andcomplexity in control: Motor primitive(Thoroughman & Shadmehr, 2000, Nature).

Input =“desired movement direction θ”
Deﬁnition oftarget direction

Input① Motor primitive

Perturbation

θ=0
π/2

-π/2

θ

Ai (✓) = exp(

③ Movement error② Motor command

p
1
2

2

(✓

2

'i ) )④ Motor learning

Simplicity in learning … linear learning equation of W
Complexity in learning … linear combination of A (nonlinear function)

What determines neural activity in motor learning? …
No consensus.
1. Force (Evarts, 1964, Jnp)
2. Desired movement direction (Georgopoulos et al., 1984, JNS)
3. Desired movement speed & position (Moran et al., 2007, Jnp)
4. Actual movement (Gonzalez-Castro et al., 2011, PLoS Compt Biol)
5. Aiming movement direction (Taylor & Ivry, 2011, PLoS Compt Biol)
6. Reward (Huang et al., 2009, Neuron)
7. Uncertainty (Kording & Wolpert, 2004, Nature)
8. Visual and proprioceptive information (Brayanov et al., 2011, JNS)

Our hypothesis: predicted errors in theupcoming movement determine neural activity

・I will succeed
・I will fail

1. Introduction
2. Results - mathematics
3. Results - behavioral experiment
4. Results - ﬁtting to conventional data
5. Results - simulation
6. Conclusion
7. Prospective error in a competitive gamePrediction of conventional models:
When movement error is 0 on average (het i = 0), no motor learning is facilitated.
Learning rule:Wt+1 =

Wt + N et AT ( t ) =Current Biology
354

Wt

Vol 19 No 4

A

B

Braun et al., 2009100

y-position (cm)

error
= 30°
error
= -30°

Initial angular error (rad)

120

80
60

Movement
error (deg)

40
20

1
0.8
0.6
0.4
0.2
0

0

-60

-40

-20

0

20

x-position (cm)

40

60

D

Control group
Random group

1.2

0

10

20
30
Learning trial

40

100
80

mulative error

However, this prediction contradicts random learning, or structural learning
60(Braun, 2009, Curr Biol).
40

50

E

logical findings16,17 showing that some motor-related neurons ments. First,
To reproduce
randomlearning,
A executedshould movements.
be correlated toe.
encodethe PE rather
than thedesired orcharacteristics
⌘
⌘ the
In
the
present
study,
first,
weanalyticallyrevealthatohWt+1 i = hWt i + N het A(✓t )i = hWt i + N Cov(etWeA(✓focused))tactivities of motor primitives need to be determined based on the formed withinab① Motor planningDeﬁnition oftarget direction② Motor primitive
Input① Motor primitive

θ=0
π/2

-π/2

③ Motor command

Perturbation② Motor command③ Movement error
Perturbation

θ⑥ Update of prospective error④ Movement errorp④ Motor learning⑤ Motor learning

Figure 1 | Model schematic. (a) A schematic representation of our model. Prospective error det
these activities determines motor commands and movement error is a learning signal for thefield:
subjects
to movecursor or of
his/her
hand towards the target. (c) Visuomotor rotaNote:
A → need
beforethetheinitiation
movement.
2e → after the end of movement. A cannot
becorrelated toe. | D
NATURECOMMUNICATIONS| 6:5925

What are inputs x in motor learning ?

Error will be -30°.

Error will be 30°.

Movement error (deg)Our proposal: We predict movement error before the initiationof movement and the predicted movement error (prospective
error) aﬀects neural activity and motor learning.

Test trials

30

constant
random

25
20
15
10
5
0
10

20

Trial

30

40

50

Model

θ
θ
θ
θ

A(θ)
A(θ)

W
W
Learning

PreferredPreferredprospective
prospective
Preferred
error
error
== 00
prospective

errorwill
willbebe10
0. . .
error
will
be
10
error
error
will
be
0.

error = 10

…
…
…
…

Prospectiveerror
error
Prospective
error
Prospective
error
Prospectiveu = WA(θ)

1. Desired movement direction & prospective error.
2. Neural activities A. (e.g., A…Gaussian)
3. Motor command: u =N
i=1 Wi Ai ()

4. Modify W to minimize prediction error.

ter adaptation
5. Update prospective error. êt+1 = êt + ↵(etêt )

1. Introduction
2. Results - mathematics
3. Results - behavioral experiment
4. Results - ﬁtting to conventional data
5. Results - simulation
6. Conclusion
7. Prospective error in a competitive gameGroup1…Error changes in each trial

Test trials
30° visuomotor rotation

Error = -30°

Error = 30°

Error = -30°

Error = 30°

120

y position (mm)100Group2…Error changes in every 2 trials
Error = -30°

Error = -30°

Error = 30°

Error = 30°8060

Movement error = 30°40
20

Movement error (deg)Movement

0

Prediction
Prediction by
by
conventional
conventional models
models30
30Group1
Group1
Group2
Group2

10
10

00

20
20

30
30

40
40

50
50

-20

0

20

x position (mm)

40

60

3030

Group1
Group1
Group2
Group220
2000Group1
Group1
Group2
Group2

2020

1010

10
10

10
10

-40

(N=24)
Prediction
Experimentalresults
results(N=24)
Predictionby
byour
ourmodelmodel Experimental
30
30

20
20

-60

1010

2020

3030

Trial
Trial

4040

5050

00

1010

2020

3030

40 40

50 50

1. Introduction
2. Results - mathematics
3. Results - behavioral experiment
4. Results - ﬁtting to conventional data
5. Results - simulation
6. Conclusion
7. Prospective error in a competitive gameLow
uncertainty
不確実性低い

0

−0.4

Data
Model fit–!/4 –!/6 –!/12

0!/12 !/6!/4

Group 2
Group 3
10

Movement error at the t-th trial

300
200

High
不確実性高いGroupuncertainty
1

10

0

HistogramMotor command (°)

Force at the (t+1)-th tri

Theory1:
Bayesian(Kording,
Wolpert,2004,
Nature)
20
0.4

20

30

40

100
0

50

0

0.2

0.4

Learning spe

Trial

Figure 4 | Results of our behavioural experiment. (a) Generated force at the (t þ 1)-th trial after experiencing a movement error esmallest mean squared error. This estimate is a weighted sum of the mean of the priorchannel trials in group 1 (black dotted line, mean±s.e.m., n ¼ 12). The green solid line shows the fitting of our model (R2 ¼ 0.9950)
the sensed
feedback
position:(mean±s.e.m., n ¼ 12 for each group)and learning
curvespredicted by our model (R2 ¼ 0.8638 for group 1 (green), R2 ¼ 0.7967 fo
2
R2 ¼ 0.7968 for group 3 (blue)). Notably, the parameters were fit to datafrom only group 1, and ourjmodel
predicted the learning curve
j2sensedprior
with these parameters. Data for the adaptation to the
30! and¼# 30!
visuomotor
rotationsgroup. (c) Histogra½
1cm$ þ are2 included2in eachxsensedxestimated22
jsensedþ jprior
jsensed þ jpriorlearning speed. Vertical solid lines denote the mean values of each
distribution.

Fast learning

Slow learning

Körding and Wolpert31

2

0.4

Resulting mean squared error

Wei and Körding

Thoroughman and

0.8

Data Model fitting

R > 0.93The mean squared error
0.6
0.2 (MSE) is determined by integrating thesquared error over a
Deviations (cm)

1
0.8
0.6
0.4
0.2
0
−0.2
−0.4
−0.6
−0.8
−1

Given that we know j2prior ; we can estimate the uncertainty in the feedback j sensed
linear regression from Fig. 2a.
1
32

Deviations (cm)Deviation from target (cm)Mathematical model:

Prospective error hypothesis:
possible sensed feedbacks and actual lateral shifts
0.4＊Higher
uncertainty
causesdifficulty
in
0
ð1 ð1

σ0

σM
σL
σ∞

Data Model fitting

MSE ¼

Da

2
0.2
ð
x
2
x
Þ
pð
x
jx
Þpð
x
dxtrueestimated
true
sensedtrue
true Þdx sensed
predicting error, resulting in slower
learning.

−0.2
21 21

0

−0.4
For model 1, x estimated¼ x sensed, and this gives MSE ¼ j2sensed−0.2 :
1.4 1.6the
1.8 result
0 0.2 0.4 0.6 0.8 1 1.2 Using2
50
100 1500for4model 82 gives 0
for x–8estimated–4from above
True lateral shift (cm) 2
Trial
disturbance (cm)2MSE ¼ jsensed j2prior =ðj2sensed þ jVisual
Þ;
which
is
always
lower
than
the
MSE
for
model
1
prior
31. Solid
variance (a)of theprior
equal tovarianceof thefeedback,
MSE
for model
2 is0
Figure 5 | Model fitting to datathein crcns.org.
Data
fromisKörding
andtheWolpertlines
show thethe
fit ofour model(R2 wasHistogram

Motor command (°2011

Group 2eedback is relevant-8-4 -2 -1 0 1 2 Group43
8300
2: non-quadratic error (Kording, Wolpert, 2004, PNAS)
alTheory
way to20 integrate
Visual Disturbance (cm)
hat all the cues are200
起こりうる誤差
起こりえない誤差
Probableerror
Improbableerror from all trials plotted
FIG. 2. Group
The1 data from a typicalsubject. A: deviations
r inverse variances！？
as a function of the previous visual disturbances ()t # 1). Each dot stands for

10Group 2100
a single reachthe gray error bar shows the mean and SE (Note: it is small)Group and3

for each visual disturbance 0type. Data points are spread in the x direction for!/4
10(8) 20better
30visibility.
40
50 mean deviations
00.2
0.4 as 0.6
0.8 of the size ofB:
are plotteda function
Trial disturbances for the same typical Learning
visualsubject.speed
Each line stands for data fromtrials of different lags.
ment. (a) Generated force at the (t þ 1)-th trial after experiencing a movement error e in the force
0

t
Fast
learning
stimate
of
the
hand
Slow
learning2
ean±s.e.m., n ¼ 12). The green solid line shows the fitting of our model (R ¼ 0.9950). (b) Actual dataorningthe curves
hand predicted
position,by our model (R2 ¼ 0.8638 for group 1 (green), R2 ¼ 0.7967 for2group 2 (red) andN!x vis, ! "earameters
Fig. 2A)
were fit to data from only group 1, and our model
predicted
for groups 2 and 3x̂ hand" x visthe
S learning curves
2
Mathematicalmodel:n to the 30! and # 30! visuomotor rotations are included in each group.N!x(c) Histogram
vis, ! " #ofcbootstrapped(10)

mean values of each distribution.

Deviations (cm)

0.4 directly
ossible to
However, if we are
0.2
ion of visual error
second term
since
0
ting
Eq. 8 by recogniz2
2−0.2
tor !prop/(!prop(owledging the fact
−0.4

8 2

–8

–4

where the free parameter s characterizes the magnitude of the influence of a visual disturbance on future trials. The variable ! charac1
terizesthecan still be
32typical distance up to which a disturbance
Wei and Körding
Thoroughman and Taylor39
explainedby the assumptionof relevant visual feedback and c is the
0.8
Data Model fitting
Wei a& Kording,2008,
Jnp the form of
free parameter and also
constant that
affects
Data Model fitting
0.6
Largererror results in slower learning.
2p(relevant
!
x
,
x
)
(see
Eq.
5).
vis
prop
R = 0.8947
0.4 position is affected not only by visual error
Estimation of the hand
Prospective
error
hypothesis:
at the current trial but also
by those fromprevious
trials (Fig. 1D). To
0.2
capture this time-varying
effect, visual
disturbances
up to four
trials＊Largeerror
causesdifficultyinpredicting
0(identified as trial lag )t # 1, 2, 3, 4) before the current trial were
error,
resultingin wasslower−0.2
assessed. The choice ofchoosing
four trials
basedlearning.
on the finding
Deviations (cm)

(9)

0

4

8

disturbance (cm)urophysiol • VOL 101Visual• FEBRUARY 2009 • www.jn.org

0

50

100

150

200

250

300

Trial

(a) Data from Körding and Wolpert31. Solid lines show the fit of our model (R2 was 0.9315, 0.9448,

1. Introduction
2. Results - mathematics
3. Results - behavioral experiment
4. Results - ﬁtting to conventional data
5. Results - simulation
6. Conclusion
7. Prospective error in a competitive gamePreferred pro

Preferred
prosp
Prospect
Preferred pro

Motor com

Prospective
Motor cPreferred prospe

unchanged.
Thus, when a 30!-perturbation was re-imposedin the opposite
learning
phase. In fact,randoma longerinitiallearning0 phase
0.5
adapts to a0 30! visual rotation for 50 trials with an observationnoise, that is, et ¼ pt !Figure
xt þ
xt, where
xt is a Gaussiannoisewith a0.4
mean ofand a the9a
indicatestheresultofasimulation
in
0 −20
0
−20which−10the memory
followedbyabrief
periodofa!
30!-visuomotor
rotation
in
which
the
adaptation
to
a
novel
environment(for
relearning
phase,
the primitives
maintaining
are phase)
was associatedwithsloweradaptationintheoppositelearning
0.4
0.3
2
−20
−20
variance
of
s
.
When
s
is
large,
uncertainty
is
large
for
the
observationofthemovement
error.
(a)
Trial-by-trialchangeofxaveraged
across100
Anterogradeinterference.
Anterogradeinterferenceisaphesubjects
experienceda
30!-visuomotor
rotation(initial
learning
−40
−40
G
t
−10 Gwhich contributes to accelerating adaptation to the
0.5
0.3learning
reactivated,phase (Fig.
8e).−40 phase) and finally a series
0.2−20(oppositeoftrialsclockwisevisuomotor
rotation)interferes
with
the
−40
−60B isrotation
−60
¼
Ax
!
Be
to
the
simulated
x
shown
in
a,
where
A
is
a
forgetting
rateerror-clamp
andansimulations.
(b)
Adaptation
rate
after
fittingastate-spacemodelxphase)
followedbyabrief
periodof!
30!-visuomotor
nomenon
in
which
the
adaptation
to
a
novel
environment
(for
t
þ
1
t
t
t
0.2
0
−20
0.1
30!-perturbation relative to the initial learning
phase.
−60
4 −60 −80
Adaptation
to
−30
−80
adaptation
rate.
(c) Previously
reported adaptation
rate(reproduced
fromtheWei and
Körding(opposite
phase)
andfinallyaseriesoferror-clamp
trialsexample,clockwisevisuomotor
rotation)interferes
with
19,20
0.1 ).learning−0.5
perturbationofhave also noted that eventhetwo-state
Previous−30 studies−80−80
recovery. 10
Motormemory10 20 30 40 50
20 30
40 50is not
60 easily eliminated
10
60
20
30
40 Spontaneous
50
negative value
10 20 30 40
5060
10 developed
20 30 Trial
40 50
60model comprising
fast 30and 40slow50processes,
which was
10
60
20
Trial
Trial
once it is formed. After a sufficientamount of force-field training,
Trial
Trial
Trial
to explain the savings, cannot explain the experimental result a short exposure to the opposing force field appears
to reverse the Krakauer et al.18
×10−3 Adaptation
Weightvalue value
Activityof eachprimitive×10−3
toWeightActivityof
each
primitive
30
18
that savings still exist even after a sufficientnumberof
Adaptation
to30
Krakauer
et al.memory content).
0.9
motor
output
(that
is,
the
motor
However,−330
perturbation
of
3
×10
0.9
8080
30 30
80
1
Weightvalue
Activityof
each
primitive
35
80 phase.
washout trials following the initial learning
As shown in during theAdaptation
perturbation
0.8 0.9
1
forgetting
process30of the motor memory,
motorto ofvalue
2.5 positive
3 the
35
0.8
60
60
Initiallearning
80
80
30
Day
1
perturbationof2030 even with aInitial
30
positive
value
Fig. 7e,sufficiently
long60washout phase,
our memory0.7for
25
60
2be
learning
0.8the0.8
originallytrainedforce field cansponta30
40
40
25
20
0.7
Relearning
positive
value
60
2
60
1. This phenomenon
model1020can still accountfor the savings
effect when
the neously 0.6
20
Relearning
15
40 0.8Dayis2 called
0.7 2040
spontaneousrecovered
1.5
20 40
1
20
15
40
0.6
0.6
0.5
1,23,24
20
1
forgetting
rate
is
close
to
1.
0.6
20
20
0.6
Krakauer+, 2000,
010
recovery0.5 . 0 20
0 20
0.4 0.5
0
2
4
6
8
10
12
14
−200.40 0 the result of a Trial
0 −20 0Figuresimulation
0in0 which the
0 2
4
6
8
10 12
14 9a indicates
−10 0
0.5
0.3 0.4
0.4
0.4
Trial
Natlearning
Neurosci
−40
−40
−20
−20
Anterogradeinterference.
Anterogradeinterferenceisaphesubjects
experienceda
30!-visuomotor
rotation(initial"=0.9586
−20
−20
−1
0.2
−10
0
−1
−20
"=0.9586
0.3 0.3 −600.2
10Adaptation
to
0.2
10 in which the adaptation to a−40
bya brief period of"=0.9786
a ! 30!-visuomotor
rotation
nomenonnovel−60
environment (for
−40
−40
−40
0.1
"=0.9786phase) followed
−0.5
0
−20
perturbationof−30
0.2 0.2 −800−80"=0.9986−2−2"=0.9986(opposite learning−60−60
phase)
andfinallyaseriesoferror-clamp
trialsexample,clockwise visuomotor rotation)withthe−60 interferes−60
0Adaptation
negative
value
0
Adaptation
1 to to

Theory 3: Multiple timescale model (Smith, 2006, PLoS Biol)60

1030 15040 20050 60
10 10 5020 100
30 300250
Cycle#
NumberTrialof washout trials

−80
−80

D

D

Mean (°)

Motor command (°)

Motor command (°)

0.1
10
50 60
5 20
10 30
154020020
30
300 0.1
0
50
100
150
250 25−80
Trial
Numberwashout
Trial
10 20 30of40
50 60 trials
70 80 90 100

Preferred prospective error (°)

20

Preferredprospective
error (°)
Mean(°)
Preferredprospective
Normalizedsavingserror
effect(°)

5Preferred prospective error (°)

60
30
40
50
15
20
25
30 −80Trial
10 20 30 40
50
60
70
80
90
100
Trial90 100
10 20 30 40 50 60 70 80

−30

10

Prospective error (°)
Prospective
error
(°)
Normalizedsavings
effectPreferredprospective
Movementerror(error°) (°)

Prospective error (°)
Preferred prospective error (°)

Movement error (°)

Motor command (°)

Motor command (°)

1. Savings - A-B-A paradigm, faster learning speed in relearning phase -−3−3perturbation of
perturbation of Cycle #negative value
30 40 50 60 70 80 90 100
negative value
1010202030
40 50 60
70 80 90 100
18
Krakauer etTrialal.

10 20 30 40 50 60 70 80 90 100
Trial30(a–d) To examine our model’s ability to explain the short-term savings effect, we considered an experimen
Figureability
7 | Savings.
Trial
Trial
Trial
30 7 | Savings.
Figure(a–d) To examine our model’s
to explain
the short-term savings effect, we considered an experimentin whicha 30! visual
22
×10−3
1
Weightvalue
Activityof
each
primitive
Sing
and
Smith
35
rotation
was applied
forby30
initial learning
phase),
followedby a ! 30!
visuomotor
rotationfor
Adaptation
to 5 trials (the opposite lear
22Day
rotation was applied for 30 trials (the initial learning
phase),
followeda trials! 30!(thevisuomotorfor 5 trials(theandoppositelearning
and another1 phase),325
30
0.9 rotation
Sing
Smith
Initial learning
0.8
80
30
80
perturbationof30
25 rotationof0.8
the 30!
visualfor 30change
trials (thephase).
(a) Trial-by-trialchangeof pt andsavings.
xt averaged
across 10 simulatio
set of the 30! visual rotation for 30 trials (therelearning
phase).
(a)
Trial-by-trialofand x0.8
across10 simulationsof short-term
25sett averaged
0.8pt relearning
20
Relearning
30
positive value
60
Day 2 260
20 activity of each primitive, with a strong
15(b ) The
colourhigh activity.
reddenotesthe indicatingprospective
error,
êt. Verticaldottedare the prospective error, êt(b white) The20
activityindicatingof eachprimitive,
withThea strong
colourhighactivity.
The redlinelines
denotes
0.7 linewhite20
40 0.640
0.6
1 onprimitive.
20drawn
0.6whichdrawn10at the trials when the phases switched.
The horizontal
dottedlinethedenotestheswitched.
line onê0.6
oflineeachatthe trials
whenphases
The horizontal
dotted lineparameters
denotes thewhich êtBlue¼ 0. (c) Weighting parametet ¼ 0. (c) Weighting
20
20
15
0.4
20
0.5
0 2 weighting
46
8 of parameters
1012
14 and to0(d)Trialsand redto compensate
for
perturbations
positive
negativevalues,forrespectively.
Comparisonof xnegativered
indicate
compensate
perturbationsof positive
andt betweenvalues, respectively. (d) C
00.4colours
0 colours indicate weighting parametersand
Trial
0
0
13 Trials
0.4
the initial learning and relearning phases. (e)15
Persistenceof the savings
effectand its dependenceon
the forgetting
rate (l¼ 041
0.9586(best-fit
parameter
forforgetting rate (l ¼ 0.958
0.4
the initialand relearningof the savings
effectand its
dependence
on theTrialsTrials−20−20
10 learning"=0.9586 phases. (e) Persistence
0.2
−10
−1
0
Trial
112
Trials13
Trials0.2anofexperiment
10 of group 1), 9786 or 9986). We simulated
the data
in whicha 430!
visual0.3
rotation−40
wasfor 60(theinitiallearning
phase)
the data
or 9986).
an appliedexperimentin trials
which
aTrials30!
visualrotation
was applied for 60 trials (the"=0.9786−40 group 1), 9786
10230Trials
TrialsWe simulated
410−20
Initial learning
0.2
10
−2
369(the0.2
followed by a 0! visuomotor rotation (washout
phase),
of0 Trial
the 1230!
visualrotation
was 0imposedfor 20relearning
phase).was
The imposed for 20 trials (th5 and another set"=0.9986−60
Trials−60
followed(washout
phase),
and anothersettrials
of112
theTrials
30!
visual Adaptation
rotation
Trials0 by a 0! visuomotor rotation
to
0.1
Oppositelearning
0
50
100
1
30
230
TrialsWe define the savings−30 axis
4 Trials
100
Trials
300
0
50
100
150
200
250
horizontal
denotesthe lengthofwashout
trials.
(Inset)comparisonofx
betweenthe
initiallearning
andrelearning
phases.
−80
−80
t
−3
5 Initial
10 learning
15
20
25the 30
horizontal axis denotes the length of the washout trials. (Inset) comparisonof xTrials
initial learningand relearning phase
perturbationofCycle # 369
t between the5
0 x inNumber
12 Trials
0 10 andof fivewashout
trials10 integral
20 30 4050
70zone:80 90the100effect as0 theofTrialthe60
grey
difference
of
the
first
trialsbetweenthe
initiallearning
relearning
phases.
This
valueshouldbe0
if
negative
value
20
30
40
50
60
70
80
90
100
t
10
20
30
40
50
60
70
80
90
100
of
difference of0xt in the first
between the initial learning and relearning phases.
10
20the grey
30 zone:40 the50
learning
50five trials100
5 Opposite
10 15 20
25 30 35 40 45 effect as the integral
100 Trials
there
are
no
savings,
and
the
value
is
positivewhen
thelearning
speed
in
the
relearningphase
ishigher
than
that
in
the
initiallearning
phase.
The
savings
Trial
Trial
Trialin the relearning
CLE
NATURECOMMUNICATIONS| DOI:
10.1038/ncomms6925
Trial
ARTICLECOMMUNICATIONS| DOI:
10.1038/ncomms6925
are notosavings,
andthevalue NATURE
is savings
positivewhen
thelearning
speed
phase
is
higherin the initial lea
Trial
0thereability
Figure7|
Savings.
(a–d)
To
examineourmodel’s
explaintheshort-term
effect,weconsideredanexperiment
in
which
a
30!
visualthan that
18 (adapted22by permission from Macmillan
effects
werenormalizedbysettingthemaximal
value
to
be
1.
(f)
Previously
reportedsavings
byKrakauer
et
al.
18
10
20
30 setting
40 the50maximal value to beSing
Smith
effects
were normalized
1. (f)and
Previously
reportedsavingsbyet al. (adapted by pe
5 10 15
20
25 30
40 45
18,To
rotation
appliedfor 3530
trials(the initial
learning
phase),
followedby bya !
30! visuomotor
rotationfor simulated
5 trials(theoppositelearning
phase),
andKrakauer
another
25
Publishers
Ltd: Nature
Neuroscience
copyright
1999).
Figure8was| Anterogradeinterference.
determinewhether
our
model
could
explainthis
effect,weanexperiment
in
which
a
30!
visualrotation
0.8
Trial Neuroscience18, copyright 1999).
Publishers
Ltd: Nature
Trial rotation for 30 trials (the relearning
1
1Trial-by-trial
set
of
the
30!
visualphase).
(a)change forof p50xt averaged
acrosslearning
10 simulations
short-termsavings.
t and
Data
30 (the initial learning phase) was followed by a ! 30!
for 50 trials
visuomotor
rotation
trials(the opposite
phase).of(a)Trial-by-trialchange of
Data
Smith
et al.1
30
Smith et al.1 Mutli-rete model
30
Mutli-rete model
20
0.8
0.6
Gain-specific
model
0.8
(bp )6andThexactivityof
each
primitive,
withastrongwhite
colourindicatinghighactivity.
The
red
line
denotestheprospective
error,ê.
Verticaldottedlines
are theGain-specific
model
COMMUNICATIONS| 6:5925Single-state| DOI: model
10.1038/ncomms6925|highwww.nature.com/naturecommunications
tacross 10Tosimulations.
(bwhether)6 Activities
ofNATURE
each primitive,
with athis
strongwhite
colourindicating
activity.
The
red linedenotesSingle-statemodelexperiment
Anterogradeinterference.
determine
our
model
could
explaineffect,wesimulated
an
in
which
a
30!
visualrotation | www.naturett averaged
NATURECOMMUNICATIONS|
6:5925|
DOI:
10.1038/ncomms6925
0.6
200.6line denotes the line on which êt ¼ 0. (c) Weighting parameters of each primitive. Blue
Error clamp
drawn at the trials when20the phases
switched. The
horizontal
dottedError
clamp& 2015
Macmillan
All rights
reserved.
prospective
error, êt. phase)
Vertical was
dottedline is drawn
at 30!
thetrial
atPublishers
which rotation
theLimited.
initialfor
learning
phaseswitchesto Publishers
thelearning
phase.
The horizontal
0.4
20 learning
0opposite
Trials phase).
0.4(theinitialfollowedby
a 15!
visuomotor
50andtrials(theoppositelearning(a)Trial-by-trialchange of& 2015
Macmillan
Limited.
Allrightsreserved.
0.4
10
and redcolours
indicate10weighting
parameters
to compensate
for perturbationsof positive
negativevalues,respectively.
(d)Comparisonofx
13 Trials
t between
dotted line denotes the line on which êt ¼ 0. (c) Weighting
parametersof
each
primitive.
Blue
and
red
colours
indicateweightingparameters
tocompensate
0.2
41 Trials
0.2
eragedacross10andsimulations.
(b ) Activitiesof10eachprimitive,
with0andaTrialstrong
white colour
indicatinghigh
The redline denotes
the0.2the forgetting
the initial
relearning
phases.
(e) Persistenceof thesavings
effectits dependenceonrate(l Trials¼activity.
0.9586 (best-fit
parameterfor
0learning
112
00 respectively. (d)for perturbationsofpositiveandnegativevalues,Comparisonofx
betweenthe
initiallearning
and
oppositelearning
phases.
In
thet
10Vertical
230 Trials
4aTrialsthe datagroup 1),
9786Initialor9986).
We simulated
an experiment30!
visual rotation
was applied
trials (thelearning
initial learning
phase)
error,ê−10dottedline
is drawn
at thetrial
at which0inthewhich
initiallearning
phase switches
tofor
the60opposite
phase.
The horizontal
learning
t. of
369 Trials–0.2
5−10 the negative part of xt is drawn (red –0.2
opposite learning phase,line in a). (e)Trial-by-trial change
learning phase. Each dotted line
12 Trials
0 of xt in the opposite
followedthe
by a 0! on
visuomotor(washout
phase),
and anotherof therotationwas
imposedfor 20 100
trials(the relearning
phase).
Opposite0 colours
50indicate
denoteswhichofêxtrotation¼ 0.learning(c)
Weightingof seteach
primitive.
Blueand redweightingparameters
toThecompensate
100 30!
Trials visual–0.4 parameters−20theline–0.4phase. (f) Previously reported savings by Sing and Smiath22 (reproduced from a previous
denotesdependence−20
t on the length of the initial learning
00
horizontal
axis denotesthelength of the washout
trials. (Inset) comparisonof xofand relearning
phases. Welearning
define thephases.
savings In thet between the initial
–0.6
22
tions
negative
the learning
initial learningand opposite–0.6
10 Comparison
20
30
40 xt between
50−30
studyof).positive
5 10 and15 −3020
25 30 35values,
40 45 respectively. (d)effect as the integral of the grey zone: the difference–0.8 of xt in the first five trials between the initial learning and relearning phases. This value should be 0 if
arning phase, the
of
x
is
drawn
(red100
line in–0.8a).Trial(e)100
Trial-by-trialchangeof xt in300
the opposite
learning phase. Each dotted line
20 negative
40 Trial
60part80
100
250
300
350
100
150
200
t
20 is 4060 when80the learning
250 than150
200 is higher
there are no savings, andTrial
the value
positive
speed inTrialthenumberrelearning
phasethat in 350
the initial learning phase. The savings

2. Anterograde interference - A-B paradigm, slower learning speed in B phaseAdaptation index

Preferredprospective
Mean(°)error (°)
Adaptationindex

Motor command (°)

Prospective error (°)
Normalized savings effectPreferred prospective error (°)
Motor command (°)

Motor command (°)

MotorMovement
command error(°) (°)
Motor command (°)

Trial

Sing+, 2010,
PLoS Compt Biol

Adaptation index

Adaptation

Motor command (°)

Adaptation

Motor command (°)

Motorcommand(°) (°)
Motor
command

3. Spontaneous recovery - recovery of memory trained for a long time -Smith+, 2006,
PLoS Biol

22 (reproduced from a previous
NATURE COMMUNICATIONS
6:5925 | of
DOI:the10.1038/ncomms6925| www.nature.com/naturecommunications
7
Trial
Trial number
dependenceof x on the |length
initial learning phase.
(f) Previously reported
savings by Sing
18 and SmiathProspective error modelMulti-timescale model(Smith, 2006, PLoS Biol)①Savings、②Interference、③Spontaneous recovery⑦Transfer⑥Random learning ④Uncertainty ⑤Error modiﬁcation

No modelBayes model

Bayes model(Kording, 2004, Nature)(Wei, 2009, Jnp)

1. Introduction
2. Results - mathematics
3. Results - behavioral experiment
4. Results - ﬁtting to conventional data
5. Results - simulation
6. Conclusion
7. Prospective error in a competitive gameError will be -30°.
Prospective error model
1. anterograde intereference (Sing, 2010, PLoS CB)
2. spontaneous recovery (Smith, 2006, PLoS Biol)
3. relevance of error (Wei, 2009, Jnp)

Error will be 30°.

4. short-term savings (Krakauer, 2000, Nat Neurosci)
5. long-term savings (Zarahn, 2008, Jnp)
6. Kalman ﬁlter (Kording, 2004, Nature)
7. structural learning (Braun, 2009, Curr Biol)

1. Based on math to reproduce random learning, we propose a novel
hypothesis: the prospective error is encoded in motor planning.

2. Based on our behavioral experiment, we validated our prediction which any
conventional model never predicts - strong prediction power -.
3. Our model can explain several phenomena which were separately explainedby different computational models - a step towards a unified model of motor
learning -.

1. Introduction
2. Results - mathematics
3. Results - behavioral experiment
4. Results - ﬁtting to conventional data
5. Results - simulation
6. Conclusion
7. Prospective error in a competitive game

1) D
2)

Is prospective coding eﬀective
when there are other players?

Keiji Ota
(TUAT / NYU)

・I will succeed
・I will fail

Where you aim
in the presence of an opponent?

d risk-sensitive
processin 2017, SfN
Ota & Takiyama, in decision
preparation, Ota& Takiyama,Acknowledgement
Takiyama lab@ TUAT:

Keiji Ota, PhD
Takuji Hayshi, PhD
Daisuke Furuki
Koutaro Ishii

Collaborators:

Daichi Nozaki, PhD(Univ. Tokyo)
Taiki Komatsu, PhD(Nihon Univ.)
Masaya Hirashima, PhD (CiNet)
Mitsuaki Takemi, PhD(Univ. Tokyo)
Shota Hagio, PhD(Univ. Tokyo)
Yasushi Naruse, PhD(NiCT)
Masahiro Shinya, PhD (Hiroshima Univ.)
Takeshi Sakurada, PhD (Jichi Med. Univ.)
Shin-ichi Furuya, PhD(SONY)
Hirofumi Sekiguchi, PhD (Jobu Univ.)

Grant & fund:

JSPS, Casio, Okawa information science, Nakashima
international memorial, Kayamori information(JSPS PD、NYU)(JSPS PD)