122016.1.14  doya@oist.jp

99RTf

OTfECGfS

GfETOT

9Tmer ofand(seetheant’ionndm),on)peaks before levelling off. F1 (monkey identity) and F4 (humanexpression) were encoded at a much lower level than the others. The
response latency of this neuron was 53 ms, and the latencies for theinformation about each category were as follows: G, 45; F1, 157; F2,93; F3, 125; and F4, 125 ms. Thus, the earliest transient discharge
conveyed global information, and the later sustained discharge
14STRIATE CORTEX
579
IN CAT
RECEPTIVE FIELDS.
encoded one category of the fine information
best
found by changing the size, shape and orientation of the stimulus until a clear
response was evoked. Often when a region with excitatory or inhibitory
Of the 86 face-responsive neurons,neurons(13%)
areas in thedid
receptive not
the neighbouring opposing
responses was11
establishedfield could only be demonstrated indirectly. Such an indirect method isindicated by using acodedshortare (50%)
in Fig.category,
3B, where two flanking areas
encode significant information inillustrated
any43
slit in various positions like the hand of a clock, always including the very
either global or fine category information, and 32 (37%) coded both! andcategories. To clarify how single neurons+7
encoded both globalAB

mm

-I

-m_

CD(Hubel & Wiesel 1959)
cA

B

1

aS~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~THIPPOCAMPALPLACEn(Rosenblatt 1962)

TT87

UNITS

Fig. 3. Same unit as in Fig. 2. A, responses to shinling a rectangular light spot, 1° x 8° ; centre ofslit superimposed on centre of receptive field; successive stimuli rotated clockwise, as shownto left of figure. B, responses to a 1° x 5° slit oriented in various directions, with one end
WALLalways covering
the centre ofthe receptive field: note that this central region evoked responses
when stimulated alone (Fig. 2 a). Stimulus and background intensities as in Fig. 1; stimulus(Blakemore & Cooper 1970)(Amari 1967)

duration 1 sec.

centre of the field. The findings thus agree qualitatively with those obtainedwith a small spot (Fig. 2 a).
Receptive fields having a central area and opposing flanks represented acommon pattern, but several variations were seen. Some fields had long narrow
central regions with extensive flanking areas (Figs. 1-3): others had a large
central area and concentrated slit-shaped flanks (Figs. 6, 9, 10). In many
fields the two flanking regions were asymmetrical, differing in size and shape;in these a given spot gave unequal responses in symmetrically corresponding
372lPHYSIO.20,CXL,VIIT
Downloaded from J Physiol (jp.physoc.org) by guest on October
2012(O’Keefe 1976)

RACK-213-4-j(Bruce, Desimone, Gross 1981)
FIG.

2. Placefieldsforall placeunits

except21342

andthosefromanimal217.

distributed around the maze. The concentrationof fields from the otheranimals in arm B may have reflected the fact that many of the rats spent
their “free time” in this arm. The fact that the initial search for units
was conducted there might also have introduced
a bias towards
units
active in that area. In any case, it was clear that the majority of fields
were not located in those places which contained the rewards
or other3(Sugase et al. 1999)

dFIG.

AB

3. Placefieldsfor placeunits

fromC

animal217.

Da(Fukushima 1980)

ConvNet (Krizhevsky, Sutskever, Hinton, 2012)
GoogleBrain (2012)

f I
9f T
9f

G

G9T

G
99

ras

M f

G

M fG

GG    Morimoto & Doya, 2000)

()

Os

“”

OGT
Population

Virtual agents
15-25

Robots

Genes
Weights for top layer NN

w1, w2, …, wn
Weights shaping rewards

v1, v2, …, vn
Meta-parametersαγλτkτ 0( 5)1Tsaverage fitness in the final 20 generations in all experiments. The learning performance wasestimated as the number of time steps the mating behavior was selected divided with number of mating
events. The seven types of markers indicate the number of energy sources in the environment for eachsimulation.

Foragers and Trackers
15Forager12%(a)Percent of population

weight forface visibility
w5

2.8

Tracker(b)

Roamers
Foragers
Stayers

Trackers

Figure 3. Example trajectories of the learned behaviors for the roamer strategy and thestayer strategy. (a) The roamer ignores8% the tail-lamp of the mating partner and executes the learned
foraging behavior to capture the energy source. (b) The stayer executes the learned waiting behavior
and adjusts its position according to the trajectory of the mating partner.

2.4

2Foragers
Roamers
Trackers
Stayers

1.6
−0.5

0w1

0.5

bias for mating(a)

100%

1

4%

0%
00.2

0.4

0.6

0.8

Ēmenergy level
for mating(b)Roamers1

: V(s) = E[ r(t) + γr(t+1) + γ2r(t+2)…| s(t)=s]
Q(s,a) = E[ r(t) + γr(t+1) + γ2r(t+2)…| s(t)=s, a(t)=a]

これらのステップを実現する回路機構は?
greedy: a = argmax Q(s,a)
Boltzmann: P(a|s) ∝ exp[ β Q(s,a)]: TD
δ(t) = r(t) + γV(s(t+1)) - V(s(t))
これらのパラメタの制御機構は?
ΔV(s(t)) = α δ(t)ΔQ(s(t),a(t)) = α δ(t)   
δ(t) = r(t) + γV(s(t+1)) – V(s(t))



 r V r V  r V(Schultz et al. 1997)

(Doya 2007)

V(s)�
δ�

Q(s,a)�

(Ito & Doya, 2015 JNS)
0.5-1spokingLeft

Center

1-2s
CenterRight

Cue tone

Left
Right
Rwd tone

PelletNo-rwdCue toneエサ出口(L, R)

Left tone(900Hz)(50%,0%)

Right tone
(6500Hz)(0%, 50%)

Free-choice tone
(White noise)	
(90%, 50%)
(50%, 90%)
(50%, 10%)
(10%, 50%)

(DLS)
CRC

R

(DMS)

(VS)

cue

Q

phase 1 2/

L/R

3 4 56 7higher QL
lower QL

ActionRewardQL
QR(Ito & Doya, 2015)
firing(Hz)

QL
trialshigher QL
lower QL

V

higher QR
lower QR
ActionRewardQL
QR

firing(Hz)

QLQRtrials 

Ito and Doya • Striatal Representation in Action Selection

J. Neurosci., February 25, 2015 • 35(8):3499

3 %*' (DLS) – $!. &2$4
3%*' (DMS) – ,01 5&  #6%*' (VS) – "/( )+##- ?

forebrain
cerebral cortex (a)neocortex
paleocortex:
archicortex:
basal nuclei (b)neostriatum:
paleostriatum:archistriatum:
diencephalon
thalamus (c)hypothalamus (d)brain stem
midbrain (e)hindbrainpons (f)cerebellum (g)medulla (h)spinal cord (i),,Doya, 1999)
大脳皮質：教師なし学習：状態表現�
出力�

入力�

大脳皮質�
大脳基底核：強化学習：報酬予測�
報酬信号�
大脳�
基底核�

視床�
黒質�

出力�

入力�

小脳�

小脳：教師あり学習：内部モデル�
目標出力�

下オリーブ核�

+�

誤差信号�
入力�

-�出力�

­−
tjj

t
a = argmaxa Q(s,a)
ta = argmaxa [r+V(f(s,a))]t : s’=f(s,a)
a = g(s)aas
Qs’aaisgsVf

Condition 1  Condition 3

ACondition 2  Condition 3

B–
Alan Fermin

ACondition 1  Condition 3

B+DCD1C312

2

3

DCondition 2  Condition 3

Akihiro Funamizud#101

100
Locomotion 35
speed0(cm/sec)-10
15
Licking(Hz)Reward0120
Soundangle(degree) -15
Sounded04

Neuro

Sound-source -50
0distance(cm)100#101
Locomotion 35
speed0(cm/sec)-10
15
Licking(Hz)Reward0120
Sound511 neurons
angle(degree) -15
002

SoundedjodParietal
speaker
V
812100 Pm
posterior parietal cortex (PPC)
speakerTime(sec)Regression
812511 neurons
Neurons encoding the diTime (sec)posterior medial 4area(PM)(Example: Parietal corteHistologySite 3:

Site 3:mm
Parietal
Site 1 -2.16
cortex
Site 2 Secondary
visualSite 3(V2)cortex-3.56 mm
ASite 1Site 2Site 3

ParietalcortexSecondaryvisual(V2)cortex
AL500 Pm L100 Pm

Layer 1
Layer 2- Neural
Layer 4
activities ofLayer 3- Neural
34
185parietal andLayer 4
activities of
0 67
067
0.06V2 cortices
inparietal andCont.
Inter.
layer 2, 3 andV2 cortices inlayer 2, 3 and Layer 505 wereLayer 5 100 Pm
5 were-0.03recorded'f/f

AP -1.66 mm

AAV injection: Layer 1
1.5 – 2.5 mm
posterior,
Layer 2
1.4 mm right ofbregmaLayer 3

AAV injection:
Continuous䠚 Intermitte
1.5 – 2.5 mm
1posterior,
1.4 mm right ofbregmaNeuron #AA

recorded

67 50 33 17•3010Dixfi

33
Goal distance (cm)106

Normalized
Normalized
activitymean activity

Normalized
Normalized
mean activity
activity
meanp(fi|x)
m: -1p(x|f1,…,fN)
Intermittent1-4
10

Normalized
mean activity

0.07

bLikelihood: P(x |s )ized
tivitylized
tivity00670∏ip(fi|x)p(x)0Normalized
likelihood

0.14

Posterior(Likelihood)

0.1
Sound0.1
zone

Bayesian
Bayesian
decoder
decoder00.07

00-1-267
33067
330-4 (cm)
PM: N = 17
Goal distance
100.07
16Posterior

067

Bayesian
decoder

00.14

0Posterior
Posterior

0 x
PPC:
43
PM: NN==1711a0.1

6Soundzone

Intermitte
3Normalizmean acti

-3
3067
33
0 Estimated-3distance
30.14
No0.1
soundzonerior

Normalized
mean activity

PPC: N = 431

Normalized
mean activityNormalimean aca

Posterior: BayesianP(s |x )307Prior: •P(s|xon
22
OIST: t(2014~2023)

成成機能マップの作成

行動解析
マーモセットに適した
行動課題の選択と課題関連
神経回路の抽出

２．革新脳の達成目標
02

２．革新脳の達成目標

Ｂ．神経回路マップ作成のための革新的技術

602① 高解像度・広領域・深部観察・高時間分解能
脳機能画像データ
神経回路構造・機能解析技術 の開発
jA. 霊長類の脳構造・機能マップの作成
02○可視化プローブの開発 602②霊長類脳における行動と対応付けされた○組織操作技術（透明化等）の開発○広範囲観察のための顕微鏡技術の開発
機能マップの作成○超高密度電極アレイの開発○ヒト脳イメージングのための新規MRI技術のo行動解析において重要な回路に特化して細胞レベルの活動を解析

行動解析

細胞活動の網羅的記録
dt

マーモセットに適した
行動課題の選択と課題関連
神経回路の抽出 1o p10
100 10jdn

脳機能画像データ
10517.3
10.4jj

1t

j(STDP)o (NEST)

n6
d 80
860
40p   (2013.8)

極限を探究する基礎科学のフロンティアで，実験・観測や「京」を用いた個別計算科学の成果にもかかわらず答えの出ていない難問に，ポスト「京」のみがなし得る新しい科学の共創と学際連携で挑み，解決を目指す。
⑪ 複数の社会経済現象の相互作用のモデル構築とその応用研究
複雑かつ急速に変化する現代社会で生じる様々な問題に政策・施策が俊敏に対応するために，交通や経済等，社会活動の個々の要素が互いに影響し合う効果を取り入れて把握・分析・予測するシステムを研究開発する。
⑫ 太陽系外惑星（第二の地球）の誕生と太陽系内惑星環境変動の解明
宇宙，地球・惑星，気象，分子科学分野の計算科学と宇宙観測・実験が連携する学際的な取組により，観測・実験と直接比較可能な大規模計算を実現し，地球型惑星の起源，太陽系環境，星間分子科学を探究する。
⑬ 思考を実現する神経回路機構の解明と人工知能への応用
革新技術による脳科学の大量のデータを融合した大規模多階層モデルを構築し，ポスト「京」での大規模シミュレーションにより思考を実現する脳の大規模神経回路を再現し，人工知能への応用をはかる。
なお，これらの重点課題及び萌芽的課題に関し，「別添１ポスト「京」で重点的に取り組むべき社会的・科学的課題の概要」に記載の内容の詳細（サブf If TCyber Rodent
9f
G9f
GLASSO, Deep learning
G9G
Deep generative network
T99
Probabilistic programming

  γγγV(t) = E[ r(t) + γr(t+1) + γ2r(t+2) + …]  γ:faγ

γ+100

1001

V = -22.9+1001-20 -20 -20-20 -20 -20

V =18.7G234 step

11100+5012-100
3 4 stepV =-25.1

234 step

2-1003 4 step+501

V = 47.3n

(Doya, 2002)oドーパミン: 報酬予測誤差 δ
アセチルコリン: 学習速度 αノルアドレナリン: 探索の絞り β�
セロトニン: 予測の時間スケール γnStefan Elfwing
ATRfMRI
jBernd Kuhn
Alan FerminjATR
Nicolas Schweighofer (USC)ATR­−

OIST Neural Computation Unit