Bayesian mechanics ofself-organising systems

Takuya Isomura
B rai n Intel l i gence T heory U ni t, R IK E N C enter for B rai n S ci ence
「 学習 物理学の 創成」 領域セミ ナー＠上智 ⼤学, 4 Ju l y 2 0 2 4

1/40

磯村 拓哉
理化学研究所 脳神経科学研究センター 脳型知能理論研究ユニット ユニットリーダー
⽂科省科研費 学術変⾰領域（A）「予測と⾏動の統⼀理論の開拓と検証（2023〜
2027）」領域代表
takuya.isomura@riken.jp@TakuyaIsomura
https://cbs.riken.jp/jp/faculty/t.isomura/

研究の動機
専攻︓理論神経科学
研究テーマ︓脳の知能が持つ普遍的な特性を数学を使って表現すること• ⽣物のような⼈⼯知能を作りたい• そのために⽣物の知能を理解したい問い• ⽣物の知能の本質的原理は︖• ⽣物の脳が機械より優れている点は︖
2/40

What is intelligence?

<<<

Unified theory of the (brain) intelligence

3/40

Electric circuit

NeuronEnergy/potential

Neuralactivity
Neural activity
Time

4/40

Algorithm
Finite statemachineLeft/right mover

Read/write head∙∙∙0

Sufficiently long tape

1

0

1

1

1

0

∙∙∙

Cells with finite alphabet
Turing machine

5/40

Bayesian inference
Free energy
Expectation
Expectation! input|state ! state! state|input =! input

6/40

The free-energy principle (FEP)• Perception, learning, and action of all biological organisms
are determined to minimise variational free energy.
• By doing so, they perform variational Bayesian inference of
external milieu states.
Friston, Nat Rev Neurosci, 2010

7/40

Observations o# = E! − ln ) *, ,, ! + ln . ,, !

Parameters !

The free-energy principle (FEP)
Generative modelFree energy F• Perception, learning, and action of all biological organisms
Hidden states s
Posterior belief s
are determined to minimise variational free energy.

s, "• By doing so, they perform variational Bayesian inference of
external milieu states.
Friston, Nat Rev Neurosci, 2010

8/40

Observations o# = E! − ln ) *, ,, ! + ln . ,, !
Parameters !

Generative modelFree energy Fs, "Hidden states s

Posterior belief s

Policy /Preference prior C

Action 0
Expected free energy G/

9/40

Neural activity equation

Bayesian inference

IntegralMinimisation

Helmholtz energy 1

Variational free energy ℱ
Naturallyequivalent

Isomura & Friston, Neural Comput, 2020; Isomura et al., Commun Biol, 2022

10/40

Variational trinity
Sensory inputs

External states

Bayesian inference

ActionSensory inputs

External states Neural net dynamics

Action11/40

• Introduction• Bayesian mechanics of self-organising systems• Experimental validation of Bayesian mechanics in neurobiology• Evolutionary emergence of universal machines

12/40

Bobby McFerrin Demonstrates the Power of the Pentatonic Scale, YouTube
https://www.youtube.com/watch?v=ne6tB2KiZuk

13/40

⽣成モデル
(

)

$

st3
!"

#"

ot1

st1

θ
st2

ot2

θ
ot3

θ
st1

st3

%"&'

θ
st

st2

%"&(

θ

仮説

%"&

%")'

!"! = $ "! , & + noise/!" = !0! − $ 0! , 2-! = . "! , & + noise/!# = -! − . 0! , 2

⽣成モデル!3 "4 , $4 , % = !3 "4 |$4 , % !3 $4 |% !3 %⼊⼒ ot, 隠れ状態 st, パラメータ θ の分布, mはモデル構造
Friston, PLoS Comput Biol, 2008

15/40

⾃由エネルギー原理
サプライズ︓⼊⼒ ot の主観的珍しさ・起こりにくさ− log !3 "4(近似)事後分布︓ot を知った時の st と θ に関する主観的予想
7 ,"

7 !

9"#
8"

9$
"

7 ," , ! = 7 ," 7 !

変分⾃由エネルギー︓⽣成モデルの状態の主観的起こりにくさ。F ≥ サプライズ( ≔ − log !3 "4 + /56 0 $4 , % ||!3 $4 , %|"4
= E! "$ ! # [ − log 2$ 3% , 5% , 6 ] − E! "$ ! # [ − log 8 5% − log 8 6 ]予測誤差
Friston, PLoS Comput Biol, 2008

要素間の独⽴性17/40

Synchronization of two metronomes, Filmed at Ikeguchi Laboratory
https://www.youtube.com/watch?v=feEBzjqishQ

18/40

Bayesian mechanics of self-organising systems
Consider a Langevin equation (x is fast variables, o is sensory inputs, 3 is white Gaussian noise):aaTime change of xa2̇ = 3 2, " − 4: 5 2, " + 6

Sensory
Sensory
inputsSensory
inputs""inputs "Sensory
Sensory
inputsSensory
inputs""inputs "Solenoidal flow%
External
ExternalstatesExternalstates!% !
statesInternal
Internal!! states
Internalstates!
!
states !
!

Using autonomous states 4 = 5 % , 5 &!
!

defined as < :, -6 = ∫'()*!

Gradient flow

Noise

External
ExternalstatesExternalstates$ Bayesian$states
Bayesian$ inference
Bayesianinference% inference
$ % $
% $

, the ‘path’ of 4, 46 = 4 0 ≤ 9 & ≤ 9 , and internal states : = 4,
6 ; , the Hamiltonian is
5 & − $ 5, - ) + > 5, - ?9 & + @. With an operator + = - . , the dynamics are 4̇ = !4 − B/ <̇ + 3--

Naturally
Naturally
Naturally%equivalentequivalentequivalent%With coarse-grained path vector 46 = 4 0 % , 4 ?9 % , … , 4 9ActionAction# of Action#path #
Imaginary time change

and imaginary time derivative operator B0 = B! − !,#Action# D is#imaginary time and B is D’s derivative
4; 87 = −4=< 9 + 6:ActionAction
0

Gradient descent Noisebb Trajectory
TrajectoryofTrajectory! of !
of !

bPath Pathof ! of Path!
of !
! " ! "

Time Timeevolution
evolutionTimeof evolution
theofpath
the pathof the path! "State State
spacespace!State!space !
Real time
Real "timeReal" time "Viewpoint
Viewpoint
movingViewpoint
movingwith !movingwith" ! "with ! "###! 0 ! 0! 0

Isomura, arXiv, 2023

ImaginaryImaginarytime
Imaginary$time $ time $19/40

Bayesian mechanics of self-organising systems
Environment$ = !,& !(! , ) where !& = ! 0 ≤ , ! ≤ ,System"!

)#The path’s dynamics follow an imaginary time gradient descent
F' ? = −F& A + G)

Sensory input #This follows the Fokker-Planck equation that minimises the Helmholtz energy
11 > ? , *@ = A ?, *@ + log > ?
C% &Variational free energy ℱ 7 H , *@ = − log I( H, *@ + log 7 H) *Remarkably, 1 is naturally equivalent to ℱ, where ? encodes H (for C = 1)

System’s "System’s "Matching of eigenfrequency

Environment’s "!

Environment’s "!

Minimising ℱ yields the posterior distribution (Bayes’ theorem)
I( H, *@
7 H =
I( *@
Isomura, arXiv, 2023

20/40

Hamiltonian matching within variational quantum eigensolvers

Isomura, arXiv, 2023

21/40

Abstraction
degree
The free-energy principle/ Bayesian mechanics
K. J. Friston

H. von Helmholtz???
K̇ = LMNO K, *Rate coding model

Hebbian plasticity

Hodgkin-Huxley eq.

Spike-timing
dependent plasticity

Ṗ = IQR × I*,T

D. O. Hebb

A. L. Hodgkin & A. F. Huxley

22/40

The dynamics of neural network states

When the posterior is characterised by

9 ≔ ;&:% , <&:% , =, > follow a gradient@ ≔ A&:% , B&:% , C, D , variational freedescent on Helmholtz energy ?

energy ℱ is a function of @ and 3&:%

ℱ "!:# , &! "!:# , $4=
4; ;U ∝ −4;UNaturallyEquivalent

1

?+

4ℱ
4; >U ∝ −
4>U

ℱ

V+

Ø For any neural network that minimises 1, there is a generative model that satisfies ℱ ≡ 1.
Ø This implies that the recapitulation of external dynamics is an inherent feature of neural systems.

Isomura & Friston, Neural Comput, 2020; Isomura et al., Commun Biol, 2022

23/40

C. Winnerless competition principle
1. Stimulus-dependent competition
mical principles
neuroscience WLC networks
2. in
Self-organized
3. Stable heteroclinic sequence
4. Relation to experiments
2
D. Sequence learning
E. Sequences in complex systems with random
4
connections
4
F. Coordination of sequential activity
4
4 V. Conclusion
7 Acknowledgments
8 Glossary
8 References

1248
1248
1249
1250
1251
1252

Canonical neural network
Rabinovich, Varona, Selverston & Abarbanel, Rev Mod Phys, 2006

1254
1256
1258
1259
1259
1260

9
0 “Will it ever happen that mathematicians will know
4of the functional parts anḋ = −.1$FIG.
F2ℎ1. E!Color− >1online"− .Illustrationneuro3 H E − >3 − .5 E − >5 + I
1 enough about the physiology of the brain, and@ E$
electrical properties of neurons. !a" The neuron receives inputs
2 physiologists enough of mathematical discovery, for effi̇ = F on Eits −F /D tree.E These inputs may or
through F
synapsesdendritic

6
7
may not leadtothegenerationofaat the spike generȧ = ℎ6 E − ℎ /D8 spikeℎEtion zone of the cell body that travels down the axon and trig6
Ḣ = transmitter
H6 E −release
H /DinE synapses of the axonal
Hodgkin & Huxley, 1952
gers chemical
9 the8tree. If there is a spike, it leads to transmitter release andReduction
I. WHAT ARE THE PRINCIPLES?
9activates the synapses of a postsynaptic neuron and the process9
is repeated. !b" Simplified electrical circuit for a membrane
A. Introduction
patch of a neuron. The nonlinear ionic conductances are volt0
Canonical neural network
2D
Hodgkin-Huxley
equationage dependent and correspond to different ion channels. This
Building dynamical models to study the neural basis
type of electrical circuit can be used,- to model isopotential
w of behavior has a long tradition !Ashby, 1960; Block,
K
̇
=
−sig
K" +
P- − P7 *" + ℎ- − ℎ7
FIG. 1. !Color online" Illustration of the functional parts and single neurons. Detailed"
models that describe
the morphology
FitzHugh-Nagumo
model- 1962; Rosenblatt,
1962;
Freeman,1972,
2000".
The
unelectrical properties of neurons. !a" The neuron receives inputs of the cells use several isopotential compartments imple- derlying idea governing neural control of behavior is thethroughCanonicalsynapses on itsneuron
dendritic model
tree. These inputs may or mented by these circuits coupleda longitudinal
resistance;
8950:6;2;5:36
6<4/8<=.>./01by2344/56
three-step
structureofnervousthat
have
may not lead to the generation of a systems
spike at the
spikegenera- these are called compartmental models. !c" A typical spiked evolved
overofbillions
years,
bethe
stateditstrig- event is of the order of 100 mV in amplitude and 1 – 2 ms intion zone
the cellof
body
thatwhichtravelscan
down
axoninandsimplest
form as transmitter
follows: Specializedneuronstransform
gers chemical
release in thesynapsesof the axonal duration, and is followed by a longer after-hyperpolarization
Connections
environmental
into ait neuralThis encoded
tree. If therestimuli
is a spike,leads tocode.
transmitter
release andperiod during which the neuron is less likely to generate aninformation
travelsalong
pathways
toand
thethebrain
activates thesynapsesof aspecificpostsynaptic
neuronprocess other spike; this is called a refractory period.
is repeated.
!b" system
Simplified
electricalof
circuitfor ofa membraneor central
nervouscomposedbillions
nerve 5! :firing intensities, -! :sensory inputs, K:synaptic weights, ℎ:firing thresholds

cient cooperation
to be possible?”
Hodgkin-Huxley
equation—Jacques Hadamard
4

25/40

Comparison between neural networks and variational Bayes

Neural network

Generative process

Neural network formation(Single-layer feed-forward)

Variational Bayes formation(POMDP)

POMDP||
Partially observable
Markov decision process

Prior belief) ," = Cat ^Environment

Likelihood) *" |," , f = Cat f,"

Agent

Learning of parameters\ ≃ *" ⊗ 8" ⊘ 1 ⊗ 8"

Inference of hidden states
8" = [ ln \ ⋅ *" + ln ^Isomura & Friston, Neural Comput, 2020

26/40

In principle, any neural network performs Bayesian inference
Equivalent

Neural activityK̇ ∝ −L K + P* + ℎBayesian inference

Integral w.r.t.
neural activity K

@

ℱ=g

"?-

Variational free energy

Minimisation w.r.t.
posterior expectationof hidden states 8"8" h ln 8" − ln \ h *" − ln ^Naturally equivalent

K A
K
.
1=j
ln
− ln %K
K-&.

Derivative w.r.t.
synaptic strengths P

-%
.
-&
.

k
*
− *
k7

lT

Helmholtz energy

Minimisation w.r.t.
parameters ofgenerative model \V( = sig K( , ;( = ℎ( − ln K
V( 15̅ = 1 − 5, K

Synaptic plasticity

LearningK̇ ∝ MNO ×M-"9 − ℎ-FO-"9Q9RS

Equivalent
Isomura & Friston, Neural Comput, 2020; Isomura et al., Commun Biol, 2022

27/40

Reverse engineering of generative models and prediction of self-organisation
Real brain (neuronal dynamics)

Reverse engineering ofgenerative model

1 Recording neuronal activity

4 Identify generative model & free energy ℱ

Identify circuit structure

2 Assign canonical neural network
IntegralArtificial brain (Bayesian)

Derivative
Natural equivalence

Estimate constants

3 Identify Helmholtz energy !

5 Derive synaptic plasticity algorithm
Time integral

6 Predict learning process

Prediction of experimental
results (diagnosis)
Isomura, Neurosci Res, 2022

28/40

• Introduction• Bayesian mechanics of self-organising systems• Experimental validation of Bayesian mechanics in neurobiology• Evolutionary emergence of universal machines

29/40

Experimental validation of the free-energy principle with in vitro neural networks

Isomura et al., PLoS Comput Biol, 2015; Isomura & Friston, Sci Rep, 2018; Isomura et al., Nat Commun, 2023

30/40

⼀般化同期＝知覚・運動・コミュニケーションの計算モデル
Response preference to hidden sources

1

Student

016
Response to s=(0,1)[spike]Student

0.4
0.4 o
1
1
nt
100
o
Environment's parameter Environment's parameter
x 2 2

0

Session #d
Set of states $"! ≡ (を最⼩化して得られる分布200Free energy

0
1
ox
t
n
tio

40
ec
n
n Session #
Co

0
Co
nn
ec

tio
n

to
x

2

0
1
ox
t
n
tio

2

2

c
ne
n
Co

Hamiltonian matching within in vitro neural networks
ErrorSet
in !of− !! "$"!
Error in !Set
Error in !% " − !& ""! "states#" −$ ""of!states
$"

30

30
12/14

40
Error [%]

Set of states $!

0.4

Error [%]

c

0

0
Co
n
0.4 necti0Error [%]

0Teacher 2

Probability p

Teacher

Response change[spike]0*

Hamiltonian matching within in vitro neural networks+"+!
s=(1,1)
Parameter before training
Parameter after training
Error in !% " − !& ""s=(1,0)Teacher 1100s=(0,1)1
1
2×105
Session #s=(0,0)

Error

b

0

Probability p

Response to s=(1,0)[spike]16

System's parameter

No preferenceaPredicting subsequent plasticity

⼀般にはコミュニケーション相⼿ごとにモデルを変える必要がある(Isomura et al., 2019)

Free energy

社会的知能・コミュニケーションには他者を推論
s1-preferring1
することが必須。互いに推論すると⼀般化同期がs2-preferring
起きる(Friston & Frith, 2015)

System's parameter

Response change[spike]Plasticity follows free energy gradient

Probability p

Probability p

The free-energy principle predicts learning in neuronal networks

0
0

100
Session #

0
0

100

0

Session #Isomura et al., Nat Commun, 2023

100
Session #31/40

Modulation of STDP by Dopamine Is Mediated by D1-Like Receptors.

Possible neuronal substrates
for
activeinference
Dopamine
receptors
are classified
into 2 groups, D1-like recep-

tors (D1/D5 receptors), which activate adenylate cyclase (AC),and D2-like receptors (D2/D3/D4 receptors), which inhibit AC(31). The Dopamine
activation of D1-likereceptors hasSTDP
a central role inmodulates

Zhang et al., PNAS, 2009
RE S EAR CH | R E P O R T S

!

1.4

pre

W
=
⏟̇

∝

()*+,-./ 0,1+-2

I
⏟

post

× 235K × 2LM

34105+-46

Neuromodulator

Neurotransmitter
Glial factor

Dopamine
Noradrenaline
Acetylcholine
Serotonin
GABA
D-serine

Ctrl

expressing Cre specific to dopaminergic neurons(Fig. 1A and fig. S1). The direct pathway–constituting
MSNs, which mainly express dopamine 1 recep1.2
tors (D1Rs) (13), were labeled by an AAV vector
with a specific promoter for D1R-MSNs (Fig. 1A

STDP Ratio

two-photon uncaging. For optogenetic stimulation of dopaminergic fibers, a Cre-dependent
adeno-associated virus (AAV) vector expressing
channelrhodopsin-2 (ChR2) was injected into
the ventral tegmental area (VTA) of DAT-Cre miceand fig. S1). In acute coronalDA
slices, including
the nucleus accumbens(NAc)core, whole-cell**recordings were obtained from the identified D1R**
MSNs. Dendritic
spines were visualized by meansof two-photon microscopy (980 nm) detecting

1.0

0.8

-100

-50
0
50
Spike-Timing Interval (ms)100

Fig. 3. Summary of STDP induction in the presence of dopamine showing
altered ST window. Each data point is the averaged STDP ratio (mean $ SEM)
for all experiments with a given !t. **, significant difference (P & 0.01,
Student’s t test) in STDP induction between DA and corresponding control
Yagishita et al., Science, 2014
experiments.

Delayed modulationn " 11, dopamn " 13, dopam
pairs. (G) Cum
spike pairs. (H)ments with dif(P & 0.05 for * a
DA and corresp
were replottedfacilitating L
such as theprefrontal c
receptors areadded the Dsion solutio
SCH23390 bwith a longe
pairs, 1.02 $only; Fig. 5 A
10 spike pai
dopamine ot-LTD (!t "t-LTP by docontrol, P &the possibilit

13030 ! www.pnas.org"cgi"doi"10.1073"pnas.0900546106

Fig. 1. A temporal profile of dopamine actions on spine enlargement. (A)Injection of AAV vectors for ChR2 and the D1R-MSN marker (PPTA-mCherry)in 3-week-old DAT-Cre mice. (B) Selective stimulation of dopaminergic anddopamine [(G), 58 spines, 14 dendrites]. (H) Amplitudes of spine enlargements with or without dopamine. **P = 0.0041 by Mann-Whitney U test. (I)
STDP with repetitive activation of dopaminergic fibers containing ChR2 (blue32/40

Canonical neural networks perform active inference
Neural activity!̇ , ∝ −sig 8% ! ,+ .% − .& # , + A% − A& ! , − Δ, + ℎ% − ℎ&6:4,;-37 34;*-6+1,9 7*..14-=̇ , ∝ −sig 8% = ,Sensory inputs+ 0% − 0& ! , − Δ, + @% − @&6:4,;-37 34;*-6+1,9 7*..14-

Sensation-5.165(+)%-5.165(+)

Middle layer

'

# "
&

Synaptic plasticity0̇ ∝ 1 − 25 ,! "Output layerΓ "Modulator$ "6#", , 678 , / − 6#", , 1/ ⊙ sig 0'()*+,-(.

01223,4

Decision5('1(6-,-37

Variational free energy
11! = # $% ⋅ ln $% − ln * ⋅ +% − ln ,- ⋅ $%./ − ln 0 + # 4% ⋅ ln 4% − 1 − 2Γ1,% ln 9 - ⋅ $%./ − ln : + ; ln <
%2/

1

==>
D

? @
? @

%2/

A

ln

? @
? @

− ln

C/
B

C/
B

CD
B

CD
B

+ @
+ @

− ln

C/
E

C/
E

CD
E

CD
E

? @ − Δ<? @ − Δ<

−

G/
GD

1

H@ + >
D

I @
I @

A

ln

I @
I @

− 1 − 2Γ <, @

ln

JK/

JK/

JKD

JKD

? @ − Δ<? @ − Δ<−L/LD

H@ + ; 1

Neural network cost function

Isomura et al., Commun Biol, 2022

33/40

Sensation

Sensory inputs

Trajectory of the agent in a maze

! "%Middle layer

'

# "
&

Output layerΓ "Modulator$ "Decision

Isomura et al., Commun Biol, 2022

34/40

• Introduction• Bayesian mechanics of self-organising systems• Experimental validation of Bayesian mechanics in neurobiology• Evolutionary emergence of universal machines

37/40

Summary•

The dynamics of self-organising systems that minimise a shared Helmholtz energy can be castas minimisation of variational free energy. Therefore, Bayesian mechanics is an apt
explanation for these systems, including neurobiological systems.

•

Bayesian mechanics can be applied to both classical and quantum systems and offers
foundational predictions such as Hamiltonian matching.

•

Our reverse engineering technique enables the systematic identification of generative models
from various empirical data.

•

This can predict the self-organisation of in vitro neural networks. This approach can in principle
be applied to in vivo brain activity data.

•

Bayesian mechanics implies that universal machines can emerge through natural selection.
40/40