Creating Mathematics
That Describes Brain andI f
Informationti(1) theth structure
tt
off information:i fti
surprise,i
probability,b bilitand geometry
RIKEN
Brain Science InstituteShunichi Amari

Mathematics is a culture ofhumankind
Babylonian Mathematics
Arabic, Chinese, Greek, Japanese
Mathematics
Euclidean geometry

the brain structure
the concept of number- animal, chimpanzee
geography- mouse,
geography
mouse geometry-human: abstract thinking

The Brain Structure-theth pleasureloff thithinking
ki thithings outt

abstract thinking
games puzzles
games,
mathematics, cultureTh science
Theiandd technology
t h lin the 21st century
What is human?

Physics

Mathematical science
BiologygyWhat is society?

fusionInformation science
Mathematical thinking
Human sciencePhysics-- a huge empire
Physics
Theory of relativity
Quantum mechanics

Economics

Biology

Chemistry

Condensed
matter physics

nanotechnology
electronics

Complex systems … information

mathematics

Biology

Natural history

Reductionism

Molecular biology

Diversity of species

Genes
Biotechnology

Structure and function

Fundamental principles of life

Brain/System biology

Information science/engineering
Turing and Gögel’s
gel’s Curse

Telephoneradio
Vacuumtube

Information theory
cybernetics
ComputerTransistor

Internet

Informationsociety

Artificial Intelligence ,Robots,Brain, Bioinformatics

Mathematics-- an ivory tower
Mathematics
Modern mathematics
BourbakiMathematicalsciences

physics

informatics

engineering

biology

The mathematical principle ofinformation: the amount of
information
A or B? -1 bit
A B C : k characters
A,B,C…
ｆ（ｋ） bits

The additivitymof
information}
kｘｘｘｘｘｘｘ
ｘｘｘｘｘｘｘ

ｘｘｘｘｘｘｘ
f(km)= f(k) + f(m)f(xy) = f(x) + f(y)f(k) = log kyf '( xy ) = f '( x )yff '( y ) = cf '( y ) = c / yInformation and probability
p = 1/kf(p) = -log
log p = log kxxxxxxxxxxxxnmm
I ( p ) = I ( ) = lo g n − lo g m = − lo g pn

information, probability
Information sourcex1 x2 x3 L

Information entropy

pi = ProbP b { x = i}

H = −∑ pi log
l piInformation contraction
p = Prob { x = 1}

011001110101
nentropy

H = − p logg p − ((1 − p))log(1g( − p) nH2nlog 22nHnH= nH bitsFrequently
Ftl
appear2n2nH

Law of large
g numbers
0101100110L → #1 = np;

n = 1010,000;000; p = 00.3
3p = Prob { x = 1} H = − p logg p − ((1 − p ) log(1g( − p )∑ x = np ± A
in

An → 0, c, ∞ ?
n∑ xi = np ± 2

nCnpnp ) !( n − np ) !
(=n!

⎛n⎞
n! = ⎜ ⎟⎝e⎠n

Cnp = 2nnH

2π n ≈ nn

Informationtransfer
facsimile image
facsimile,
image, audio
audio,…

information

information
compresstransferddecodedInformation transfer: speech
pathcompressSpeec
Speechpath

decodenoise: error

0X10Y
101q10q

M t l information
Mutuali fti0X 10Y
1I ( X ,Y ) = H ( X ) − H ( X | Y )= H ( X ) + H (Y ) − H ( X , Y )Information capacity: C = 1 − H(q)Shannon’s ggreat discovery:ytransfer without errors

di itnC
C digits

010 L 0n digits

010 L 0nC digitstransfer
encodedecodedi itnC
C digitsn digits

The Probability of error is “0” !

Signal and code

2n2nHH=2n (1− H )2n=2nC

2nH ( q )

N dimensional ball andN-dimensional
intersectionnnn −1(r + ε )r + nε rnε== 1+nnr
rrr

V = cn rnError correcting code：Hamming code
Computers CDs
Computers,CDs, …
⎡ x1 ⎤⎡1 0 0 1 1 0 1 ⎤ ⎢⎥⎢0 1 0 1 0 1 1⎥ ⎢ x2 ⎥ = 0⎢⎥ ⎢ M⎥⎢⎣ 0 0 1 0 1 1 1 ⎥⎦ ⎢⎥⎢⎣ x 7 ⎥⎦2 4 → 27

Galois field, algebraic geometry codes

Information geometry
Controlling
Information theorysystems
Statistical
Neural networks
science
Optimization,
Information
physics
combination
science
Artificial
maththIntelligence
Riemann manifolds- dual connection structure

Manifolds of probability distribution family

What is Information ggeometry?
yS = { p ( x; μ , σ )}2⎧
1
⎪ ( x − μ ) ⎪⎫p ( x; μ , σ ) =exp ⎨−⎬22σ2πσ⎩⎪⎭⎪{ p ( x )}σS = { p ( x;θ )}Riemann geometry
Dual affine connection

μp ( x; μ , σ )The invariance principle: S = { p ( x, θ )}1 Independent of the parameterization1.

ξ = ξ (θ ) ,p ( x, ξ )D = ∑ θi2 ≠ ∑ ξi2

2. Independent of the apparent scale ofthe probability variable
2y = y ( x),p ( y,θ )∫ p ( x,θ1 ) − p ( x,θ2 ) dx

ξθ

≠ ∫ | p ( y, θ1 ) − p ( y, θ 2 ) |2 dy

The two geometrical structure
Riemannian and affine connection
ds 2 = ∑ g ij (θ ) d θ i d θ j1
ds = D ⎡⎣ p ( x , θ ) : p ( x , θ + d θ ) ⎤⎦2
2Fisher information⎡ ∂⎤∂g ij = E ⎢lo g plo g p ⎥∂θ j⎢⎣ ∂ θ i⎥⎦

Affine connection
affine connection

covariant derivative
Πc X = Y& X& X=X(t)geodesics
測地線
ΠX
X=XX X(t)
s=∫i
jg(θ)
dθdθ
∑ ij

minimal distancestraight
g lineGeometry
E lid
EuclidBolyai Lobachevsky
Bolyai,
Gauss
Riemann
Einstein
String theory∗∇,∇
Two affine connections ()(Π , Π )*e-geodesiclog r ( x, t ) = t log p ( x ) + (1 − t ) log q ( x ) + c ( t )m-geodesicr ( x, t ) = tp ( x ) + (1 − t ) q ( t )q ( x)p ( x)

Neuronal firinggx1x2x3xn

p (x) = p ( x1 , x2 ,..., xn ), xi = 0,1
ηi = E[ xi ]---- firing rate

vij = Cov[ xi , x j ] ----correlation
What is higher-order correlations?
Are theyy orthogonallygy decomposable?
pP b bili ti iinference
Probabilistic
fp ( x, y , z , r , s )p(x, y, z | r, s)
x, y, z,... =1,−1