3. Distributed dynamical systems

3. Distributed dynamical systems

Probabilistic cellular automata provide useful toy models of a
wide range of physical and biological systems. A

cellular
automaton

consists of a

collection

of cells, each equipped with
neighbors. Two important important examples are

Example 2

(Conway’s game of life).

The cellular automaton is a grid of

deterministic

cells
with outputs

{

0

,

1

}

. A cell outputs 1 at time

t

iff:
(i) three of its neighbors outputted 1s at time

t

-

1

or
(ii) it and two neighbors outputted 1s at

t

-

1

. Remarkably,
a sufficiently large

game of life

grid can implement any
deterministic

computation

[

3

]

.

Example 3

(Hopfield networks).

These are probabilistic cellular automata

[

4

,

1

]

, again with outputs

{

0

,

1

}

. Cell

n

k

fires with

probability

proportional to

p

(

n

k

,

t

=

1

|

n

∙

,

t

-

1

)

∝

exp

[

1

T

∑

j

→

k

α

j

⁢

k

⋅

n

j

,

t

-

1

]

.

Temperature

T

controls network stochasticity.

Attractors

{

ξ

1

,

…

,

ξ

N

}

are embedded into a network by setting
the connectivity matrix as

α

j

⁢

k

=

∑

μ

=

1

N

(

2

⁢

ξ

j

μ

-

1

)

⁢

(

2

⁢

ξ

k

μ

-

1

)

.

It is useful to take a finer perspective on cellular automata
by decomposing them into spacetime coordinates or occasions

[

2

]

. An occasion

v

l

=

n

i

,

t

is a cell

n

i

at a time point

t

. Two occasions are linked

v

k

→

v

l

if there is a

connection

from

v

k

’s
cell to

v

l

’s (because they are neighbors or the same cell)
and their time coordinates are

t

-

1

and

t

respectively for
some

t

, so occasions form a directed graph. More generally:

Definition 4

.

A

distributed

dynamical system

𝐃

consists of
the following data:

𝐃

1.

Directed graph

.
A graph

G

𝐃

=

(

V

𝐃

,

E

𝐃

)

with a

finite set

of
vertices or occasions

V

𝐃

=

{

v

1

⁢

…

⁢

v

n

}

and
edges

E

𝐃

⊂

V

𝐃

×

V

𝐃

.

𝐃

2.

Alphabets

.

Each vertex

v

l

∈

V

𝐃

has finite alphabet

A

l

of outputs and finite alphabet

S

l

:=

∏

k

∈

s

⁢

r

⁢

c

⁢

(

l

)

A

k

of inputs, where

s

⁢

r

⁢

c

⁢

(

l

)

=

{

v

k

|

v

k

→

v

l

}

.

𝐃

3.

Mechanisms.

Each vertex

v

l

is equipped with

stochastic

map

𝔪

l

:

𝒱

⁢

S

l

→

𝒱

⁢

A

l

.

Figure 1:

Mapping

a cellular automaton to a distributed
dynamical system.

Taking any cellular automaton over a finite time interval

[

t

α

,

t

ω

]

initializing the mechanisms at time

t

α

with fixed values (initial conditions) or
probability distributions (noise sources) yields a distributed
dynamical system, see Fig.

1

. Each cell of the
original

automaton

corresponds to a series of occasions in
the distributed dynamical system, one per time step.

Cells with memory – i.e. whose outputs depend on their
neighbors outputs over multiple time steps – receive inputs
from occasions more than one time step in the past. If a cell’s
mechanism changes (learns) over time then different mechanisms
are assigned to the cell’s occasions at different time points.

The

sections

below investigate the compositional

structure

of
measurements: how they are built out of submeasurements.
Technology for tracking subsystems and submeasurements is
therefore necessary. We introduce two closely related

categories

:

Definition 5

.

The

category of subsystems

𝚂𝚢𝚜

𝐃

of

𝐃

is a

Boolean lattice

with objects given by sets of

ordered
pairs

of vertices

𝐂

∈

2

¯

V

𝐃

×

V

𝐃

and arrows given by

inclusions

i

12

:

𝐂

1

↪

𝐂

2

. The initial and terminal
objects are

⊥

𝐃

=

∅

and

⊤

𝐃

=

V

𝐃

×

V

𝐃

.

Remark 2

.

Subsystems are defined as ordered pairs of vertices,
rather than subgraphs of the directed graph of

𝐃

.
Pairs of occasions that are not

connected

by edges are

ineffective

; they do not contribute to the
information-processing performed by the system. We include
them in the

formalism

precisely to make their lack of
contribution explicit, see Remark 3 (

http://planetmath.org/4measurement#Thmrem3

).

Let

s

⁢

r

⁢

c

⁢

(

𝐂

)

=

{

v

k

|

(

v

k

,

v

l

)

∈

𝐂

}

and similarly for

t

⁢

r

⁢

g

⁢

(

𝐂

)

. Set the input alphabet of

𝐂

as the

product

of the output alphabets of its source occasions

S

𝐂

=

∏

s

⁢

r

⁢

c

⁢

(

𝐂

)

A

k

and similarly the output alphabet of

𝐂

as the product of the output alphabets of its target occasions

A

𝐂

=

∏

t

⁢

r

⁢

g

⁢

(

𝐂

)

A

l

.

Definition 6

.

The

category of measuring devices

𝙼𝚎𝚊𝚜

𝐃

on

𝐃

has objects

Hom

𝚂𝚝𝚘𝚌𝚑

⁡

(

𝒱

⁢

A

𝐂

,

𝒱

⁢

S

𝐂

)

for

𝐂

∈

2

¯

V

𝐃

×

V

𝐃

. For

𝐂

1

↪

𝐂

2

define arrow

r

21

:

Hom

⁡

(

𝒱

⁢

A

𝐂

2

,

𝒱

⁢

S

𝐂

2

)

→

Hom

⁡

(

𝒱

⁢

A

𝐂

1

,

𝒱

⁢

S

𝐂

1

)

[

𝒱

⁢

A

𝐂

2

→

𝑇

𝒱

⁢

S

𝐂

2

]

↦

[

𝒱

A

𝐂

1

→

π

A

♮

𝒱

A

𝐂

2

→

𝑇

𝒱

S

𝐂

2

→

π

S

𝒱

S

𝐂

1

]

,

where

π

A

and

π

S

are shorthands for projections
as in Example 1 (

http://planetmath.org/2stochasticmaps#Thmeg1

)

The reason for naming

𝙼𝚎𝚊𝚜

𝐃

the category of measuring
devices will become clear in §4 (

http://planetmath.org/4measurement

)
below. The two categories are connected by

contravariant
functor

ℱ

:

Theorem 4

(structure presheaf).

The

structure

presheaf

F

taking

ℱ

𝐃

:

𝚂𝚢𝚜

𝐃

𝚘𝚙

→

𝙼𝚎𝚊𝚜

𝐃

:

𝐂

↦

Hom

⁡

(

𝒱

⁢

A

𝐂

,

𝒱

⁢

S

𝐂

)

⁢

and

⁢

i

12

↦

r

21

satisfies

the gluing axiom but has non-unique descent.

Proof:

Functor

ℱ

is trivially a presheaf since it is
contravariant. It is an

interesting

presheaf because
the gluing axiom holds.

For gluing we need to show that for any collection

{

𝐂

j

}

j

=

1

l

of subsystems and sections

𝔪

j

♮

∈

ℱ

𝐃

⁢

(

𝐂

j

)

such that

r

j

,

j

⁢

i

⁢

(

𝔪

j

♮

)

=

r

i

,

j

⁢

i

⁢

(

𝔪

i

♮

)

for all

i

,

j

there exists section

𝔪

♮

∈

ℱ

𝐃

⁢

(

⋃

j

=

1

l

𝐂

j

)

such that

r

i

⁢

(

𝔪

♮

)

=

𝔪

i

♮

for all

i

. This reduces
to finding a conditional distribution that causes

diagram

in

𝙼𝚎𝚊𝚜

𝐃

to commute. The vertices are conditional distributions and the arrows are marginalizations, so rewrite as

where

p

(

x

|

w

)

=

∑

v

,

z

p

(

x

,

z

|

v

,

w

)

p

m

⁢

a

⁢

x

⁢

H

(

v

)

and similarly
for the vertical arrow. It is easy to see that

p

(

x

,

y

,

z

|

u

,

v

,

w

)

:=

p

(

x

,

y

|

u

,

w

)

p

(

x

,

z

|

v

,

w

)

p

(

x

|

w

)

satisfies the requirement.

For

ℱ

to be a sheaf it would also have to satisfy
unique descent: the section satisfying the gluing axiom must
not only

exist

for any collection

{

𝐂

j

}

j

=

1

l

with

compatible

restrictions

but must
also be

unique

. Descent in

ℱ

is not unique
because there are many

distributions

satisfying the requirement
above: strictly speaking

r

is a marginalization

operator

rather than restriction. For example, there are many
distributions

p

⁢

(

y

,

z

)

that marginalize to give

p

⁢

(

y

)

and

p

⁢

(

z

)

besides the

product distribution

p

⁢

(

y

)

⁢

p

⁢

(

z

)

.

■

The structure presheaf

ℱ

depends on the graph
structure and alphabets; mechanisms play no role. We now

construct

a family of sections of

ℱ

using the
mechanisms of

𝐃

’s occasions. Specifically, given a
subsystem

𝐂

∈

𝚂𝚢𝚜

𝐃

, we show how to glue its
occasions’ mechanisms together to form joint mechanism

𝔪

𝐂

. The mechanism

𝔪

𝐃

=

𝔪

⊤

of the
entire system

𝐃

is recovered as a special case.

In general, subsystem

𝐂

is not

isolated

: it receives
inputs along edges

contained

in

𝐃

but

not

in

𝐂

. Inputs along these edges cannot be assigned a fixed
value since in general there is no preferred element of

A

l

. They also cannot be ignored since

𝔪

l

is defined
as receiving inputs from all its sources. Nevertheless, the
mechanism of

𝐂

should depend on

𝐂

alone. We
therefore treat edges not in

𝐂

as sources of extrinsic
noise by marginalizing with respect to the

uniform distribution

as in Corollary 3 (

http://planetmath.org/2stochasticmaps#Thmthm3

).

For each vertex

v

l

∈

t

⁢

r

⁢

g

⁢

(

𝐂

)

let

S

l

𝐂

=

∏

k

∈

s

⁢

r

⁢

c

⁢

(

l

)

∩

s

⁢

r

⁢

c

⁢

(

C

)

A

k

. We then have projection

π

l

:

𝒱

⁢

S

l

→

𝒱

⁢

S

l

𝐂

.
Define

𝔪

l

𝐂

:=

[

𝒱

S

l

𝐂

→

π

l

♮

𝒱

S

l

→

𝔪

l

𝒱

A

l

]

.

(1)

It follows immediately that

𝐂

is itself a distributed
dynamical system defined by its graph, whose alphabets are
inherited from

𝐃

and whose mechanisms are constructed
by marginalizing.

Next, we tensor the mechanisms of

individual

occasions and glue
them together using the

diagonal map

Δ

:

S

𝐂

→

∏

v

l

∈

t

⁢

r

⁢

g

⁢

(

𝐂

)

S

l

𝐂

. The
diagonal map used here

1

1

which is

surjective

in the
sense that all rows contain non-zero entries

generalizes

X

→

Δ

X

×

X

and removes redundancies in

∏

l

S

l

𝐂

, which may, for example, include the
same source alphabets many times in different factors.

Let mechanism

𝔪

𝐂

be

𝔪

𝐂

:=

[

𝒱

S

𝐂

→

ι

Δ

⊗

v

l

∈

t

⁢

r

⁢

g

⁢

(

𝐂

)

𝒱

S

l

𝐂

→

⊗

v

l

∈

t

⁢

r

⁢

g

⁢

(

𝐂

)

𝔪

l

𝐂

𝒱

A

𝐂

]

.

(2)

The dual of

𝔪

𝐂

is

𝔪

𝐂

♮

:=

[

𝒱

A

𝐂

→

𝒱

S

𝐂

]

.

(3)

Finally, we find that we have constructed a family of sections
of

ℱ

:

Definition 7

.

The

quale

𝐪

𝐃

is the family of
sections of

ℱ

constructed in
Eqs. (

1

), (

2

) and
(

3

)

𝐪

𝐃

:=

{

𝔪

𝐂

♮

∈

ℱ

⁢

(

𝐂

)

=

Hom

⁡

(

𝒱

⁢

A

𝐂

,

𝒱

⁢

S

𝐂

)

|

𝐂

∈

𝚂𝚢𝚜

𝐃

}

.

The construction used to glue together the mechanism of the
entire system can also be used to construct the mechanism of
any subsystem, which provides a window – the quale – into
the compositional structure of distributed processes.

References

1

DJ Amit (1989):

Modelling brain

function

: the world of attractor neural
networks

.

Cambridge University Press.

2

David Balduzzi (2011):

Detecting emergent processes in cellular automata with
excess information

.

preprint

.

3

ER Berlekamp, JC Conway &
RK Guy (1982):

Winning Ways for your Mathematical Plays

.

2, Academic Press.

4

JJ Hopfield (1982):

Neural networks and physical systems with emergent
computational properties

.

Proc. Nat. Acad. Sci.

79,
pp. 2554–2558.

Title

3. Distributed dynamical systems

Canonical name

3DistributedDynamicalSystems

Date of creation

2014-04-30 18:41:46

Last modified on

2014-04-30 18:41:46

Owner

rspuzio (6075)

Last modified by

rspuzio (6075)

Numerical id

15

Author

rspuzio (6075)

Entry type

Feature

Classification

msc 60J20

Classification

msc 81P15

Classification

msc 18F20

Generated on Sat Feb 10 12:01:23 2018 by

LaTeXML