4. Measurement

4. Measurement

This

section

adapts Definition 1 (

http://planetmath.org/1introduction#Thmdefn1

)
to distributed
stochastic systems. The first step is to replace elements of

state space

X

with stochastic maps

d

i

⁢

n

:

ℝ

→

𝒱

⁢

S

𝐃

, or equivalently probability distributions
on

S

𝐃

, which are the system’s inputs.

Individual

elements of

S

𝐃

correspond to Dirac distributions.

Second, replace

function

f

:

X

→

ℝ

with mechanism

𝔪

𝐃

:

𝒱

⁢

S

𝐃

→

𝒱

⁢

A

𝐃

.
Since we are interested in the compositional

structure

of
measurements we also consider submechanisms

𝔪

𝐂

.
However, comparing mechanisms requires that they have the same
domain and range, so we extend

𝔪

𝐂

to the entire
system as follows

𝔪

𝐂

=

𝒱

⁢

S

𝐃

→

𝜋

𝒱

⁢

S

𝐂

→

𝔪

𝐂

𝒱

⁢

A

𝐂

→

π

♮

𝒱

⁢

A

𝐃

.

(1)

We refer to the

extension

as

𝔪

𝐂

by abuse of notation.
We extend mechanisms implicitly whenever necessary without
further comment. Extending mechanisms in this way maps the
quale into a cloud of points in

Hom

⁡

(

𝒱

⁢

A

𝐃

,

𝒱

⁢

S

𝐃

)

labeled by objects in

𝚂𝚢𝚜

𝐃

.

In the special case of the

initial object

⊥

𝐃

, define

𝔪

⊥

=

𝒱

⁢

S

𝐃

→

𝜔

ℝ

→

ω

♮

𝒱

⁢

A

𝐃

.

Remark 3

.

Subsystems differing by non-existent edges (Remark 2 (

http://planetmath.org/3distributeddynamicalsystems#Thmrem2

))
are mapped to the same mechanism by this construction, thus
making the fact that the edges do not exist explicit within
the

formalism

.

Composing an input with a submechanism yields an output

d

o

⁢

u

⁢

t

:=

𝔪

𝐂

∘

d

i

⁢

n

:

ℝ

→

𝒱

⁢

A

𝐃

, which is a probability distribution on

A

𝐃

. We are now in a position to define

Definition 8

.

A

measuring device

is the dual

𝔪

𝐂

♮

to the mechanism of a subsystem. An

output

is a
stochastic map

d

o

⁢

u

⁢

t

:

ℝ

→

𝒱

⁢

A

𝐃

. A

measurement

is a composition

𝔪

𝐂

♮

∘

d

o

⁢

u

⁢

t

:

ℝ

→

𝒱

⁢

S

𝐃

.

Recall that stochastic maps of the form

ℝ

→

𝒱

⁢

X

correspond to probability distributions on

X

. Outputs as
defined above are thus probability distributions on

A

𝐃

, the output

alphabet

of

𝐃

. Individual
elements of

A

𝐃

are recovered as Dirac vectors:

ℝ

→

δ

a

𝒱

⁢

A

𝐃

.

Definition 9

.

The

effective

information

generated by

𝐂

1

in the

context

of subsystem

𝐂

2

⊂

𝐂

1

is

e

i

(

𝔪

𝐂

2

→

𝔪

𝐂

1

,

d

o

⁢

u

⁢

t

)

:=

H

[

𝔪

𝐂

1

♮

∘

d

o

⁢

u

⁢

t

∥

𝔪

𝐂

2

♮

∘

d

o

⁢

u

⁢

t

]

.

(2)

The

null context

, corresponding to the empty subsystem

⊥

=

∅

⊂

V

𝐃

×

V

𝐃

, is a special
case where

𝔪

𝐂

♮

∘

d

o

⁢

u

⁢

t

is replaced by the

uniform distribution

ω

𝐃

♮

on

S

𝐃

.
To simplify notation define

e

i

(

𝔪

𝐂

,

d

o

⁢

u

⁢

t

)

:=

e

i

(

𝔪

⊥

→

𝔪

𝐂

,

d

o

⁢

u

⁢

t

)

.

Here,

H

[

p

∥

q

]

=

∑

i

p

i

log

2

p

i

q

i

is the
Kullback-Leibler divergence or

relative entropy

[

1

]

. Eq. (

2

) expands as

e

i

(

𝔪

𝐂

2

→

𝔪

𝐂

1

,

d

o

⁢

u

⁢

t

)

=

∑

s

∈

S

𝐃

⟨

𝔪

𝐂

1

♮

∘

d

o

⁢

u

⁢

t

|

δ

s

⟩

⋅

log

2

⟨

𝔪

𝐂

1

♮

∘

d

o

⁢

u

⁢

t

|

δ

s

⟩

⟨

𝔪

𝐂

2

♮

∘

d

o

⁢

u

⁢

t

|

δ

s

⟩

.

(3)

When

d

o

⁢

u

⁢

t

=

δ

a

for some

a

∈

A

𝐃

we
have

e

i

(

𝔪

𝐂

2

→

𝔪

𝐂

1

,

δ

a

)

=

∑

s

∈

S

𝐃

p

𝔪

𝐂

1

(

s

|

a

)

⋅

log

2

p

𝔪

𝐂

1

(

s

|

a

)

p

𝔪

𝐂

2

(

s

|

a

)

.

(4)

Definition

8

requires some unpacking. To relate
it to the classical notion of measurement, Definition 1 (

http://planetmath.org/1introduction#Thmdefn1

),
we consider system

𝐃

=

{

v

X

→

𝑓

v

Y

}

where the alphabets of

v

X

and

v

Y

are the sets

A

v

X

=

X

and

A

v

Y

=

Y

respectively, and the
mechanism of

v

Y

is

𝔪

Y

=

𝒱

⁢

f

. In other words,
system

𝐃

corresponds to a single

deterministic

function

f

:

X

→

Y

.

Proposition 5

(classical measurement).

The measurement

(

V

⁢

f

)

♮

∘

δ

y

performed when deterministic function

f

:

X

→

Y

outputs

y

is

equivalent

to the

preimage

f

-

1

⁢

(

y

)

.
Effective information is

e

⁢

i

⁢

(

V

⁢

f

,

δ

y

)

=

log

2

⁡

|

X

|

|

f

-

1

⁢

(

y

)

|

.

Proof:
By Corollary 2 (

http://planetmath.org/2stochasticmaps#Thmthm2

)
measurement

(

𝒱

⁢

f

)

♮

∘

δ

y

is conditional
distribution

p

𝒱

⁢

f

(

x

|

y

)

=

{

1

|

f

-

1

⁢

(

y

)

|

if

⁢

f

⁢

(

x

)

=

y

0

else

.

which generalizes the preimage. Effective information follows
immediately.

■

Effective information can be interpreted as quantifying a
measurement’s precision. It is high if few inputs cause

f

to output

y

out of many – i.e.

f

-

1

⁢

(

y

)

has few

elements

relative to

|

X

|

– and

conversely

is low if many
inputs cause

f

to output

y

– i.e. if the output is
relatively insensitive to changes in the input. Precise
measurements say a lot about what the input could have been
and conversely for vague measurements with low

e

⁢

i

.

The point of this paper is to develop techniques for studying
measurements constructed out of two or more functions. We
therefore present

computations

for the simplest case,
distributed system

X

×

Y

→

𝑔

Z

, in considerable
detail. Let

𝐃

be the graph

with obvious assignments of alphabets and the mechanism of

v

Z

as

𝔪

Z

=

𝒱

⁢

g

. To make the

formulas

more readable
let

𝔪

X

⁢

Y

=

𝒱

⁢

g

,

𝔪

X

⁣

∙

=

𝒱

⁢

g

∘

π

X

⁢

Y

,

X

♮

and

𝔪

∙

Y

=

𝒱

⁢

g

∘

π

X

⁢

Y

,

Y

♮

. We then obtain

lattice

The remainder of this section and most of the next analyzes
measurements in the lattice.

Proposition 6

(partial measurement).

The measurement performed on

X

when

g

:

X

×

Y

→

Z

outputs

z

, treating

Y

as extrinsic
noise, is conditional distribution

p

(

x

|

z

)

=

{

|

g

x

×

Y

-

1

⁢

(

z

)

|

|

g

-

1

⁢

(

z

)

|

if

⁢

g

⁢

(

x

,

y

)

=

z

⁢

for some

⁢

y

∈

Y

0

else,

(5)

where

g

x

×

Y

-

1

⁢

(

z

)

:=

p

⁢

r

Y

⁢

(

g

-

1

⁢

(

z

)

∩

{

x

}

×

Y

)

. The effective information generated by the
partial measurement is

e

i

(

𝔪

X

⁣

∙

♮

,

δ

z

)

=

log

2

|

X

|

+

∑

x

∈

X

p

(

x

|

z

)

⋅

log

2

p

(

x

|

z

)

.

δ

z

)

|

g

-

1

(

z

)

|

⋅

(6)

Proof: Treating

Y

as a source of extrinsic noise yields

𝒱

⁢

X

→

π

♮

𝒱

⁢

X

⊗

𝒱

⁢

Y

→

𝒱

⁢

g

𝒱

⁢

Z

which takes

δ

x

↦

1

|

Y

|

⁢

∑

y

∈

Y

δ

g

⁢

(

x

,

y

)

.
The dual is

𝔪

X

⁣

∙

♮

=

π

X

⁢

Y

,

X

∘

(

𝒱

⁢

g

)

♮

:

δ

z

↦

∑

x

∈

X

|

g

x

×

Y

-

1

⁢

(

z

)

|

|

g

-

1

⁢

(

z

)

|

⋅

δ

x

.

The computation of effective information follows immediately.

■

A partial measurement is precise if the preimage

g

-

1

⁢

(

z

)

has small or empty

intersection

with

{

x

}

×

Y

for most

x

, and large intersection for few

x

.

Propositions

5

and

6

compute
effective information of a measurement relative to the null
context provided by

complete

ignorance (the uniform
distribution). We can also compute the effective information
generated by a measurement in the context of a submeasurement:

Proposition 7

(relative measurement).

The information generated by measurement

X

×

Y

→

𝑔

Z

in the context of the partial measurement where

Y

is unobserved noise, is

e

i

(

𝔪

X

⁣

∙

→

𝔪

X

⁢

Y

,

δ

z

)

=

∑

x

∈

X

g

x

×

Y

-

1

⁢

(

z

)

g

-

1

⁢

(

z

)

log

2

|

Y

|

g

x

×

Y

-

1

⁢

(

z

)

.

(7)

Proof: Applying Propositions

5

and

6

obtains

e

i

(

𝔪

X

⁣

∙

→

𝔪

X

⁢

Y

,

δ

z

)

=

∑

(

x

,

y

)

∈

g

-

1

⁢

(

z

)

1

|

g

-

1

⁢

(

z

)

|

log

2

[

1

|

g

-

1

⁢

(

z

)

|

⋅

|

g

-

1

⁢

(

z

)

|

⋅

|

Y

|

|

g

x

×

Y

-

1

⁢

(

z

)

|

]

which simplifies to the desired expression.

■

To interpret the result decompose

X

×

Y

→

𝑔

Z

into a family of functions

ℛ

=

{

Y

→

g

x

×

Y

Z

|

x

∈

X

}

labeled by elements of

X

, where

g

x

×

Y

⁢

(

y

)

:=

g

⁢

(

x

,

y

)

.
The precision of the measurement performed by

g

x

×

Y

s

log

2

⁡

|

Y

|

g

x

×

Y

-

1

⁢

(

z

)

. It follows that
the precision of the relative measurement,
Eq. (

7

), is the expected precision of the
measurements performed by family

ℛ

taken with
respect to the probability distribution

p

(

x

|

z

)

=

g

x

×

Y

-

1

⁢

(

z

)

g

-

1

⁢

(

z

)

generated by
the noisy measurement.

In the special case of

g

:

X

×

Y

→

Z

relative
precision is simply the

difference

of the precision of the
larger and smaller subsystems:

Corollary 8

(comparing measurements).

e

i

(

𝔪

X

⁣

∙

→

𝔪

X

⁢

Y

,

δ

z

)

=

e

i

(

𝔪

X

⁢

Y

,

δ

z

)

-

e

i

(

𝔪

X

⁣

∙

,

δ

z

)

Proof: Applying Propositions

5

,

6

,

7

and simplifying obtains

e

⁢

i

⁢

(

𝔪

X

⁢

Y

,

δ

z

)

-

e

⁢

i

⁢

(

𝔪

X

⁣

∙

,

δ

z

)

=

log

2

⁡

|

X

|

⋅

|

Y

|

|

g

-

1

⁢

(

z

)

|

-

∑

x

|

g

x

×

Y

-

1

⁢

(

z

)

|

|

g

-

1

⁢

(

z

)

|

⁢

log

2

⁡

|

X

|

⋅

|

g

x

×

Y

-

1

⁢

(

z

)

|

|

g

-

1

⁢

(

z

)

|

=

log

2

⁡

|

Y

|

|

g

-

1

⁢

(

z

)

|

+

∑

(

x

,

y

)

∈

g

-

1

⁢

(

z

)

1

|

g

-

1

⁢

(

z

)

|

⁢

log

2

⁡

|

g

-

1

⁢

(

z

)

|

|

g

x

×

Y

-

1

⁢

(

z

)

|

=

e

i

(

𝔪

X

⁣

∙

→

𝔪

X

⁢

Y

,

δ

z

)

.

■

References

1

E T Jaynes (1985):

Entropy

and Search Theory

.

In CR Smith & WT Grandy,
editors:

Maximum-entropy and Bayesian Methods in
Inverse Problems

, Springer.

Title

4. Measurement

Canonical name

4Measurement

Date of creation

2014-04-22 19:36:22

Last modified on

2014-04-22 19:36:22

Owner

rspuzio (6075)

Last modified by

rspuzio (6075)

Numerical id

6

Author

rspuzio (6075)

Entry type

Feature

Generated on Sat Feb 10 12:01:29 2018 by

LaTeXML