Part

II

Cache

Awareness

in

Blocking

Techniques

O.

Temam

C.

Fricker

W.

Jalby

Abstract

To

date

data

locality

optimizing

algorithms

mostly

aim

at

providing

efficient

strategies

for

blocking

and

reordering

loops

But

little

research

has

been

devoted

to

the

final

step

i.e.

computing

the

optimal

block

size

Optimal

block

sizes

are

currently

computed

as

if

a

cache

behaves

as

a

local

memory

i.e.

cache

interference

phenomena

are

ignored

Case-studies

have

already

shown

that

cache

interferences

can

greatly

affect

the

optimal

block

size

The

purpose

of

this

paper

is

to

propose

a

methodology

for

estimating

interference

misses

in

a

regular

do-loop

nest

and

use

that

knowledge

to

derive

the

optimal

block

size

First

the

different

types

of

interference

phenomena

are

identified

and

a

method

for

predicting

their

occurrence

and

evaluating

their

impact

is

proposed

Second

current

techniques

for

computing

the

optimal

block

size

are

analytically

and

experimentally

shown

to

yield

far

below

optimal

performance

Third

cache

interference

phenomena

and

even

TLB

behavior

are

taken

into

account

in

the

computation

of

the

optimal

block

size

which

proves

to

yield

near-optimal

performance

and

consequently

make

blocking

techniques

safe

Reciprocally

it

is

also

shown

that

even

when

no

capacity

miss

occurs

blocking

techniques

can

be

used

to

significantly

reduce

the

number

of

cache

interferences

Contents

II

Cache

Awareness

in

Blocking

Techniques

Cache

Awareness

in

Blocking

Techniques

Introduction

Motivations

and

Goals

Position

of

the

Problem

Modeling

Cache

Behavior

Basic

Concepts

Reuse

Set

Interference

Set

Evaluating

the

Actual

Estimating

the

Optimal

Block

Size

TLB

Capacity

Misses

Cache

Interference

Misses

Extension

to

Spatial

Reuse

Further

Issues

in

the

Evaluation

of

Cache

Interferences

Global

Algorithm

Conclusions

and

Future

Directions

Chapter

Cache

Awareness

in

Blocking

Techniques

Introduction

Motivations

and

Goals

As

mentioned

in

-LSB-

-RSB-

three

types

of

cache

misses

can

be

distinguished

cold-start

misses

which

are

compulsory

capacity

misses

and

interference

-LRB-

or

conflict

-RRB-

misses

Capacity

misses

occur

when

cache

space

is

unsufficient

to

store

all

data

to

be

reused

Interference

misses

occur

when

two

data

are

mapped

to

the

same

cache

location

-LRB-

in

direct-mapped

caches

-RRB-

The

object

of

data

locality

optimizing

algorithms

-LSB-

-RSB-

is

to

reduce

capacity

misses

only

While

they

are

mostly

used

for

cache

performance

optimization

most

of

them

are

designed

for

local

memories

in

the

sense

that

they

ignore

cache

interferences

due

to

the

mapping

function

used

in

caches

Cache

awareness

in

blocking

techniques

is

generally

limited

to

taking

into

account

the

influence

of

the

cache

line

size

This

is

already

a

significant

enhancement

because

spatial

locality

exploitation

-LRB-

i.e.

stride-one

access

-RRB-

should

often

be

favored

over

temporal

locality

exploitation

However

of

these

techniques

deal

with

cache

interferences

except

for

-LSB-

-RSB-

which

suggests

data

locality

optimizing

techniques

could

be

made

more

accurate

if

cache

conflicts

are

considered

and

illustrates

this

point

with

an

example

Another

important

paper

on

the

topic

is

-LSB-

-RSB-

Using

an

example

it

is

shown

that

cache

interferences

strongly

influence

the

optimal

block

size

value

The

purpose

of

our

paper

is

to

present

a

framework

for

predicting

and

estimating

cache

interferences

in

numerical

loop

nests

and

use

that

framework

to

derive

the

optimal

block

size

of

a

blocked

loop

nest

This

paper

is

focused

on

direct-mapped

caches

which

are

more

sensitive

to

cache

interferences

Position

of

the

Problem

Computing

the

Optimal

Block

Size

Blocking

can

be

decomposed

into

two

steps

first

perform

strip-mining

loop

interchanging

and

skewing

according

to

a

certain

strategy

and

then

compute

the

optimal

block

size

The

first

step

constitutes

the

bulk

of

most

data

locality

optimizing

algorithms

-LSB-

-RSB-

The

second

step

is

the

computation

of

the

optimal

block

size

Most

of

these

algorithms

do

not

address

this

issue

except

that

the

authors

of

-LSB-

-RSB-

provide

in

-LSB-

-RSB-

a

case-study

of

matrix-matrix

multiply

where

the

optimal

block

size

is

rigorously

evaluated

by

considering

self-interferences

For

the

general

problem

of

computing

the

optimal

block

size

of

any

loop

nest

the

most

elaborate

treatment

can

be

found

in

-LSB-

-RSB-

so

we

will

start

from

that

point

In

-LSB-

-RSB-

for

each

reference

the

set

of

data

to

be

reused

is

called

the

reference

window

and

the

optimal

block

size

is

the

solution

of

the

following

optimization

problem

notion

is

very

similar

to

the

working

set

of

a

reference

Problem

Let

B

the

block

size

of

any

blocked

loop

in

the

loop

nest

Let

Ncapacity

-LRB-

B

-RRB-

the

number

of

capacity

misses

of

an

n-deep

loop

nest

Let

W

-LRB-

B

-RRB-

the

sum

of

the

window

sizes

over

all

references

Find

B

that

minimizes

Ncapacity

-LRB-

B

-RRB-

under

the

constraint

that

all

windows

fit

in

cache

i.e.

W

-LRB-

B

-RRB-

CS

-LRB-

where

CS

is

the

cache

size

-RRB-

Because

Ncapacity

-LRB-

B

-RRB-

only

corresponds

to

compulsory

and

capacity

misses

the

optimal

block

sizes

obtained

are

inaccurate

In

order

to

obtain

a

more

accurate

solution

the

following

problem

must

be

solved

Problem

Let

Ninterference

-LRB-

B

-RRB-

the

number

of

interference

misses

Find

B

that

minimize

Ncapacity

-LRB-

B

-RRB-

Ninterference

-LRB-

B

-RRB-

under

the

constraint

W

-LRB-

B

-RRB-

CS

So

the

problem

sums

up

to

computing

the

total

number

of

cache

misses

in

a

loop

nest

Note

that

the

optimization

proposed

in

this

paper

is

not

a

data

locality

optimizing

algorithm

in

itself

On

the

contrary

it

is

a

complement

to

most

existing

algorithms

However

strategies

that

rely

on

the

evaluation

of

cache

misses

for

directing

loop

reordering

can

be

strongly

affected

by

this

enhancement

Blocking

to

Reduce

Cache

Interferences

Cache

interferences

operate

by

disrupting

the

reuse

of

data

Interferences

are

diverse

and

frequent

because

any

type

of

reuse

can

be

disrupted

temporal/spatial

-LSB-

-RSB-

In

general

the

larger

the

reuse

distance

for

a

given

datum

the

higher

the

probability

it

is

flushed

from

cache

because

the

more

data

are

loaded

before

reuse

occurs

Now

the

initial

goal

of

blocking

is

to

reduce

the

amount

of

data

to

be

reused

-LRB-

i.e.

to

be

kept

in

cache

-RRB-

A

side-effect

of

this

strategy

is

to

decrease

the

reuse

distance

Consequently

even

when

no

capacity

miss

occurs

blocking

can

prove

to

be

efficient

for

reducing

cache

interferences

Organization

of

the

Paper

Section

is

a

theoretical

section

where

the

framework

used

for

modeling

interferences

is

presented

Section

shows

how

each

type

of

interference

misses

is

computed

and

how

this

evaluation

is

used

for

computing

the

optimal

block

size

The

examples

are

also

used

to

show

that

blocking

is

a

method

for

reducing

the

number

of

interference

misses

when

no

capacity

miss

occurs

Finally

an

algorithm

is

proposed

for

systematically

evaluating

the

optimal

block

size

of

a

do-loop

nest

Experiments

All

experiments

have

been

conducted

on

an

HP/PA-RISC

workstation

-LRB-

except

for

figure

f

-RRB-

In

the

paper

all

dimensions

are

expressed

in

double-precision

floating

data

or

bytes

i.e.

the

size

of

an

array

element

The

cache

parameters

and

the

notations

used

in

the

paper

are

the

following

cache

size

CS

kbytes

array

elements

cache

line

size

LS

bytes

array

elements

the

cache

is

direct-mapped

TLB

size

TS

entries

page

size

PS

kbytes

elements

It

was

experimentally

found

that

a

TLB

miss

is

times

more

expensive

than

a

cache

miss

All

loops

were

compiled

using

the

highest

optimization

level

the

sake

of

simplicity

the

same

block

size

is

used

for

all

loops

Further

performance

improvements

might

be

obtained

with

a

distinct

block

size

for

each

loop

that

the

number

of

compulsory

misses

is

not

considered

because

it

is

independent

of

B

More

generally

any

term

independent

of

B

is

ignored

reference

reuses

its

own

data

reference

reuses

the

data

of

another

reference

is

introduced

in

section

Note

that

is

particularly

large

on

the

HP

on

a

SUN

Sparcstation

Modeling

Cache

Behavior

In

this

section

the

techniques

used

to

estimate

the

number

of

interference

misses

are

presented

Though

the

technique

applies

to

all

types

of

reuse

the

method

is

illustrated

with

self-dependence

temporal

reuse

because

it

is

a

frequent

type

of

reuse

and

because

the

associated

reuse

distance

is

large

so

that

it

is

likely

to

be

victim

of

interferences

Extensions

to

spatial

and

group-dependence

reuse

can

be

found

in

the

associated

technical

report

Still

the

extension

to

spatial

reuse

is

illustrated

in

section

Basic

Concepts

Evaluating

the

miss

ratio

of

a

loop

nest

amounts

to

counting

for

each

reference

the

number

of

cache

misses

due

to

disruption

of

-LRB-

spatial

or

temporal

-RRB-

locality

exploitation

So

for

each

reference

it

is

necessary

to

determine

when

reuse

occurs

-LRB-

i.e.

the

loop

level

where

reuse

occurs

for

the

first

time

-RRB-

Consider

loop

a

The

reuse

of

reference

F

-LRB-

-RRB-

occurs

on

loop

and

the

reuse

of

reference

G

-LRB-

-RRB-

occurs

on

loop

This

reuse

loop

level

defines

the

set

of

data

to

be

reused

For

reference

F

-LRB-

-RRB-

it

is

equal

to

fF

-LRB-

-RRB-

F

-LRB-

N

N

-RRB-

g

while

it

is

equal

to

fG

-LRB-

-RRB-

G

-LRB-

N

-RRB-

g

for

reference

G

-LRB-

-RRB-

Similarly

this

loop

level

also

defines

the

sets

of

data

of

all

other

references

that

can

interfere

with

the

reuse

set

For

reference

F

-LRB-

-RRB-

the

interference

sets

are

fG

-LRB-

-RRB-

G

-LRB-

N

-RRB-

g

and

fE

-LRB-

-RRB-

E

-LRB-

N

-RRB-

g

-LRB-

G

and

E

can

flush

elements

of

F

from

cache

before

they

are

reused

-RRB-

see

figure

a.

For

reference

G

-LRB-

-RRB-

the

interference

sets

are

fF

-LRB-

-RRB-

F

-LRB-

N

-RRB-

g

and

fE

-LRB-

-RRB-

g.

On

each

iteration

of

the

reuse

loop

the

number

of

cache

misses

due

to

disruption

of

reuse

is

equal

to

the

size

of

the

intersection

-LRB-

in

cache

lines

-RRB-

between

the

reuse

set

and

the

interference

sets

Considering

the

problem

this

way

implicitly

makes

abstraction

of

time

considerations

i.e.

when

interferences

occur

The

problem

is

then

equivalent

to

computing

the

intersection

size

between

several

sets

Summing

these

intersections

over

all

iterations

of

the

reuse

loop

provides

the

total

number

of

cache

interference

misses

for

a

given

reference

The

next

sections

provide

a

formal

treatment

of

this

method

Reuse

Set

Within

a

set

of

array

elements

to

be

reused

it

is

possible

that

two

elements

map

to

the

same

cache

location

These

elements

then

alternately

flush

each

other

from

cache

before

they

can

be

reused

Therefore

they

should

not

be

counted

within

the

set

of

elements

that

can

be

effectively

reused

It

is

necessary

to

distinguish

between

theoretical

reuse

set

and

actual

reuse

set

Definition

Assuming

loops

jn

are

perfectly

nested

-LRB-

with

jn

the

innermost

loop

-RRB-

the

reuse

loop

level

l

of

a

reference

R

rnjn

is

defined

as

l

max

fk

j

rk

g.

The

ri

are

the

loop

indices

coefficients

of

the

array

subscript

obtained

after

linearization

Definition

For

any

array

reference

R

rnjn

which

reuse

loop

is

l

the

theoretical

reuse

set

is

equal

to

TRS

-LRB-

R

-RRB-

rnjn

-LRB-

ji

Ni

-RRB-

i

lg

Ni

correspond

to

the

loop

boundaries

The

coefficients

ri

are

modified

so

that

the

loop

boundaries

can

be

expressed

as

ji

Ni

This

condition

excludes

most

loops

with

non-constant

boundaries

Definition

The

actual

reuse

set

ARS

-LRB-

R

-RRB-

is

the

cache

footprint

of

the

theoretical

reuse

set

TRS

-LRB-

R

-RRB-

excluding

all

cache

lines

for

which

mapping

conflicts

occur

technical

report

is

not

referenced

here

because

of

the

review

process

is

anonymous

that

only

simple

dependences

are

considered

for

instance

the

reuse

associated

with

reference

A

-LRB-

-RRB-

is

not

considered

The

most

frequently

found

dependences

are

simple

ones

as

mentioned

in

-LSB-

-RSB-

Definition

kTRS

-LRB-

R

-RRB-

k

is

the

size

of

the

theoretical

reuse

set

of

R

expressed

in

array

lines

i.e.

the

number

of

cache

lines

used

by

TRS

-LRB-

R

-RRB-

assuming

an

infinite

cache

size

Definition

kARS

-LRB-

R

-RRB-

k

is

the

size

of

the

actual

reuse

set

of

R

expressed

in

cache

lines

Example

Consider

loop

a

The

reuse

loop

level

of

G

-LRB-

-RRB-

is

-LRB-

or

loop

-RRB-

The

reuse

set

of

G

-LRB-

-RRB-

is

equal

to

-LRB-

N

-RRB-

g

where

is

the

base

address

of

array

G

and

N

is

the

leading

dimension

of

arrays

E

F

G

In

this

case

kTRS

-LRB-

R

-RRB-

k

N

array

elements

The

actual

reuse

set

is

equal

to

the

cache

lines

corresponding

to

cache

locations

f

-LRB-

-RRB-

mod

CS

-LRB-

N

-RRB-

g.

Assuming

a

cache

size

of

array

elements

-LRB-

words

-RRB-

and

an

-LRB-

words

-RRB-

cache

line

the

actual

reuse

set

of

G

is

equal

to

N

cache

lines

if

N

it

is

equal

to

N

cache

lines

if

N

and

it

is

empty

if

N

Interference

Set

The

definition

of

the

set

of

array

elements

that

can

interfere

with

a

reuse

set

is

very

similar

to

the

definition

of

a

reuse

set

Definition

For

any

array

reference

R

rnjn

that

can

interfere

with

a

reuse

set

defined

on

loop

level

l

the

theoretical

interference

set

is

equal

to

TIS

-LRB-

R

-RRB-

rnjn

-LRB-

ji

Ni

-RRB-

i

lg

In

opposition

to

the

actual

reuse

set

when

two

elements

of

a

theoretical

interference

set

map

to

the

same

cache

line

this

line

is

still

counted

in

the

actual

interference

set

Definition

The

actual

interference

set

AIS

-LRB-

R

-RRB-

is

exactly

the

cache

footprint

of

the

theoretical

interference

set

TIS

-LRB-

R

-RRB-

Definition

As

for

the

reuse

set

kTIS

-LRB-

R

-RRB-

k

and

kAIS

-LRB-

R

-RRB-

k

denote

respectively

the

size

of

the

theoretical

and

actual

interference

set

of

R.

Example

Consider

loop

a

The

reuse

of

F

-LRB-

-RRB-

occurs

on

loop

So

the

corresponding

interference

set

of

G

-LRB-

-RRB-

is

defined

on

loop

The

interference

set

of

G

-LRB-

-RRB-

is

equal

to

-LRB-

N

-RRB-

g

In

this

case

kTIS

-LRB-

R

-RRB-

k

N

array

elements

Assuming

again

a

cache

size

of

array

elements

-LRB-

words

-RRB-

and

a

-LRB-

words

-RRB-

cache

line

the

actual

interference

set

of

G

is

equal

to

N

cache

lines

if

N

and

it

is

equal

to

cache

lines

if

N

Evaluating

the

Actual

Consider

a

reference

R

rnjn

of

which

we

want

to

compute

the

cache

footprint

on

loop

level

l

-LRB-

i.e.

the

footprint

is

defined

by

loops

n

n

l

-RRB-

Observation

After

iterating

on

any

loop

i

l

i

n

the

data

layout

in

cache

is

assumed

to

be

a

set

of

intervals

of

cache

locations

characterized

by

the

average

size

S

of

an

interval

and

the

average

distance

oe

between

two

consecutive

intervals

Consequently

a

recursive

process

can

be

applied

for

each

loop

level

i

l

i

n.

Problem

Assuming

an

initial

data

layout

of

intervals

of

size

S

separated

by

a

distance

of

oe

cache

locations

-LRB-

a

layout

obtained

after

iterating

on

loops

n

n

i

-RRB-

the

problem

is

to

determine

the

final

layout

i.e.

average

size

average

distance

after

N

iterations

of

loop

i.

section

can

be

skipped

if

in-depth

comprehension

of

the

computations

is

not

sought

for

e.

the

distance

between

any

two

consecutive

intervals

is

constant

Actual

interference

set

For

problem

a

recursive

process

can

also

be

applied

Let

CS

and

oe

After

a

number

of

iterations

the

intervals

wrap

around

the

cache

area

of

size

Within

each

area

of

cache

locations

the

layout

of

intervals

is

identical

So

only

one

such

area

of

size

can

be

studied

Within

one

such

area

the

spacing

between

two

consecutive

intervals

is

equal

to

mod

Let

us

consider

a

recursive

application

of

this

process

Observation

On

recursion

level

k

the

size

of

the

area

considered

is

equal

to

oek

and

the

spacing

between

two

consecutive

intervals

in

one

such

area

is

equal

to

oek

The

recursion

process

stops

on

a

level

s

when

all

iterations

of

loop

i

have

been

considered

or

when

overlapping

occurs

i.e.

if

oes

S.

Starting

at

this

point

it

is

possible

to

determine

the

footprint

within

an

area

of

size

oes

and

assuming

the

layout

is

regular

the

footprint

within

the

whole

cache

Proposition

Let

N

nsb

oes

c

r

-LRB-

with

r

N

mod

b

oes

c

-RRB-

In

b

oes

c

r

areas

of

size

oes

ns

intervals

of

size

S

have

been

brought

and

in

the

remaining

r

areas

ns

intervals

have

been

brought

Proposition

Let

fi

-LRB-

x

S

oes

-RRB-

S

-LRB-

x

-RRB-

oes

Then

the

cache

footprint

size

of

all

intervals

is

equal

to

max

b

oes

c

r

fi

-LRB-

ns

S

oes

-RRB-

r

fi

-LRB-

ns

S

oes

-RRB-

For

this

layout

of

intervals

the

average

size

of

an

interval

is

equal

to

max

b

oes

c

r

fi

-LRB-

ns

S

oes

-RRB-

r

fi

-LRB-

ns

S

oes

-RRB-

oes

and

the

spacing

between

intervals

is

equal

to

oes

Remark

oek

oek

oek

mod

oek

so

that

all

oek

can

be

computed

a

priori

with

the

cost

of

one

application

of

Euclide

algorithm

-LSB-

-RSB-

In

practice

this

fact

also

makes

the

process

non-recursive

since

the

level

at

which

recursion

stops

can

be

determined

beforehand

After

the

process

has

been

applied

for

all

loops

i

l

i

n

the

resulting

layout

in

cache

is

the

actual

interference

set

of

the

reference

defined

on

loop

l.

Actual

reuse

set

The

reasoning

is

nearly

identical

for

the

actual

reuse

set

except

that

cache

lines

where

overlapping

occurs

must

be

excluded

Proposition

Let

fr

-LRB-

x

S

oes

oes

-RRB-

S

if

x

S

oes

-LRB-

x

-RRB-

-LRB-

S

-RRB-

if

fi

-LRB-

x

-RRB-

oes

max

S

oes

-LRB-

x

-RRB-

-LRB-

S

-RRB-

-LRB-

fi

-LRB-

x

-RRB-

oes

-RRB-

if

fi

-LRB-

x

-RRB-

oes

The

number

of

elements

that

can

be

reused

is

equal

to

max

oes

r

fr

-LRB-

ns

S

oes

oes

-RRB-

r

fr

-LRB-

ns

S

oes

oes

-RRB-

An

example

of

application

of

the

whole

process

can

be

found

in

section

Note

that

the

process

is

inaccurate

because

the

layout

of

intervals

is

rarely

perfectly

regular

So

the

error

can

become

significant

after

multiple

applications

-LRB-

i.e.

for

multiple

loop

levels

-RRB-

Fortunately

a

max

-LRB-

-RRB-

reuse

set

defined

over

two

loop

levels

corresponds

to

a

loop

nest

where

reuse

occurs

on

the

third

loop

For

such

a

reuse

set

the

algorithm

needs

only

to

be

applied

once

-LRB-

except

if

the

access

stride

is

not

equal

to

one

-RRB-

Reuse

sets

defined

over

three

loops

-LRB-

i.e.

loop

nests

-RRB-

are

less

common

in

primitives

and

real

codes

-LSB-

-RSB-

or

are

less

likely

to

be

exploited

-LRB-

for

instance

loop

blocking

is

rarely

performed

over

more

than

the

two

innermost

loops

-RRB-

Estimating

the

Optimal

Block

Size

Section

describes

a

simple

heuristic

for

evaluating

TLB

misses

Section

is

the

most

important

one

it

is

shown

how

to

evaluate

cross-interference

and

self-interference

misses

Then

the

process

of

computing

the

optimal

block

size

is

illustrated

with

examples

and

a

formal

algorithm

is

provided

in

section

In

section

extension

to

spatial

reuse

is

discussed

In

section

further

issues

related

to

the

evaluation

of

interference

misses

are

discussed

TLB

Capacity

Misses

In

next

sections

the

computations

of

cache

misses

are

used

to

derive

the

optimal

block

size

However

most

processors

also

use

a

TLB

which

can

breed

TLB

misses

TLB

misses

are

often

more

costly

-LRB-

though

less

frequent

-RRB-

than

cache

misses

and

should

not

be

neglected

Usually

a

TLB

is

a

small

on-chip

fully

-LRB-

or

at

least

highly

-RRB-

associative

cache

Consequently

TLB

misses

are

mostly

capacity

misses

TLB

misses

TLB

misses

for

a

reference

can

be

estimated

the

following

way

Let

A

anjn

a

reference

with

ji

Ni

TLB

capacity

misses

occur

if

there

exists

a

loop

where

the

number

of

pages

crossed

is

larger

than

TS

-LRB-

the

TLB

size

in

number

of

entries

-RRB-

i.e.

such

that

nTLB

min

PS

PS

where

PS

is

the

page

size

-LRB-

in

array

elements

-RRB-

Then

the

number

of

TLB

misses

for

that

reference

is

approximately

equal

to

NTLB

nTLB

min

Influence

on

the

optimal

block

size

computation

The

general

optimization

problem

to

be

solved

for

obtaining

the

optimal

block

size

is

the

following

W

-LRB-

B

-RRB-

CS

find

B

that

minimizes

cache

misses

-LRB-

B

-RRB-

Assuming

one

TLB

miss

is

times

more

costly

than

a

cache

miss

the

optimization

problem

obtained

by

considering

TLB

misses

is

the

following

W

-LRB-

B

-RRB-

CS

find

B

that

minimizes

cache

misses

-LRB-

B

-RRB-

TLB

misses

-LRB-

B

-RRB-

Cache

Interference

Misses

Consider

the

reuse

set

of

a

reference

R

-LRB-

reuse

occurs

on

loop

l

-RRB-

and

the

corresponding

interference

set

of

a

reference

Definition

The

cache

position

of

a

set

corresponds

to

the

first

element

of

this

set

i.e.

if

a

set

is

defined

on

loop

l

its

cache

position

is

defined

by

OE

-LRB-

set

-LRB-

R

-RRB-

-RRB-

-LRB-

rljl

-RRB-

mod

CS

Lemma

The

relative

cache

position

of

the

interference

set

of

with

respect

to

the

reuse

set

of

R

is

equal

to

OE

-LRB-

AIS

-LRB-

-RRB-

-RRB-

OE

-LRB-

ARS

-LRB-

R

-RRB-

-RRB-

Example

Consider

loop

a

The

reuse

of

F

-LRB-

-RRB-

occurs

on

loop

So

the

corresponding

interference

set

of

G

-LRB-

-RRB-

is

defined

on

loop

The

position

of

the

actual

reuse

set

of

F

-LRB-

-RRB-

is

equal

to

OE

-LRB-

ARS

-LRB-

F

-LRB-

-RRB-

-RRB-

-RRB-

-LRB-

-RRB-

mod

CS

and

the

position

of

the

actual

interference

set

of

G

-LRB-

-RRB-

is

equal

to

OE

-LRB-

AIS

-LRB-

G

-LRB-

-RRB-

-RRB-

-RRB-

-LRB-

-RRB-

mod

CS

-LRB-

where

M

is

the

leading

dimension

of

array

G

-RRB-

The

relative

position

of

the

two

sets

is

then

-LRB-

-RRB-

mod

CS

-LRB-

-RRB-

mod

CS

Internal

Cross-Interferences

Definition

If

ffi

is

independent

of

jl

the

cross-interferences

between

R

and

are

called

internal

cross-interferences

Proposition

If

the

cross-interferences

between

R

and

are

internal

cross-interferences

the

size

of

the

intersection

-LRB-

in

cache

lines

-RRB-

denoted

CL

-LRB-

ARS

-LRB-

R

-RRB-

AIS

-LRB-

-RRB-

ffi

-RRB-

between

ARS

-LRB-

R

-RRB-

and

AIS

-LRB-

-RRB-

is

constant

over

the

iterations

of

loops

jl

The

number

of

cache

misses

due

to

such

crossinterferences

is

then

equal

to

Nl

CL

-LRB-

ARS

-LRB-

R

-RRB-

AIS

-LRB-

-RRB-

ffi

-RRB-

Proposition

The

intersection

between

two

intervals

of

size

and

separated

by

a

distance

of

ffi

jOE

-LRB-

-RRB-

OE

-LRB-

-RRB-

j

cache

locations

is

equal

CL

-LRB-

ffi

-RRB-

min

-LRB-

-LRB-

ffi

-RRB-

-RRB-

min

-LRB-

-LRB-

ffi

CS

-RRB-

-RRB-

LS

For

instance

kARS

-LRB-

R

-RRB-

k

and

kAIS

-LRB-

-RRB-

k

If

the

sets

correspond

to

a

collection

of

intervals

-LRB-

instead

of

one

single

interval

-RRB-

each

interval

of

the

reuse

set

is

compared

with

each

interval

of

the

interference

set

The

size

of

the

intersection

is

obtained

by

summing

over

all

these

subcases

External

Cross-Interferences

For

external

cross-interferences

an

approximate

or

an

accurate

evaluation

can

be

used

Due

to

paper

length

constraints

the

approximate

evaluation

is

shown

below

and

the

accurate

evaluation

is

presented

in

the

appendix

Approximate

Evaluation

The

approximate

evaluation

is

a

simple

probabilistic

reasoning

assuming

ffi

-LRB-

the

distance

between

the

reuse

set

of

R

and

the

interference

set

of

-RRB-

is

a

random

variable

with

a

uniform

distribution

Interferences

are

averaged

over

all

values

of

ffi

Hence

the

proposition

Proposition

The

approximate

number

of

cache

misses

due

to

external

cross-interferences

between

R

and

over

execution

of

the

loop

nest

is

equal

to

fa

-LRB-

ARS

-LRB-

R

-RRB-

AIS

-LRB-

-RRB-

-RRB-

Nl

CS

ffi

CS

X

ffi

CL

-LRB-

ARS

-LRB-

R

-RRB-

AIS

-LRB-

-RRB-

ffi

-RRB-

where

CL

-LRB-

kARS

-LRB-

R

-RRB-

k

kAIS

-LRB-

-RRB-

k

ffi

-RRB-

is

the

function

defined

in

section

and

ceiling

functions

have

generally

been

omitted

in

this

paper

when

computing

a

number

of

cache

lines

because

this

approximation

proved

to

have

very

little

influence

on

precision

Computation

of

the

Optimal

Block

Size

When

Cross-Interferences

Occur

Consider

loop

a.

Internal

cross-interferences

between

A

-LRB-

-RRB-

and

B

-LRB-

-RRB-

can

occur

as

well

as

external

cross-interferences

between

C

-LRB-

-RRB-

and

A

-LRB-

-RRB-

and

B

-LRB-

-RRB-

-LRB-

see

figure

b

-RRB-

Let

us

define

ffi

j

-LRB-

mod

CS

-RRB-

-LRB-

mod

CS

-RRB-

j

If

CS

then

capacity

misses

occur

for

both

A

-LRB-

-RRB-

and

B

-LRB-

-RRB-

Therefore

blocking

is

applied

on

loop

Theoretical

optimal

block

size

The

window

size

for

A

-LRB-

-RRB-

and

B

-LRB-

-RRB-

is

equal

to

B

The

theoretical

number

of

cache

misses

for

A

-LRB-

-RRB-

B

-LRB-

-RRB-

in

the

blocked

loop

is

equal

to

LS

The

number

of

cache

misses

for

C

-LRB-

-RRB-

is

equal

to

LS

Consequently

the

optimization

problem

is

the

following

LS

CS

find

B

that

minimizes

-LRB-

all

terms

are

independent

of

B

-RRB-

And

the

solution

is

any

value

within

h

LS

i

Let

us

consider

Bth

because

whenever

two

block

sizes

perform

identically

the

largest

possible

block

size

should

be

picked

to

decrease

the

loop

overhead

Estimated

optimal

block

size

Internal

cross-interferences

No

self-interference

occurs

and

let

us

ignore

for

the

moment

the

impact

of

external

cross-interferences

For

the

sake

of

simplicity

we

now

assume

ffi

CS

ffi

Since

ARS

-LRB-

A

-LRB-

-RRB-

-RRB-

ARS

-LRB-

B

-LRB-

-RRB-

-RRB-

B

CL

-LRB-

ARS

-LRB-

A

-LRB-

-RRB-

-RRB-

ARS

-LRB-

B

-LRB-

-RRB-

-RRB-

ffi

-RRB-

min

-LRB-

-LRB-

B?ffi

-RRB-

B

-RRB-

min

-LRB-

-LRB-

B

ffi?CS

-RRB-

B

-RRB-

LS

-LRB-

B?ffi

-RRB-

LS

since

B

CS

and

ffi

CS

ffi

So

the

new

optimization

problem

is

the

following

LS

B

Bth

find

B

that

minimizes

BLS

-LRB-

B

ffi

-RRB-

At

this

point

the

solution

would

be

Bint

est

ffi

External

cross-interferences

It

can

now

be

considered

that

B

Bint

est

i.e.

B

is

chosen

so

that

no

internal

cross-interference

occurs

Now

external

cross-interferences

can

also

influence

the

optimal

block

size

Consider

reference

A

-LRB-

-RRB-

its

actual

reuse

set

is

an

interval

of

B

cache

locations

and

the

corresponding

actual

interference

set

of

C

-LRB-

-RRB-

is

also

an

interval

of

B

cache

locations

Using

the

approximate

expression

of

section

the

number

of

external

cross-interferences

between

A

-LRB-

-RRB-

and

C

-LRB-

-RRB-

is

equal

to

CSLS

-LRB-

B

B

reuse

loop

-RRB-

The

result

is

the

same

for

B

-LRB-

-RRB-

Consequently

the

new

optimization

problem

is

the

following

LS

B

Bint

est

find

B

that

minimizes

CSLS

At

this

point

the

solution

would

be

Bcache

est

LS

TLB

misses

Now

let

us

consider

TLB

misses

A

-LRB-

-RRB-

B

-LRB-

-RRB-

are

unlikely

to

breed

TLB

misses

But

reference

C

-LRB-

-RRB-

corresponds

to

It

can

be

normalized

to

-LRB-

where

-RRB-

so

that

all

loop

indices

have

constant

boundaries

The

number

of

pages

crossed

over

loop

is

equal

to

nTLB

min

PS

According

to

section

if

nTLB

TS

TLB

misses

occur

Here

-LRB-

or

-RRB-

PS

and

TS

so

nTLB

TS

and

the

condition

is

fulfilled

Therefore

the

number

of

TLB

misses

is

equal

to

NTLB

nTLB

The

new

optimization

problem

is

the

following

LS

B

ffi

find

B

that

minimizes

CSLS

B

And

the

solution

is

Best

min

ffi

r?CSLS

min

PS

Here

r?CSLS

min

PS

when

min

PS

Note

that

using

ffi

for

loop

and

PS

for

loop

would

yield

the

same

performance

that

loops

and

can

not

be

interchanged

because

of

the

stride-one

access

of

reference

C

-LRB-

-RRB-

Figure

c

illustrates

the

necessity

to

precisely

evaluate

every

type

of

cache

misses

and

TLB

misses

Figure

d

shows

the

performance

variations

with

the

block

size

for

different

values

of

ffi

Using

blocking

to

reduce

interferences

When

no

capacity

miss

occurs

blocking

can

still

be

used

to

remove

internal

cross-interferences

between

A

-LRB-

-RRB-

and

B

-LRB-

-RRB-

The

optimization

problems

are

identical

except

that

Bth

As

shown

by

the

term

CSLS

blocking

also

reduces

external

crossinterferences

by

decreasing

the

reuse

distance

Figure

e

illustrates

the

impact

of

blocking

for

Self-Interferences

Assume

the

reuse

loop

level

is

l

for

reference

R.

Proposition

The

number

of

cache

misses

due

to

self-interferences

of

R

is

equal

to

Nl

-LRB-

kTRS

-LRB-

R

-RRB-

k

kARS

-LRB-

R

-RRB-

k

-RRB-

Intuitively

each

cache

line

excluded

from

the

actual

reuse

set

because

of

conflicts

generates

one

cache

miss

each

time

the

reuse

set

is

referenced

hence

the

proposition

Computation

of

the

Optimal

Block

Size

When

Self-Interferences

Occur

Consider

loop

a.

G

-LRB-

-RRB-

and

F

-LRB-

-RRB-

breed

most

memory

references

Main

interference

misses

are

either

cross-interferences

between

G

-LRB-

-RRB-

and

F

-LRB-

-RRB-

or

self-interferences

of

F

-LRB-

-RRB-

G

-LRB-

-RRB-

can

at

most

flush

B

of

the

elements

of

the

theoretical

reuse

set

of

F

-LRB-

-RRB-

while

selfinterferences

can

prevent

the

reuse

of

all

elements

-LRB-

see

figure

b

-RRB-

So

self-interferences

are

likely

to

be

the

main

cause

of

interference

misses

Theoretical

optimal

block

size

References

E

-LRB-

-RRB-

F

-LRB-

-RRB-

and

G

-LRB-

-RRB-

all

exhibit

temporal

reuse

According

to

-LSB-

-RSB-

the

window

size

of

these

references

is

respectively

and

B

The

number

of

misses

corresponding

to

each

reference

is

respectively

BLS

LS

and

BLS

The

optimization

problem

is

therefore

B

CS

find

B

that

minimizes

BLS

-LRB-

Terms

independent

of

B

are

ignored

-RRB-

And

the

solution

is

Bth

-LRB-

CS

-RRB-

pCS

Estimated

optimal

block

size

Self-interferences

Let

us

first

ignore

the

impact

of

external

cross-interferences

and

only

consider

self-interferences

The

first

goal

is

to

find

a

block

size

B

such

that

no

self-interference

occurs

The

problem

is

the

following

B

Bth

find

B

such

that

kTRS

-LRB-

F

-LRB-

-RRB-

-RRB-

k

kARS

-LRB-

F

-LRB-

-RRB-

-RRB-

k

According

to

section

if

fi

-LRB-

ns

S

oes

-RRB-

oes

then

the

intervals

of

size

S

in

one

area

of

size

oes

overlap

with

the

intervals

of

the

next

area

and

self-interferences

occur

So

the

first

condition

is

fi

-LRB-

ns

S

oes

-RRB-

oes

which

implies

S

oes

oes

oesoes

CS

The

second

condition

is

that

within

one

area

of

size

oes

two

consecutive

intervals

should

not

overlap

i.e.

S

oes

Since

here

S

B

these

conditions

are

equivalent

to

B

Bself

est

min

oes

oes

oesoes

CS

oes

For

instance

for

M

Bself

est

oes

The

actual

optimal

block

size

Bact

experimentally

found

is

Bact

and

the

performance

obtained

for

Bself

est

and

Bact

are

close

-LRB-

see

figure

c

as

explained

below

Bself

est

Best

in

this

example

-RRB-

C

Original

Loop

DO

DO

C

-LRB-

-RRB-

A

-LRB-

-RRB-

B

-LRB-

-RRB-

CONTINUE

C

Blocked

Loop

DO

B

DO

DO

min

-LRB-

-RRB-

C

-LRB-

-RRB-

A

-LRB-

-RRB-

B

-LRB-

-RRB-

CONTINUE

Cache

Reusable

Non-reusable

Reuse

Sets

Interference

Sets

C

-LRB-

-RRB-

ffi

B

-LRB-

-RRB-

A

-LRB-

-RRB-

Loop

a

Original

and

Blocked

Loop

Figure

b

Cache

layout

after

an

iteration

of

loop

60 80 100 120

140 160 180 200

Execution

Time

Number

of

References

-LRB-

in

seconds

-RRB-

B

N

N

N

N

N

60 80 100 120

140 160 180 200

Execution

Time

Number

of

References

-LRB-

in

seconds

-RRB-

B

N

N

N

N

Figure

c

Influence

of

optimal

block

size

computation

-LRB-

-RRB-

Bact

is

the

actual

optimal

block

size

Figure

d

Influence

of

block

size

-LRB-

-RRB-

a

logarithmic

scale

is

used

100 200 300 400

500 600 700 800

Execution

Time

Number

of

References

-LRB-

in

seconds

-RRB-

ffi

B

Bth

B

Best

B

Bact

-LRB-

Average

Memory

Access

Time

-RRB-

Original

base

addresses

Shifted

base

addresses

Figure

e

Using

blocking

to

reduce

interferences

-LRB-

-RRB-

Bact

is

the

actual

optimal

block

size

Figure

f

Ping-Pong

phenomena

in

subroutine

HYD

of

code

ADM.

External

cross-interferences

It

is

now

considered

that

no

self-interference

occurs

i.e.

B

Bself

est

External

cross-interferences

do

not

influence

the

number

of

misses

of

E

-LRB-

already

maximum

-RRB-

However

according

to

the

approximate

evaluation

of

section

F

induces

misses

on

G

-LRB-

B

B

reuse

loop

-RRB-

and

G

induces

-LRB-

B

-RRB-

CSLS

misses

on

F

-LRB-

B

reuse

loop

-RRB-

The

new

optimization

problem

is

the

following

B

Bself

est

find

B

that

minimizes

BLS

CSLS

And

at

this

point

the

solution

is

Bcache

est

min

oes

oes

oesoes

CS

oes

which

is

here

equivalent

to

Bcache

est

min

oes

oes

oesoes

CS

oes

i.e.

external

cross-interferences

have

a

negligible

impact

TLB

misses

The

linearized

addresses

for

E

F

G

are

respectively

-LRB-

with

B

B

-RRB-

ffl

On

loop

no

TLB

capacity

miss

can

occur

ffl

The

number

of

pages

crossed

over

loop

respectively

corresponding

to

E

F

G

is

equal

to

nTLB

min

NPS

B

min

BPS

B

min

B.

So

if

B

TS

nTLB

B

TS

and

the

number

of

TLB

misses

is

approximately

equal

to

NTLB

nTLB

N

NB

NB

B

ffl

Assume

now

B

TS

On

loop

nTLB

min

NPS

N

min

NPS

N

min

NPS

N

Since

here

N

nTLB

TS

So

NTLB

nTLB

NB

NB

The

optimization

problem

is

now

the

following

B

Bcache

est

find

B

that

minimizes

BLS

CSLS

-LRB-

if

B

TS

B

if

B

TS

And

the

solution

is

Best

min

oes

oes

oesoes

CS

oes

TS

Extension

to

Spatial

Reuse

The

techniques

shown

in

previous

sections

can

be

extended

to

spatial

reuse

Note

that

for

spatial

reuse

at

most

LS

reuses

can

be

achieved

per

cache

line

Examples

of

spatial

interferences

are

presented

below

Using

blocking

to

reduce

spatial

external

cross-interferences

Consider

loop

e.

Both

B

-LRB-

-RRB-

and

A

-LRB-

-RRB-

do

not

exhibit

temporal

reuse

Both

exhibit

spatial

reuse

but

the

spatial

reuse

distance

for

B

-LRB-

-RRB-

is

equal

to

iteration

of

while

the

spatial

reuse

distance

for

A

-LRB-

-RRB-

is

equal

to

iterations

of

Consequently

blocking

on

can

reduce

the

spatial

reuse

distance

for

A

-LRB-

-RRB-

without

increasing

that

of

B

-LRB-

-RRB-

It

can

be

assumed

that

no

spatial

cross-interference

occurs

for

B

-LRB-

-RRB-

For

A

-LRB-

-RRB-

the

reuse

set

is

a

set

of

cache

lines

The

interference

set

of

B

-LRB-

-RRB-

corresponding

to

this

reuse

set

is

an

interval

of

cache

locations

As

suggested

in

each

cache

line

of

the

reuse

set

is

considered

separately

Each

cache

line

can

be

reused

LS

times

Using

the

approximate

evaluation

of

section

-LRB-

LS

reuse

loop

and

LS

reuses

-RRB-

the

number

of

spatial

external

cross-interference

misses

on

A

-LRB-

-RRB-

due

to

B

-LRB-

-RRB-

is

equal

to

-LRB-

LS

-RRB-

CSLS

The

number

of

misses

due

to

B

-LRB-

-RRB-

number

of

misses

is

not

symmetrical

because

the

reuse

does

not

occur

on

the

same

loop

level

for

F

and

G.

C

Original

Loop

DO

DO

reg

E

-LRB-

-RRB-

DO

G

-LRB-

-RRB-

reg

F

-LRB-

-RRB-

CONTINUE

C

Blocked

Loop

DO

B

DO

B

DO

DO

min

-LRB-

-RRB-

reg

E

-LRB-

-RRB-

DO

min

-LRB-

-RRB-

G

-LRB-

-RRB-

reg

F

-LRB-

-RRB-

CONTINUE

G

-LRB-

-RRB-

E

-LRB-

-RRB-

F

-LRB-

-RRB-

Cache

Reusable

Non-reusable

Reuse

Sets

Interference

Sets

oes

M

N

oes

Loop

a

Original

and

Blocked

Loop

Figure

b

Cache

layout

after

an

iteration

of

loop

60 80 100 120

140 160 180 200

Execution

Time

Number

of

References

-LRB-

in

seconds

-RRB-

B

N

N

N

N

N

60 80 100 120

140 160 180 200

Execution

Time

Number

of

References

-LRB-

in

seconds

-RRB-

B

N

N

N

N

Figure

c

Influence

of

optimal

block

size

computation

Bact

is

the

actual

optimal

block

size

Figure

d

Influence

of

block

size

-LRB-

N

Best

N

Best

N

Best

N

Best

-RRB-

C

Original

Loop

DO

DO

B

-LRB-

-RRB-

A

-LRB-

-RRB-

CONTINUE

C

Blocked

Loop

DO

B

DO

B

DO

min

-LRB-

-RRB-

DO

min

-LRB-

-RRB-

B

-LRB-

-RRB-

A

-LRB-

-RRB-

CONTINUE

60 80 100 120

140 160 180 200

Execution

Time

Number

of

References

-LRB-

in

seconds

-RRB-

B

N

N

N

N

N

Loop

e

Original

and

Blocked

Loop

Figure

f

Using

blocking

to

reduce

spatial

interferences

and

TLB

misses

in

loop

e.

is

equal

to

LS

The

optimization

problem

is

then

the

following

LS

B

CS

find

B

that

minimizes

-LRB-

LS

-RRB-

CSLS

And

the

solution

is

Bcache

est

LS

Let

us

now

consider

TLB

misses

The

number

of

pages

crossed

over

loop

is

equal

to

nTLB

min

BN

PS

NB

min

BN

PS

NB

NB

-LRB-

in

the

examples

N

-RRB-

So

if

B

NTS

the

number

of

TLB

misses

is

equal

to

NTLB

The

new

optimization

problem

is

the

following

LS

B

CS

find

B

that

minimizes

-LRB-

LS

-RRB-

CSLS

And

the

solution

is

Best

The

precision

of

the

optimal

block

size

evaluation

is

illustrated

in

figure

f.

Note

that

blocking

can

not

only

reduce

cache

interference

misses

but

also

TLB

capacity

misses

Ping-Pong

Ping-Pong

corresponds

to

internal

cross-interferences

where

ffi

LS

Normally

the

probability

that

two

data

map

to

the

same

cache

line

is

equal

to

LS

CS

So

such

phenomena

have

a

very

low

probability

to

occur

but

they

might

be

very

difficult

to

trace

back

in

a

program

Therefore

they

should

be

detected

at

compile-time

or

run-time

For

instance

in

subroutine

HYD

-LRB-

and

other

subroutines

-RRB-

of

the

Perfect

-LSB-

-RSB-

benchmark

ADM

ping-pong

phenomena

were

detected

A

decrease

of

the

average

memory

access

time

was

obtained

by

slightly

varying

the

base

addresses

of

arrays

P

and

PS

in

HYD

-LRB-

see

figure

f

-RRB-

Spatial

self-interferences

can

also

frequently

occur

because

of

the

widespread

use

of

as

a

matrix

dimension

This

is

especially

critical

for

blocked

algorithms

where

the

matrix

dimension

can

become

the

block

stride

-LRB-

i.e.

the

distance

between

two

blocks

-RRB-

Further

Issues

in

the

Evaluation

of

Cache

Interferences

Selecting

the

proper

dependence

Assume

that

in

loop

a

reference

B

-LRB-

-RRB-

is

replaced

with

reference

A

-LRB-

-RRB-

There

is

a

selfdependence

for

A

-LRB-

-RRB-

and

a

group-dependence

from

A

-LRB-

-RRB-

to

A

-LRB-

-RRB-

The

reuse

distance

of

the

group-dependence

-LRB-

iteration

of

-RRB-

is

much

shorter

than

that

of

the

self-dependence

-LRB-

iterations

of

-RRB-

So

for

most

elements

-LRB-

excluding

A

-LRB-

-RRB-

-RRB-

reference

A

-LRB-

-RRB-

exploits

the

group-dependence

reuse

rather

than

the

self-dependence

Therefore

interferences

on

the

group-dependence

instead

of

the

selfdependence

must

be

considered

Interferences

are

evaluated

for

the

dependence

corresponding

to

the

smallest

dependence

distance

In

general

array

subscripts

-LRB-

i.e.

dependences

-RRB-

are

simple

enough

to

make

the

selection

of

the

proper

dependence

a

tractable

task

Cumulating

interference

sets

It

is

not

possible

to

simply

add

the

number

of

cache

misses

corresponding

to

each

reference

and

each

type

of

interferences

because

some

interferences

can

be

redundant

Assume

that

in

loop

a

reference

B

-LRB-

-RRB-

is

replaced

with

reference

D

-LRB-

-RRB-

C

-LRB-

-RRB-

and

D

-LRB-

-RRB-

can

both

induce

external

cross-interferences

on

A

-LRB-

-RRB-

The

relative

cache

distance

between

the

actual

interference

sets

of

these

two

references

is

equal

to

ffiCD

j

-LRB-

mod

CS

-RRB-

-LRB-

mod

CS

-RRB-

j

If

recurs

on

each

run

because

the

arrays

are

generally

assigned

consecutively

in

memory

so

that

their

relative

base

address

remains

constant

observations

were

obtained

with

a

cache

simulator

for

an

direct-mapped

cache

with

a

cache

line

size

the

same

parameters

as

the

DEC

Alpha

-LSB-

-RSB-

MIPS

-LSB-

-RSB-

ffiCD

i.e.

the

two

actual

interference

sets

overlap

then

when

they

intersect

with

the

actual

reuse

set

of

A

C

and

D

have

a

redundant

impact

on

A

and

cache

misses

should

not

be

counted

twice

In

order

to

avoid

such

redundancy

interference

sets

should

not

considered

individually

Definition

Two

references

R

and

belong

to

the

same

translation

group

if

the

relative

cache

distance

between

R

and

is

constant

The

actual

interference

set

of

a

translation

group

is

the

union

of

the

actual

interference

sets

of

all

references

within

this

translation

group

Note

that

in

general

there

are

few

translation

groups

within

a

loop

body

Redundancies

Globally

most

redundancies

are

avoided

for

the

following

reasons

First

determining

cross-interferences

on

the

actual

reuse

set

instead

of

the

theoretical

reuse

set

avoids

redundant

evaluation

between

selfinterferences

and

cross-interferences

Second

determining

internal

cross-interferences

before

evaluating

external

cross-interferences

and

then

updating

the

actual

reuse

set

avoids

redundant

evaluation

between

internal

cross-interferences

and

external

cross-interferences

Third

redundancies

within

external

cross-interferences

are

ignored

because

they

proved

to

be

negligible

in

most

cases

is

possible

though

to

come

up

with

examples

where

redundancies

between

external

cross-interferences

are

significant

Global

Algorithm

The

global

algorithm

used

to

compute

the

optimal

block

size

in

a

loop

is

shown

on

the

figure

on

the

right

An

optimal

algorithm

would

be

to

simply

compute

all

expressions

of

the

different

types

of

cache

misses

and

TLB

misses

and

then

solve

the

optimization

problem

However

the

optimization

problem

is

then

too

complex

to

solve

So

we

propose

the

heuristic

which

simplifies

the

problem

by

eliminating

self-interferences

and

internal

cross-interferences

However

it

can

then

be

argued

that

the

estimate

of

the

optimal

block

size

can

sometimes

be

an

underestimate

Two

reasons

decreases

the

importance

of

this

point

First

it

appeared

that

the

smallest

block

size

B

LS

often

brings

good

performance

-LRB-

often

better

than

Bth

-RRB-

Second

with

our

algorithm

the

higher

the

number

of

references

the

higher

the

probability

the

optimal

block

size

is

small

on

the

other

hand

the

higher

the

number

of

references

the

higher

the

number

of

external

cross-interferences

and

small

block

sizes

actually

help

reducing

external

cross-interferences

by

decreasing

the

reuse

distance

For

each

reference

R

in

the

loop

body

Determine

the

shortest

dependence

distance

Determine

the

interference

set

and

actual

interference

set

cache

misses

due

to

Compute

the

number

of

internal

cross-interferences

For

the

translation

group

of

reference

R

Determine

the

interference

set

and

actual

interference

set

Best

is

the

estimated

Determine

the

reuse

set

theoretical

optimization

problem

Compute

the

number

of

LS

B

Bth

Compute

the

number

of

self-interferences

cache

misses

due

to

and

actual

reuse

set

Bself

est

-LRB-

R

-RRB-

maximal

block

size

without

self-interferences

Update

the

actual

reuse

set

of

R

by

removing

elements

victim

of

cross-interferences

Find

Bth

the

solution

of

the

cache

capacity

misses

Ncapacity

without

internal

cross-interferences

Bcross

int

-LRB-

R

-RRB-

maximal

block

size

LS

B

min

-LRB-

Bth

Bself

est

-LRB-

R

-RRB-

-RRB-

For

each

other

translation

group

Compute

the

number

of

TLB

capacity

misses

NTLB

LS

B

min

-LRB-

Bth

Bext

est

Bint

est

-RRB-

Compute

the

total

number

of

Compute

the

total

number

of

external

cross-interferences

Compute

the

number

of

cache

misses

due

to

external

cross-interference

misses

for

reference

R

Next

interf

-LRB-

R

-RRB-

external

cross-interferences

Next

interf

P

R

Next

interf

-LRB-

R

-RRB-

Ncapacity

Next

interf

NTLB

Find

Best

that

minimizes

optimal

block

size

Global

Algorithm

Conclusions

and

Future

Directions

Optimal

block

size

computation

as

performed

up

to

now

often

yields

poor

performance

and

therefore

restricts

the

use

of

blocking

techniques

in

production

compilers

In

order

to

obtain

near-optimal

performance

accurate

evaluation

of

the

optimal

block

size

has

been

shown

to

be

compulsory

The

influence

of

other

components

of

the

memory

hierarchy

such

as

the

TLB

can

not

be

ignored

either

Reciprocally

it

was

shown

that

interferences

can

severely

affect

the

performance

of

a

loop

even

when

all

data

to

be

reused

fit

in

cache

and

that

blocking

can

then

be

used

to

reduce

the

number

of

interferences

and

significantly

improve

performance

by

decreasing

the

reuse

distance

Several

future

directions

of

research

are

being

contemplated

simultaneously

First

the

integration

in

a

compiler

of

the

techniques

for

evaluating

interference

misses

and

computing

the

optimal

block

size

Second

the

extension

of

these

techniques

to

set-associative

caches

Third

the

study

of

non-square

our

model

a

cache

interference

occurs

when

data

map

to

the

same

cache

location

In

an

n-way

associative

cache

a

cache

interference

occurs

when

n

data

map

to

the

same

cache

set

Therefore

considering

associativity

is

merely

a

blocks

i.e.

where

the

block

size

is

different

for

each

loop

Fourth

the

techniques

used

for

evaluating

interferences

can

also

be

used

for

driving

a

selective

copying

strategy

by

evaluating

the

cost/benefit

of

copying

Copying

raises

complex

coherence

issues

so

it

must

be

used

with

great

care

However

in

some

cases

even

the

actual

optimal

block

size

yields

poor

loop

performance

Developing

a

strategy

for

combining

optimal

blocking

and

data

copying

would

yield

sustained

performance

for

a

wide

range

of

loop

nests

matter

of

changing

the

condition

under

which

interferences

occur

and

should

not

raise

much

difficulties

On

the

other

hand

the

replacement

algorithm

might

introduce

some

complexity

this

remains

to

be

investigated

instance

if

gcd

-LRB-

N

CS

-RRB-

in

loop

a

self-interferences

are

total

for

any

block

size

value

Bibliography

-LSB-

-RSB-

John

L.

Hennessy

and

David

A.

Patterson

Computer

Architecture

A

Quantitative

Approach

Morgan

Kaufmann

Publishers

Inc.

-LSB-

-RSB-

Francois

Bodin

Christine

Eisenbeis

William

Jalby

and

Daniel

Windheiser

A

Quantitative

Algorithm

for

Data

Locality

Optimization

In

Code

Generation-Concepts

Tools

Techniques

SpringerVerlag

-LSB-

-RSB-

Jeanne

Ferrante

Vivek

Sarkar

and

Wendy

Thrash

On

Estimating

and

Enhancing

Cache

Effectiveness

-LRB-

Extended

Abstract

-RRB-

In

Proceedings

of

the

Fourth

Workshop

on

Languages

and

Compilers

for

Parallel

Computing

-LSB-

-RSB-

Kathryn

S.

McKinley

Automatic

and

Interactive

Parallelization

PhD

thesis

Rice

University

Technical

Report

April

-LSB-

-RSB-

Michael

Wolf

and

Monica

Lam

A

Data

Locality

Optimizing

Algorithm

In

Proceedings

of

the

ACM

SIGPLAN

Conference

on

Programming

Language

Design

and

Implementation

volume

-LRB-

-RRB-

pages

-LCB-

June

-LSB-

-RSB-

W.

Li

and

K.

Pingali

Access

Normalization

Loop

Restructuring

for

NUMA

Compilers

In

International

Conference

on

Architectural

Support

for

Programming

Languages

and

Operating

Systems

pages

-LCB-

September

-LSB-

-RSB-

Monica

Lam

Edward

E.

Rothberg

and

Michael

E.

Wolf

The

Cache

Performance

of

Blocked

Algorithms

In

Fourth

International

Conference

on

Architectural

Support

for

Programming

Languages

and

Operating

Systems

April

-LSB-

-RSB-

She

Zhiyu

Zhiyuaun

Li

and

Pen-Chung

Yew

An

Empirical

Study

on

Array

Subscripts

and

Data

Dependendencies

Technical

Report

Center

for

Supercomputing

Research

and

Development

University

of

Illinois

at

Urbana-Champaign

August

-LSB-

-RSB-

O.

Patashnik

R.

L.

Graham

D.

E.

Knuth

Concrete

mathematics

Addison-Wesley

-LSB-

-RSB-

George

Cybenko

Lyle

Kipp

Lynn

Pointer

and

David

Kuck

Supercomputing

Performance

Evaluation

and

the

Perfect

Benchmarks

In

Supercomputing

pages

-LCB-

-LSB-

-RSB-

Digital

Equipment

Corporation

Alpha

Architecture

Handbook

-LSB-

-RSB-

J.

Heinrich

G.

Kane

MIPS

RISC

Architecture

Prentice-Hall

blocking
cache interferences
data locality
optimal block size


Chapter 18 Tracking Statistics in Communication Networks Alexandru Murgu , Risto Lahdelma University of Jyv?askyl?a Department of Mathematics P. O. Box 35 FIN-40351 Jyv?askyl?a , Finland E-mail : murgu@julia.math.jyu.fi, E-mail : rlahdelm@tarzan.math.jyu.fi Abstract . In this paper we consider the problem of tracking statistical flow patterns of multivariate type in a communication network . The flow control architectures are usually based on one step ahead prediction mechanisms as with an adaptive Markov chain control . We consider a long run approach for tracking the statistics of the flow patterns using a genetic algorithm . The necessity for a genetic algorithm approach arises when working with sufficient statistics for controlling the multivariate flow streams , because of continuously growing of the data dimensionality during the estimation phase . The performance of the genetic algorithm in handling multivariate statistics is analyzed through numerical experiments with Markovian flow patterns . 18.1 Introduction The behaviour of the ATM network dynamics in the presence of unknown traffic patterns and the proper ways of handling traffic in order to obtain more reliable and predictable performances is an important aspect when new communication The complete title of this article : Tracking Statistics in Communication Traffic Control Using a Genetic Algorithm . 241 242 Proceedings of the 1NWGA , Vaasa , Finland , 9 - 12 January 1995 ATM SwitchInput Buffers Output Buffers 1 2 n-1 n 1 2 m-1 m Fig. 1 . The general architecture of an ATM switching node . : : services are continually introduced after network design and installation . In this respect , the main difficulty is that the uncertainties about the traffic patterns and the time varying nature of network conditions are always present in the broadband ATM networks . An additional complication comes from the diverse mixture of the traffic types and the quality of service constraints -LSB- -RSB- . Here , the real network can be a practical ATM network , such as a switching node , an endto-end connection , or a simple queue -LSB- -RSB- . To effectively use network resources . i.e. , the link and buffer capacities , the traffic control in the ATM networks must be able to adapt continuously to the dynamic behaviour of traffic loads and network conditions during the network operation , that is , link failures and buffer overflows . The general problem of the adaptive traffic control in the communication networks consists of adaptively regulating the access of the external traffic into the network in order to guarantee a desired level of performance , given in the form of a quality index . As the underlying dynamics of the real network are unknown beforehand , it is necessary to find an accurate statistical description of the incoming flow patterns . In order to detect the various sources of traffic which have different generation rates of the cells -LRB- or bits or packets -RRB- , we are required to find a way of distinguishing among the possible traffic patterns and to use this information to improve the control performance index . For this , a multivariate statistics model for the flow patterns is considered , leading to a control problem which can be mapped onto a genetic algorithm as an optimization problem associated to the parameter estimation scheme . 18.2 The tracking problem The schematic representation of the traffic controller for an ATM switching node is depicted in fig. 1 . An approach giving a continuous time description of the dynamic behaviour is to consider the data traffic as a microscopic flow and by Murgu et al : Tracking Statistics in Communication Networks . 243 continuity arguments to set up the time dependent cell flows passing through a switching node . For an incoming link i and an outgoing link j , the data flow can be described by the following stochastic differential equation : dxij -LRB- t -RRB- = -LRB- qin , ij -LRB- t -RRB- ? qout , ij -LRB- t -RRB- -RRB- dt + oe -LRB- x ; t -RRB- dwij -LRB- 18:1 -RRB- where qin ; ij -LRB- t -RRB- stands for the number of cells to across the link ij , qout ; ij represents the state dependent outflow from this link , while wij is a Wiener process with zero mean and unit variance modelling the uncertainties about the external traffic pattern and the asynchronous time operation of the switching node . The flow arriving to link ij can be given in terms of the flows leaving the network links directed to node i and the external flow arriving at node i : qin ; ij -LRB- t -RRB- = ij -LRB- t -RRB- -LSB- OEi -LRB- t -RRB- ri -LRB- t -RRB- + X I -LRB- i -RRB- CkiGki -LRB- xki -LRB- t -RRB- -RRB- -RSB- -LRB- 18:2 -RRB- subject to < = OEi -LRB- t -RRB- < = 1 -LRB- 18:3 -RRB- X E -LRB- i -RRB- ik = 1 -LRB- 18:4 -RRB- < = ik < = 1 -LRB- 18:5 -RRB- where I -LRB- i -RRB- denotes the set of nodes with links directed to node i , while E -LRB- i -RRB- is the set of nodes with links originating from node i. Here , ri -LRB- t -RRB- is the mean flow from outside of the network to enter at node i , OEi -LRB- t -RRB- is the input rate control variable which gives the portion of ri -LRB- t -RRB- that is accepted at node i and ij -LRB- t -RRB- is the routing variable which gives the amount of traffic flow at node i that is routed to the adjacent node j through the link ij . The function Gij -LRB- t -RRB- represents the link utilization factor and a simple model for this factor -LSB- 328 -RSB- is given by : Gij -LRB- xij -LRB- t -RRB- -RRB- = xij -LRB- t -RRB- xij -LRB- t -RRB- + lij -LRB- 18:6 -RRB- where lij is the normalized link capacity . Let us denote the number of nodes in the network by n , the number of outgoing links by m and we consider the number of the cells in the input buffers -LRB- fig. 1 -RRB- as representing the state variables , x 2 < n . The state space formulation of the network flow control dynamics in the switching node is described by the following equation dx -LRB- t -RRB- = f -LRB- x -LRB- t -RRB- ; -LRB- t -RRB- ; OE -LRB- t -RRB- -RRB- dt + oe -LRB- x ; t -RRB- dw -LRB- 18:7 -RRB- x -LRB- 0 -RRB- = x0 -LRB- 18:8 -RRB- 244 Proceedings of the 1NWGA , Vaasa , Finland , 9 - 12 January 1995 subject to A -LRB- t -RRB- = 1 -LRB- 18:9 -RRB- < = OE -LRB- t -RRB- < = 1 -LRB- 18:10 -RRB- < = -LRB- t -RRB- < = 1 -LRB- 18:11 -RRB- where 1 is a vector of suitable dimension with all components equal to 1 and A is a rectangular matrix of dimension -LRB- n ? 1 -RRB- ? m -LRB- the links to nodes incidence matrix -RRB- , whose entries are defined as aij = -LRB- 1 if j 2 E -LRB- i -RRB- ; otherwise : -LRB- 18:12 -RRB- The matrix A is used to describe the network topology . The control problem is to choose appropriate switched routes and incoming flow rates such that , min -LRB- t -RRB- ; OE -LRB- t -RRB- J1 = Exf Z T -LRB- m X i = 1 xi -LRB- t -RRB- ? nX i = 1 iOEi -LRB- t -RRB- ri -LRB- t -RRB- -RRB- dtg -LRB- 18:13 -RRB- subject -LRB- 7 -RRB- - -LRB- 12 -RRB- . The first term in the cost functional -LRB- 13 -RRB- is the total time spent by all cells admitted in the buffering system , whereas the second term gives the total amount of the demand flow admitted into the switching node during the considered time horizon T . The parameters i ; i = 1 ; : : : ; n are the weighting factors which should be chosen so as to reflect a compromise between the average delay and the total admitted demand flow . The problem of tracking statistics associated with the discrete time version of the previously defined stochastic control problem is to minimize the following quadratic performance index J2 -LRB- bxk -RRB- = 1 2 NX k = 1 -LRB- zk ? Hkbxk -RRB- TR ? 1 k -LRB- zk ? Hk bxk -RRB- -LRB- 18:14 -RRB- where , zk = Hkx + wk -LRB- 18:15 -RRB- is the linear measurement equation of the switch state at discrete time epochs , zk is a M-dimensional composite vector containing the cummulated observations . Here , Hk is the matrix of parameters in the observation equation and depends on the current sampling step , k , for a finite time horizon of N samples . The weighting matrix R ? 1 k of dimension M?M is assumed to be positive definite and symmetric and is related to the statistics of the observed process if the measurement noise is described by wk . Murgu et al : Tracking Statistics in Communication Networks . 245 18.2.1 Sufficient statistics for flow control To estimate the n-dimensional vector bxk , we consider a least squares estimation procedure of the form bxk +1 = bxk + Kk +1 -LRB- zk +1 ? Hk +1 bxk -RRB- ; k = 1 ; : : : ; N ? 1 -LRB- 18:16 -RRB- where Kk +1 = PkHTk +1 -LRB- Rk +1 + Hk +1 PkHTk +1 -RRB- ? 1Hk +1 Pk -LRB- 18:17 -RRB- Pk +1 = Pk ? PkHTk +1 -LRB- Rk +1 + Hk +1 PkHTk +1 -RRB- ? 1Hk +1 Pk -LRB- 18:18 -RRB- The matrix Kk +1 is updated for each time step of the sampling procedure in order to adapt de minimization of cost functional J2 to the dynamics of the stochastic process modelling the input flow patterns . Working with the performance index in -LRB- 14 -RRB- , although there is a stochastic structure of the control problem , the least squares estimator is in fact an optimal Bayesian estimator . The sequential estimation of the network state is intended to track the average of the flow pattern in a nonstationary environment . However , for the real time operation of the sequential estimation procedure , the long-run mean of the flow pattern is assumed to be constant -LRB- see figs. 3-14 -RRB- . Theorem 1 . In the least squares estimation , the weighting matrix R is the variance matrix for the measurement noise and the least squares and linear minimum variance estimators are identical . Proof . For convenience , let us delete the subscript k and the italicized notation from the equations for the least squares formulation . Then , the observation equation can be simply stated as z = Hx + w -LRB- 18:19 -RRB- The least squares estimate of x is given by bx = -LRB- HTR ? 1H -RRB- ? 1HTR ? 1z -LRB- 18:20 -RRB- where R ? 1 is the weighting matrix appearing in the quadratic cost function -LRB- 14 -RRB- . To emphasize the linear structure of the estimator and to simplify the presentation , we can write bx as bx = Kz -LRB- 18:21 -RRB- where K is obviously defined by K = -LRB- HTR ? 1H -RRB- ? 1HTR ? 1 -LRB- 18:22 -RRB- 246 Proceedings of the 1NWGA , Vaasa , Finland , 9 - 12 January 1995 or in terms of P , as K = PHTR ? 1 -LRB- 18:23 -RRB- where P is defined by P = -LRB- HTR ? 1H -RRB- ? 1 -LRB- 18:24 -RRB- We can easily see that KH = I , or in other words , this is a simple statement of the unbiased nature of the least squares estimator , assuming that w is a zero mean process . Indeed , in order to check this assertion , we need to compute the expected value of the estimation error ex , ex = x ? bx = x ? Kz -LRB- 18:25 -RRB- Using z from -LRB- 19 -RRB- , we have ex = x ? K -LRB- Hx + w -RRB- = Kw -LRB- 18:26 -RRB- where we have used the relation KH = I . The expected value of ex is given by Efexg = KEfwg = -LRB- 18:27 -RRB- The variance of the least squares estimation error is given by Vex = EfexexTg = KEfwwTgKT = KVwKT -LRB- 18:28 -RRB- Substituting from the equation -LRB- 22 -RRB- for K , we obtain Vex = -LRB- HTR ? 1H -RRB- ? 1HTR ? 1VwR ? 1H -LRB- HTR ? 1H -RRB- ? 1 -LRB- 18:29 -RRB- The linear minimum variance estimate of x based on the observation equation -LRB- 19 -RRB- is given by exLMV = -LRB- HTV ? 1 w H -RRB- ? 1HTV ? 1 w z -LRB- 18:30 -RRB- and is also unbiased . The variance of the estimation error for this case is given by VexLMV = EfexLMV exTLMV g = -LRB- HTV ? 1 w H -RRB- ? 1 -LRB- 18:31 -RRB- Therefore , by letting R = Vw , the variance of the least squares estimator becomes Vex = -LRB- HTV ? 1 w H -RRB- ? 1HTV ? 1 w VwV ? 1 w H -LRB- HTV ? 1 w H -RRB- ? 1 = -LRB- HTV ? 1 w H -RRB- ? 1 = VexLMV -LRB- 18:32 -RRB- and the proof is complete . 2 Murgu et al : Tracking Statistics in Communication Networks . 247 ATM Switching Node GA Parameter Performance Multivariate environment Parameter estimation Reference measure J Estimation Optimizer Routing/Flow Controller FlowRouting Tracking Fig. 2 . Using Genetic Algorithm for tracking the flow patterns . Remark 1 . Theorem 1 shows that the least squares estimate for the stochastic control problem under study represents the sufficient statistics which are needed for building an adaptive controller , where the tracking parameter estimates are used as references in control synthesis -LRB- see fig. 2 -RRB- . The least squares parameter estimation problem is as a large scale optimization problem and is solved based on a GA strategy , as is presented in the next section . 18.2.2 The optimization problem For the estimation parameter phase , we have to solve the quadratic optimization problem as described by the equation -LRB- 14 -RRB- . The point is that the optimization problem which has been obtained sampling horizon length are considered . 18.3 Using genetic algorithms 18.3.1 Variance analysis of multivariate statistical flows The distinctive feature of traffic control design is given by the multivariate nature of the flow patterns comming from different sources of information . Due to this fact , we consider that the dependent variable -LRB- in the control problem , the state x of the input buffers -RRB- is a multivariate normal vector having a distribution with the same covariance matrix for each population of cells . The assumption on the identical covariance matrix is motivated by the fact that in the ATM switching , the cells carrying the information have a fixed length , according to B - 248 Proceedings of the 1NWGA , Vaasa , Finland , 9 - 12 January 1995 ISDN regulations . When this assumption made , the unknown input flow patterns can be adequately modelled onto the framework of the multivariate statistics . A linear model for the multivariate analysis of the variance -LSB- 320 -RSB- is described by xki = m ? -LRB- mk ? m -RRB- + -LRB- xki ? mk -RRB- -LRB- 18:33 -RRB- where xki is the dependent vector variable for the ith flow stream in the kth sampling period , k = 1 ; : : : ; N , and N is the total number of sampling periods . Here , m is the vector of total sample means -LRB- grand centrod -RRB- , mk is the mean for the sample k . We usually are not interested in the grand centroid and it is convenient to substract it from both sides of the equation , casting the linear model in terms of individual flow streams deviations from the grand centroid . Letting exki = xki ? m -LRB- 18:34 -RRB- we can write exki = -LRB- mk ? m -RRB- + -LRB- xki ? mk -RRB- -LRB- 18:35 -RRB- In this partition for the deviation of the ith pattern stream in sample k from the grand centroid , the first term represent the hypothesis effect -LRB- differences in locations of samples -RRB- and the second term represents the error effect -LRB- deviations of flow streams from their sample centroids -RRB- . Summing over all flow streams in the total sample , the squares and cross-products of the elements of the two partitions yields the following result NX k = 1 M X i = 1 xkixTki = NX k = 1 M X i = 1 -LRB- mk?m -RRB- -LRB- mk?m -RRB- T + NX k = 1 M X i = 1 -LRB- xki?mk -RRB- -LRB- xki?mk -RRB- T -LRB- 18:36 -RRB- where M is the total number of the streams expected to be encountered . However , if this is not the case , then we can let M to dependent upon k , M = Mk . The matrix in the left hand side of -LRB- 37 -RRB- is the matrix of sums of squares and crossproducts of deviations of all flow streams from the grand centroid and it is called the total matrix , T , T = NX k = 1 M X i = 1 xkixTki -LRB- 18:37 -RRB- The first matrix in the right hand side is the matrix of weighted squares and cross-products of deviations of sample centriods from the grand centriod , called the \ among-samples '' matrix , A , A = NX k = 1 M X i = 1 -LRB- mk ? m -RRB- -LRB- mk ? m -RRB- T -LRB- 18:38 -RRB- Murgu et al : Tracking Statistics in Communication Networks . 249 The second partition term is the matrix of squares and cross-products of deviations of flow streams from their sample centriods , pooled over all samples , called the \ within-samples '' matrix , W , W = NX k = 1 M X i = 1 -LRB- xki ? mk -RRB- -LRB- xki ? mk -RRB- T -LRB- 18:39 -RRB- To test the discriminating power of the sampling procedure used for detecting and tracking the flow pattern statistics , we consider the Wilk 's determinant ratio test statistic defined as ? = jWj jTj -LRB- 18:40 -RRB- It is noteworthy that the matrix W can be converted to a correlation matrix which is the best estimator of common populations , that is , the correlation matrix of the flow samples . 18.3.2 Mapping multivariate statistics By the genetic algorithm approach , an attempt is made to solve the optimization problem adaptively by dynamically accumulating and structuring the information about the problem and using this information to generate an acceptable solution . This is so because , in practical operation of an ATM switching node , the relative occurrence frequency of some traffic patterns is not known until application time . The genetic algorithm solution is used in this context for the initial \ tuning '' of the traffic control mechanism and the controller runs without any other reference for the remaining interval of the actual sampling period . The operation of the ATM switching node is considered as a complex process to which we associated an adaptive feedback loop containing two devices , the GA based parameter estimation optimizer and the routing/flow controller which has to realize the overall adaptive behaviour -LRB- see fig. 2 -RRB- . We look for a good adaptive strategy which should be robust in the sense that it should respond quickly across the whole range of nonstationary environments to which the ATM switching might be subjected . The genetic algorithm approach is considered to be useful for this particular control application because other optimization techniques are local in their own nature , in the sense that they quickly find the nearest local optimum , but leave the global optimum of a multimodal optimization surface undiscovered . From an adaptive point of view , to apply random search as with GA approach is to ignore the information about the response surface which is accumulating during the period of adaptation . We assume that the response control surfaces are stationary over the period of adaptation . This hypothesis is not restrictive , since we can change the sampling horizon length to achieve the stationarity degree that we need . In the applications , this hypothesis is fulfilled taking the constant 250 Proceedings of the 1NWGA , Vaasa , Finland , 9 - 12 January 1995 long-run flow patterns over time . The on-line control performance requires that the adaptive strategy to be evaluated in terms of average performance of the controlled process over time . The on-line control system runs with the best routing/flow controls which are generated at the beginning of a sampling period . The initial population is always taken to be the best selection of the genetic evolution from the last sampling period . The selection probabilities are defined in such a way that the expected number of offspring produced by an individual is proportional to his associated performance value . Reproduction proportional to performance results is a sequence of generations A -LRB- t -RRB- in which the best individual in A -LRB- 0 -RRB- takes over a larger and larger proportion of the population . The timevarying selection probability distributions provide a highly dynamic sampling bias such that flow patterns whose sampled mean remains consistently above the average over a period of time will experience a rapid -LRB- exponential -RRB- increase in the number of trials allocated to them . As the low cost individuals begin to dominate A -LRB- t -RRB- , there is an effective reduction in the dimensionality of the search space and a corresponding reduction in the definition lengths of individuals providing for another cycle of nearly optimal sampling . 18.3.3 The genetic algorithm The genetic algorithm which has been used for solving the optimization problem is of Contractive Mapping type -LSB- 896 -RSB- , because we want to use the contractive mapping property to find out in few generations an approximate solution for the optimization problem . The convergence toward the global optimum can not be achieved within a limited number of generations , but this is not so drastic because the outcome of the genetic computation is used only as starting reference within the actual sampling interval . The best solutions obtained from the previous sampling period are kept in a trial list and are to be the candidate solutions for the next sampling period . By this approach , we taught that the dynamical evolution of the samples populations when the flow streams are smooth enough can be tracked , thus providing a quasi-prediction -LRB- or genetic material reusability -RRB- . The structure of the algorithm is the following procedure CM-GA begin t = initialize A -LRB- t -RRB- evaluate A -LRB- t -RRB- while -LRB- not termination condition -RRB- do begin contractive mapping F -LRB- A -LRB- t -RRB- -RRB- 7 ! A -LRB- t + 1 -RRB- t t + 1 select A -LRB- t -RRB- from A -LRB- t ? 1 -RRB- Murgu et al : Tracking Statistics in Communication Networks . 251 recombine A -LRB- t -RRB- evaluate A -LRB- t -RRB- if J2 -LRB- A -LRB- t ? 1 -RRB- -RRB- < = J2 -LRB- A -LRB- t -RRB- -RRB- then t t ? 1 end end The contractive mapping which was selected is given by F -LRB- x -RRB- = x 2 -LRB- 18:41 -RRB- 18.4 Experiments with Markovian flow streams To check the performances of the tracking mechanism , we have considered a 3 inputs - 3 output ATM switching node n = m = 3 , subject to input traffic patterns displayed in figs. 3-5 and 9-11 , respectively . In these figures , the long-run flow patterns -LRB- whose ranges are usually taken into account when the link capacities are designed -RRB- , are acting as reference levels on which we considered symmetric time dependent deviations , whithin the current sampling horizon . The long-run flows were selected to be the same for all the input streams and consequently , the link capacities were considered identical . This is so because we are not interested in study the nonlinear effects due to the limited capacity of the links , but in dynamical properties of the flow control mechanism . On the time dependent mean flow patterns we experienced two categories of perturbations . The first category is given by continuous time random processes with zero mean and unit variance . These perturbations are relevant when considering the adaptation performance of the traffic controller because is providing an environment where the real flow pattern is not known exactly , but has to be detected and followed by the control mechanism . Somehow , we relate this situation of a noise filtering capability and tracking the true mean of the traffic pattern . The second category of perturbations is represented by the discrete time random jumps around the mean flow patterns which are simulating the bursty traffic as it can occur various sources of information . For these traffic patterns , the mean traffic is piecewise constant over the time , but within the constant value interval , there are random jumps at discrete times . We consider that this pattern of traffic is the most chalenging for the routing mechanism which has been embedded into the control model of the ATM switching node . Additionally , the traffic burstiness as it appears at discrete time periods rises serious stability problems for the controller , because the true mean traffic pattern has to be detected and followed within the succesive shots , with a given bound of stability , acting here as a trust region . The time horizon length has be selected to be 16 conventional units and represent an integer multiple of the sampling period -LRB- specifically , 10 samples per unit time was the fixed a setting for all experiments -RRB- . The results in the continuous time 252 Proceedings of the 1NWGA , Vaasa , Finland , 9 - 12 January 1995 perturbation case -LRB- figs. 6-8 -RRB- show that the tracking performance of the controller has been achieved in a satisfactory extent . The Wilk 's determinant ratio test ? has been found to be less then 0.74 . Indeed , the multivariate statistics analysis presented in the third section , shows that the more close on the flow stream mean is the sample mean , the small is the determinant ratio test . In the discrete time perturbation case , the determinant ratio test has been found to be 0.92 . As it can be seen from figs. 12-14 , the tracking performance is hardly affected when the high value jumps occur , inspite of the fact that same time dependent mean flow patterns were considered . It can be seen that the smoothness of the traking is by far more rough than for the continuous time perturbation case . This is not surprisingly because it is quite hard to distinguish between the occurence of a small jump and a high jump . We mention that no threshold mechanism to discriminate between these two situations was setted , arguing that in a real life environment , we can not know a priori to what values are to be setted the thresholds . However , by using the selection mechanism of the best optimal solutions of the parameter estimation phase done by running the genetic algorithm , some kind of learning over the multiple horizonts is indeed possible . In fig. 15 , it can be seen the optimization performance of the genetic algorithm , which corresponds to the discrete time perturbation case . It really achieved a satisfactory solution to the optimization problem for parameter estimation in less than 10 generations -LRB- this was the maximum number allowed for computational progress -RRB- . The population size was selected to be 20 , the elitism number was 5 . The probabilities of crossover and mutation were considered of exponentially increasing in generation index with different scalling coefficients , but this does not have great influence over the speed of convergence toward the nearly optimal solution . The parameters M , N and T invoked in the theoretical sections were chosen as follows M = 4800 ; N = 1600 ; T = 480ms -LRB- 18:42 -RRB- All the mean flow patterns which were used at the input port of the ATM switch are approximations of a sinusoid with different degress of accuracy . This type of analysis is important because any waveform of the flow shape can be approximated with a Fouries series on a given time horizon . 18.5 Conclusions The motivation in using the GA approach for the optimization problem associated to parameter estimation phase of the traffic control mechanism was the total intractability of the optimization problem using conventional techniques because of high dimensionality , when typically , new data becomes available . Because the genetic evolution operates simultaneously on a large data base -LRB- population -RRB- of information , it is relatively insen-sitive to feedback functions synthesised by the Murgu et al : Tracking Statistics in Communication Networks . 253 2 4 6 8 10 12 14 16 1.5 3.0 4.5 6.0 7.5 9.0 10.5 0.0 Fig. 3 . The offered traffic pattern at input port 1 . Traffic Time mean actual long-run 2 4 6 8 10 12 14 16 1.5 3.0 4.5 6.0 7.5 9.0 10.5 0.0 Fig. 4 . The offered traffic pattern at input port 2 . Traffic Time long-run mean actual routing/flow controller , which necessarily vary over time and have to be obtain from noise , discontinuities , delays and failures associated partial information . The GA optimization performance in this control problem setting has been intensively analyzed during the simulations with continuous or discrete time perturbations and was found to be very useful in provinding as fast near-optimal solution for the parameter estimation phase . Perhaps , more attention has to be given to the integration level between genetic algorithm optimization as a general powerful tool and adap-tive control mechanism as in the present application . 18.6 Bibliography See chapter bibliography at the end of this proceedings . 254 Proceedings of the 1NWGA , Vaasa , Finland , 9 - 12 January 1995 2 4 6 8 10 12 14 16 1.5 3.0 4.5 6.0 7.5 9.0 10.5 0.0 Fig. 5 . The offered traffic pattern at input port 3 . Traffic Time mean actual long-run 2 4 6 8 10 12 14 16 1.5 3.0 4.5 6.0 7.5 9.0 10.5 0.0 Fig. 6 . The tracking performance for the pattern at input port 1 . Traffic Time 2 4 6 8 10 12 14 16 1.5 3.0 4.5 6.0 7.5 9.0 10.5 0.0 Fig. 7 . The tracking performance of pattern at the input port 2 . Traffic Time Murgu et al : Tracking Statistics in Communication Networks . 255 2 4 6 8 10 12 14 16 1.5 3.0 4.5 6.0 7.5 9.0 10.5 0.0 Fig. 8 . The tracking performance for the pattern at input port 3 . Traffic Time 2 4 6 8 10 12 14 16 1.5 3.0 4.5 6.0 7.5 9.0 10.5 0.0 Fig. 9 . The squared traffic pattern at input port 1 . Traffic Time mean actual long-run 2 4 6 8 10 12 14 16 1.5 3.0 4.5 6.0 7.5 9.0 10.5 0.0 Fig. 10 . The squared traffic pattern at input port 2 . Traffic Time long-run mean actual 256 Proceedings of the 1NWGA , Vaasa , Finland , 9 - 12 January 1995 2 4 6 8 10 12 14 16 1.5 3.0 4.5 6.0 7.5 9.0 10.5 0.0 Fig. 11 . The squared traffic pattern at input port 3 . Traffic Time mean actual long-run 2 4 6 8 10 12 14 16 1.5 3.0 4.5 6.0 7.5 9.0 10.5 0.0 Fig. 12 . The tracking traffic perfrormance for input port 1 . Traffic Time 2 4 6 8 10 12 14 16 1.5 3.0 4.5 6.0 7.5 9.0 10.5 0.0 Fig. 13 . The tracking traffic performance for input port 2 . Traffic Time Murgu et al : Tracking Statistics in Communication Networks . 257 2 4 6 8 10 12 14 16 1.5 3.0 4.5 6.0 7.5 9.0 10.5 0.0 Fig. 14 . The tracking traffic performance for input port 3 . Traffic Time 1 2 3 4 5 6 7 8 9 10 Generations Cost Index 5 10 15 20 25 30 35 40 Fig. 15 . The performance of GA optimization for squared traffic . 18.7 Bibliography See chapter bibliography at the end of this proceedings .

O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

parameter estimation
statistics
traffic control
variance analysis