REDUCED

MEMORY

LATENCY

FOR

REGULAR

DATA

ACCESS

Mateo

Valero

Tom?s

Lang

Montse

Peiron

Eduard

Ayguad

J.M.

Llaberia

and

J.J.

Navarro

Departament

d?Arquitectura

de

Computadors

Universitat

Polit?cnica

de

Catalunya

c

Gran

Capit

s/n

M?dul

Barcelona

SPAIN

Department

of

Electrical

and

Computer

Engineering

University

of

California

at

Irvine

Irvine

Ca

USA

ABSTRACT

Address

transformation

schemes

such

as

skewing

and

linear

transformations

have

been

proposed

to

achieve

conflict-free

access

for

streams

with

constant

stride

However

this

is

achieved

only

for

some

strides

In

this

paper

we

extend

these

schemes

to

achieve

this

conflict-free

access

for

a

larger

number

of

strides

The

basic

idea

is

to

perform

an

out-of-order

access

to

a

stream

of

fixed

length

This

stream

is

then

stored

in

a

local

memory

and

used

in

subsequent

instructions

This

mode

of

operation

is

suitable

for

vector

processors

and

for

processors

with

decoupled

access

We

consider

two

cases

first

we

develop

a

scheme

to

obtain

the

largest

possible

number

of

conflictfree

strides

Both

matched

and

unmatched

memory

systems

are

considered

we

show

that

the

number

of

strides

is

even

larger

for

the

latter

case

The

hardware

for

address

calculations

and

access

control

is

described

and

shown

to

be

of

similar

complexity

as

that

required

for

access

in

order

In

the

second

part

of

the

paper

we

consider

conflict-free

access

to

power-of-two

strides

In

this

case

we

extend

a

previously

proposed

transformation

to

allow

any

initial

address

and

length

multiple

of

the

number

of

memory

modules

We

show

that

for

this

conflict-free

access

it

is

necessary

that

the

memory

be

unmatched

and

present

an

implementation

for

M

T

where

M

is

the

number

of

modules

and

T

the

module

latency

Moreover

the

implementation

allows

the

masking

of

the

latency

of

the

address

calculation

of

the

mapper

and

of

the

bus

arbiter

Introduction

To

have

sufficient

memory

bandwidth

the

memory

of

fast

processors

is

organized

as

several

modules

that

can

be

accessed

simultaneously

To

achieve

a

memory

throughput

of

one

access

per

processor

cycle

the

number

of

memory

modules

should

be

at

least

equal

to

the

ratio

between

the

memory

cycle

and

the

processor

cycle

-LRB-

matched

memory

system

-RRB-

However

to

obtain

this

throughput

the

sequence

of

requests

has

to

be

such

that

there

are

no

conflicts

in

the

accesses

This

is

achieved

for

example

with

conventional

memory

interleaving

and

for

address

streams

with

constant

odd

strides

but

not

for

other

strides

The

reduction

of

the

latency

of

access

for

streams

with

other

strides

has

motivated

the

proposal

of

other

addressing

schemes

the

use

of

buffers

as

well

as

the

increase

of

the

number

of

memory

modules

-LRB-

unmatched

memory

system

-RRB-

The

research

in

this

area

has

pursued

three

different

objectives

namely

to

achieve

conflict-free

access

for

some

strides

to

achieve

peak

throughput

-LRB-

for

very

long

streams

-RRB-

for

strides

that

are

not

conflict

free

and

to

achieve

minimum

average

latency

-LRB-

maybe

weighted

by

the

frequency

of

the

stride

-RRB-

In

this

work

our

objective

is

to

achieve

conflict-free

access

for

a

single

address

stream

of

constant

stride

with

an

arbitrary

initial

address

We

achieve

this

by

accessing

the

elements

of

the

stream

in

an

out-of-order

way

and

by

constraining

the

length

of

the

stream

to

a

multiple

of

the

period

of

the

address

mapping

for

that

stride

Since

there

is

no

address

mapping

and

access

method

that

produces

conflict-free

access

for

all

strides

in

this

paper

we

consider

two

cases

In

the

first

part

we

describe

a

scheme

that

provides

conflict-free

access

for

a

large

set

of

strides

whereas

in

the

second

part

we

concentrate

on

strides

that

are

powers

of

two

Preliminary

results

on

these

issues

were

presented

in

-LSB-

-RSB-

Address

streams

with

a

constant

stride

are

typical

of

vector

processors

-LRB-

and

scalar

processors

accessing

vectors

-RRB-

but

appear

also

in

scalar

processing

when

the

processor

has

decoupled

memory

access

and

execute

In

both

cases

to

overlap

memory

access

and

execution

the

processor

is

decomposed

into

two

independent

modules

as

shown

in

Figure

The

memory-access

module

performs

LOAD/STOREs

of

a

stream

to/from

a

register

file

or

a

local

storage

this

corresponds

to

a

generalized

block

transfer

with

stride

not

necessarily

equal

to

one

The

execute

unit

obtains

the

operands

from

this

local

storage

Figure

Decoupled

memory

access

and

execute

architecture

As

indicated

the

scheme

we

propose

limits

the

length

of

the

streams

to

be

a

multiple

of

the

corresponding

address

mapping

period

This

is

satisfactory

for

both

vector

processor

and

decoupled-access

processors

because

in

both

cases

the

access

to

large

data

structures

is

divided

Multi-module

Memory

memory

access

module

execute

unit

local

storage

into

shorter

pieces

-LRB-

strip-mining

-RRB-

Moreover

we

propose

that

the

elements

of

the

strip

be

requested

out

of

order

and

that

the

whole

strip

be

stored

in

the

local

memory

before

its

use

by

the

processor

This

prevents

the

chaining

of

LOAD/STOREs

with

other

operations

however

this

is

reasonable

because

the

complex

timing

of

memory

accesses

make

this

chaining

difficult

anyhow

Even

though

this

decoupled

operation

would

be

the

default

we

discuss

in

Section

I.

that

the

proposed

out-oforder

access

can

support

chained

access/execute

in

cases

where

ordered

access

would

make

this

mode

of

operation

impractical

The

two

main

address

transformation

schemes

proposed

in

the

literature

to

achieve

conflict-free

access

to

streams

with

strides

that

produce

conflicts

with

conventional

interleaving

are

skewing

and

linear

transformations

These

schemes

were

initially

proposed

for

array

processors

-LSB-

-RSB-

and

later

for

vector

processors

-LSB-

-RSB-

multiprocessors

-LSB-

-RSB-

and

VLIW

processors

-LSB-

-RSB-

If

the

access

is

in

order

and

the

memory

system

is

matched

these

address

transformations

provide

conflict-free

access

to

one

family

of

strides

where

the

family

defined

by

x

is

the

set

of

strides

s

with

s

odd

-LSB-

-RSB-

Moreover

for

the

case

in

which

different

streams

are

accessed

with

strides

of

different

families

dynamic

schemes

based

on

skewing

-LSB-

-RSB-

and

on

linear

transformations

-LSB-

-RSB-

were

proposed

In

general

linear

transformations

have

the

advantage

over

skewing

that

usually

the

memory

module

number

is

simpler

to

compute

A

larger

number

of

families

of

conflict-free

strides

can

be

achieved

by

increasing

the

number

of

memory

modules

-LRB-

unmatched

memory

system

-RRB-

If

M

is

the

number

of

memory

modules

and

T

is

the

ratio

between

the

memory

cycle

and

the

processor

cycle

then

at

most

-LRB-

m-t

-RRB-

families

are

conflict

free

-LSB-

-RSB-

In

-LSB-

-RSB-

conditions

are

shown

for

linear

transformations

that

produce

conflict

free-access

to

streams

with

power-of-two

strides

and

a

matched-memory

system

but

with

the

following

restrictions

particular

values

of

the

initial

address

and

length

of

the

stream

equal

to

the

number

of

modules

-LRB-

in

reality

the

scheme

is

proposed

for

an

array

computer

where

the

accesses

to

all

modules

are

initiated

simultaneously

so

that

this

length

of

stream

is

natural

however

this

is

not

the

case

for

vector

processors

in

which

the

processor

generates

one

address

per

processor

cycle

-RRB-

In

-LSB-

-RSB-

a

method

is

proposed

to

generate

the

linear

transformation

based

on

polynomials

in

Galois

fields

It

is

shown

that

for

a

given

initial

address

and

a

power-of-two

stride

there

exists

a

value

k

so

that

starting

from

the

kth

element

the

access

to

each

substream

of

length

M

is

conflict

free

-LRB-

short-term

equidistributed

in

the

paper

-RRB-

However

this

is

not

the

case

for

the

concatenation

of

several

substreams

nor

for

the

whole

stream

In

summary

in

both

of

these

cases

conflict-free

access

is

achieved

only

for

specific

initial

addresses

and

substreams

of

length

M.

Although

out

of

the

scope

of

this

paper

it

is

worthwhile

to

mention

that

techniques

have

also

been

proposed

to

improve

efficiency

for

the

cases

in

which

conflict-free

access

is

not

achieved

For

the

skewing

and

linear

schemes

mentioned

above

peak

memory

throughput

can

be

obtained

for

x

x

-LRB-

being

x

the

conflict-free

family

-RRB-

for

long

streams

by

the

use

of

buffers

-LSB-

-RSB-

Moreover

schemes

based

on

linear

transformations

have

been

proposed

to

distribute

randomly

the

modules

corresponding

to

consecutive

addresses

so

that

the

various

strides

do

not

produce

clustering

to

memory

modules

-LSB-

-RSB-

In

-LSB-

-RSB-

a

proposal

is

made

for

an

analytical

model

that

can

be

used

to

make

comparisons

among

these

linear

transformations

For

both

schemes

most

of

the

evaluations

performed

consider

long

streams

so

that

the

initial

transient

is

not

significant

and

the

throughput

is

determined

for

the

steady

state

This

throughput

is

evaluated

as

a

function

of

several

parameters

such

as

structure

of

the

transformation

number

of

buffers

and

number

of

memory

modules

Although

in

-LSB-

-RSB-

some

measurements

are

given

for

short

streams

the

effect

of

length

is

not

discussed

nor

is

the

transformation

determined

with

a

stream

length

in

mind

In

the

first

part

of

the

paper

and

to

introduce

the

out-of-order

accessing

we

use

a

matched

memory

system

and

a

linear

transformation

of

the

addresses

although

the

same

results

can

be

obtained

with

block-interleaving

-LRB-

using

an

internal

field

of

the

address

as

module

number

-RRB-

or

with

skewing

We

show

that

this

mode

of

accessing

streams

results

in

conflict-free

accesses

for

a

larger

number

of

families

of

strides

than

ordered

access

The

case

of

unmatched

memory

system

is

studied

later

showing

the

increase

in

the

number

of

conflict-free

families

obtained

The

hardware

required

for

address

calculations

is

presented

and

the

efficiency

of

the

scheme

is

evaluated

In

the

second

part

of

the

paper

we

show

an

address

mapping

and

an

out-of-order

access

method

to

achieve

conflict-free

stream

access

for

any

power-of-two

stride

which

includes

stride

one

These

strides

have

received

attention

before

because

they

appear

for

example

in

the

execution

of

algorithms

using

a

recursive

doubling

method

We

show

that

to

achieve

conflict-free

access

for

the

whole

stream

and

any

initial

address

it

is

necessary

that

the

memory

system

be

unmatched

with

M

T

where

M

is

the

number

of

memory

modules

and

T

is

the

module

latency

and

the

access

has

to

be

performed

out

of

order

We

consider

in

detail

the

case

M

T

and

describe

an

implementation

and

conditions

that

allows

the

masking

of

the

address

calculation

the

address

mapping

and

the

bus

arbitration

so

that

the

minimum

latency

is

achieved

for

a

stream

read

of

length

L.

Model

and

Condition

for

Conflict-Free

Access

We

now

describe

the

model

of

the

memory

subsystem

present

some

definitions

and

give

the

condition

for

conflict-free

access

Figure

shows

the

general

structure

of

the

system

which

has

been

used

by

previous

authors

The

memory

is

composed

of

M

modules

and

the

module

latency

is

of

T

processor

cycles

Each

memory

module

has

q

input

and

q

output

buffers

The

processor

requests

one

element

per

-LRB-

processor

-RRB-

cycle

unless

it

has

to

wait

because

the

associated

input

memory

buffer

is

full

The

latency

of

the

stream

access

is

defined

as

the

number

of

processor

cycles

from

the

time

the

processor

sends

the

first

address

until

the

last

element

is

received

We

assume

that

the

interconnection

network

is

a

single

bus

with

a

delay

of

one

cycle

Therefore

the

latency

of

a

conflict-free

access

to

a

stream

of

length

L

is

-LRB-

T+L

-RRB-

cycles

We

consider

the

stream

length

L

equal

to

with

l

m

The

first

element

of

the

stream

has

address

and

consecutive

elements

are

separated

by

a

constant

value

S

-LRB-

the

stride

-RRB-

so

that

the

ith

element

has

address

S

-LRB-

-RRB-

Note

that

the

stream

can

have

any

initial

address

Figure

General

structure

of

the

system

-LRB-

@L

-RRB-

refers

to

any

request

ordering

Since

the

memory

is

organized

in

several

modules

an

address

mapping

is

required

which

transforms

the

address

A

-LRB-

one-dimensional

-RRB-

with

binary

representation

-LRB-

in

short

also

denoted

-RRB-

into

the

two-dimensional

space

-LRB-

module

displacement

-RRB-

Since

conflicts

depend

only

on

the

module

number

part

we

only

consider

that

component

of

the

mapping

That

is

the

module

number

b

with

binary

representation

is

given

by

b

F

-LRB-

A

-RRB-

where

F

is

the

memory-module

component

of

the

address

mapping

We

now

define

the

spatial

and

temporal

distributions

of

the

elements

of

a

stream

since

these

determine

the

latency

of

the

access

Definition

The

SPATIAL

DISTRIBUTION

of

a

stream

in

the

multi-module

memory

is

the

M

tuple

SD

where

SD

-LRB-

i

-RRB-

is

the

number

of

stream

elements

in

module

i.

address

to

processor

q

q

memory

modules

arbiter

S

address

-LRB-

-RRB-

-LRB-

mL

dL

-RRB-

-LRB-

@L

-RRB-

address

generator

sequencer

mapping

processor

The

spatial

distribution

is

LATENCY-MATCHED

-LRB-

T-MATCHED

-RRB-

if

SD

-LRB-

i

-RRB-

L/T

for

all

i

-LRB-

this

implies

that

at

least

T

modules

have

SD

-LRB-

i

-RRB-

-RRB-

If

the

spatial

distribution

of

a

stream

is

T-matched

we

say

that

the

STREAM

IS

T-MATCHED

Definition

The

TEMPORAL

DISTRIBUTION

of

a

stream

is

the

sequence

of

memory-module

numbers

-LRB-

mL

-RRB-

where

mi

is

the

module

corresponding

to

the

ith

processor

request

Note

that

the

elements

can

be

requested

in

any

order

Other

authors

use

similar

terms

related

with

the

spatial

distribution

such

as

short-term

and

longterm

equidistributed

sequences

-LSB-

-RSB-

and

with

the

temporal

distribution

such

as

return

numbers

-LSB-

-RSB-

and

variability

-LSB-

-RSB-

Definition

A

temporal

distribution

is

CONFLICT

FREE

when

every

element

can

be

accessed

as

soon

as

it

is

requested

-LRB-

the

corresponding

memory

module

is

not

busy

with

a

previous

request

-RRB-

This

is

equivalent

to

stating

that

a

temporal

distribution

is

conflict

free

if

any

subset

of

T

consecutively

requested

elements

are

located

in

T

different

memory

modules

This

is

the

condition

we

will

require

Moreover

from

the

last

definition

it

follows

directly

that

a

necessary

condition

for

a

conflictfree

temporal

distribution

is

that

the

stream

be

T-matched

Because

of

this

to

determine

the

conditions

for

a

conflict-free

temporal

distribution

we

first

determine

conditions

for

a

T-matched

stream

and

then

consider

access

orders

so

that

any

T

consecutive

accesses

are

to

different

memory

modules

Since

the

spatial

distribution

is

independent

from

the

temporal

distribution

to

determine

conditions

for

a

T-matched

stream

we

use

the

canonical

temporal

distribution

defined

below

even

though

it

might

not

be

conflict

free

Definition

The

CANONICAL

TEMPORAL

DISTRIBUTION

of

a

stream

is

the

temporal

distribution

when

the

elements

are

requested

in

order

For

any

address

transformation

scheme

the

canonical

temporal

distribution

is

periodic

Consequently

we

can

define

the

canonical

temporal

distribution

in

one

period

and

state

Lemma

below

Definition

The

period

Px

of

an

address

mapping

for

a

stream

with

stride

s

is

the

period

of

its

canonical

temporal

distribution

Definition

We

call

CTPx

the

canonical

temporal

distribution

in

one

period

for

a

stream

with

stride

s

Definition

CTPx

is

T-matched

if

the

spatial

distribution

in

the

period

is

T-matched

Lemma

If

CTPx

is

T-matched

and

L

p?Px

with

p

then

the

stream

is

T-matched

Proof

This

is

evident

since

if

each

period

is

T-matched

and

the

stream

length

is

a

multiple

of

the

period

the

stream

has

to

be

T-matched

q

In

summary

we

consider

streams

with

L

p?Px

determine

the

conditions

for

having

a

T-matched

CTPx

and

then

find

a

temporal

distribution

that

has

the

property

that

any

T

consecutive

requests

go

to

different

modules

PART

I

CONFLICT-FREE

ACCESS

FOR

A

LARGE

NUMBER

OF

FAMILIES

I.

Matched-Memory

Case

We

now

discuss

the

matched-memory

case

-LRB-

M

T

-RRB-

and

generalize

to

the

unmatched

case

in

the

next

section

For

the

matched-memory

case

we

use

the

linear

transformation

bi

as

i

ai

s

t

i

t

-LRB-

-RRB-

For

the

rest

of

this

section

we

assume

this

mapping

It

has

the

property

that

when

the

elements

of

the

stream

are

requested

in

order

the

access

is

conflict

free

for

streams

with

strides

of

the

family

with

x

s

with

any

length

and

with

any

initial

address

-LSB-

-RSB-

Figure

illustrates

a

portion

of

the

mapping

for

m

t

and

s

Figure

XOR-based

linear

transformation

and

mapping

of

the

address

space

when

m

t

and

s

As

shown

in

-LSB-

-RSB-

the

period

for

a

stride

s

is

Px

t-x

We

now

determine

that

the

families

for

x?s

produce

T-matched

streams

-LRB-

if

the

length

of

the

stream

is

a

multiple

of

the

period

-RRB-

Lemma

Let

x

s

and

Px

be

the

corresponding

period

Consider

the

grouping

of

these

Px

elements

into

substreams

consisting

of

elements

each

so

that

the

jth

substream

-LRB-

j

-RRB-

contains

the

elements

-LRB-

j

i

-RRB-

with

i

-LRB-

Figure

-RRB-

For

any

of

these

substreams

all

its

elements

are

located

in

different

memory

modules

Proof

Let

Aj

with

binary

representation

be

the

address

of

the

first

element

of

the

jth

memory

module

b

s

substream

For

the

mapping

defined

in

-LRB-

-RRB-

this

element

is

located

in

memory

module

mj

such

that

mj

-LRB-

as

as

as

-RRB-

as

s

Moreover

the

element

j

i

has

address

Aj

i

s

Aj

i?s

and

is

located

in

memory

module

-LCB-

-RCB-

-LRB-

-LRB-

as

s

-LRB-

i?s

-RRB-

mod

-RRB-

mod

-RRB-

Since

s

is

odd

the

values

-LRB-

i?s

-RRB-

mod

for

i

are

all

different

As

the

bits

are

independent

of

i

the

elements

are

stored

in

different

modules

q

Figure

Stream

periods

and

substreams

Lemma

The

families

of

strides

that

produce

T-matched

CTPx

are

those

defined

by

x

s.

Proof

If

x

s

because

of

Lemma

the

elements

-LRB-

j

i

-RRB-

mod

Px

for

i

are

in

different

memory

modules

Taking

as

values

for

j

we

obtain

substreams

of

elements

mapped

into

different

memory

modules

so

each

module

contain

elements

therefore

CTPx

is

T-matched

On

the

other

hand

if

x

s

the

elements

are

mapped

into

just

t-x

modules

so

not

all

modules

are

visited

-LRB-

s

t-x

t

m

-RRB-

Therefore

CTPx

is

not

T-matched

q

Theorem

The

families

of

strides

defined

by

s-N

x

s

where

N

min

-LRB-

l-t

s

-RRB-

produce

T-matched

streams

of

length

L

Proof

For

a

stream

to

be

T-matched

it

is

sufficient

that

CTPx

be

T-matched

and

that

L

p?Px

-LRB-

Lemma

-RRB-

Because

of

Lemma

CTPx

is

T-matched

for

all

x

s

If

x

s-N

then

x

s

-LRB-

l-t

-RRB-

for

the

definition

of

N

this

can

be

rewritten

as

l

s

t

x

or

L

p?Px

for

some

p

Therefore

the

stream

is

T-mached

q

-LCB-

-RCB-

to

simplify

the

notation

we

mix

arithmetic

operations

such

as

integer

addition

and

multiplication

with

boolean

operators

such

as

XOR

The

context

is

used

to

determine

whether

the

operand

is

a

bit

string

or

the

corresponding

integer

T

elements

of

substream

Px

T

elements

of

substream

T

elements

of

substream

L

=p

Px

Note

that

for

x

s-N

when

l-t

s

it

is

possible

for

a

stream

with

length

L

to

be

T-matched

but

this

depends

on

its

initial

address

Since

we

consider

streams

with

any

initial

address

these

cases

are

not

of

interest

Corollary

For

fixed

l

and

t

the

value

of

s

defines

a

window

of

families

of

strides

that

produce

T-matched

streams

as

shown

in

Figure

Figure

Window

of

families

that

produce

T-matched

streams

of

length

Up

to

now

we

have

shown

the

conditions

for

a

stream

to

be

T-matched

However

the

access

in

order

can

lead

to

a

high

latency

because

of

an

unsuitable

canonical

temporal

distribution

For

instance

the

mapping

described

in

Figure

results

in

T-matched

streams

of

length

for

x

however

for

example

for

a

stream

of

stride

-LRB-

x

s

-RRB-

beginning

at

address

the

period

is

Px

and

the

CTPx

is

The

access

is

not

conflict

free

In

fact

only

the

family

with

x

produces

a

conflict-free

canonical

temporal

distribution

For

the

balanced

streams

that

do

not

produce

conflict-free

access

it

is

possible

to

incorporate

buffers

so

that

after

an

initial

latency

one

data

element

is

obtained

per

processor

cycle

The

worstcase

additional

latency

occurs

when

the

last

L/T

elements

of

the

stream

are

in

the

same

memory

module

In

such

a

case

the

number

of

cycles

required

to

access

the

stream

is

L+T

L.

-LRB-

-RRB-

T

so

that

the

additional

latency

is

L.

-LRB-

-RRB-

T.

Moreover

in

this

case

the

number

of

buffers

required

per

memory

module

is

L/T

I.

Reordering

We

now

show

how

to

reorder

the

access

of

the

stream

elements

so

as

to

achieve

a

conflict-free

temporal

distribution

Theorem

A

stream

of

length

L

p?Px

and

x

s

can

be

divided

into

substreams

of

the

form

described

by

Lemma

each

of

which

is

conflict-free

Proof

Since

L

p?Px

the

stream

can

be

divided

in

p

periods

of

length

Px

In

each

of

these

periods

we

use

the

substreams

defined

in

Lemma

Since

the

elements

of

each

substream

are

mapped

in

different

modules

the

temporal

distribution

of

each

substream

is

conflict

free

q

As

a

consecuence

of

Theorem

to

achieve

conflict-free

access

the

stream

is

divided

into

substreams

and

these

substreams

are

accessed

in

sequence

However

this

assures

conflict-free

s

min

-LRB-

l-t

s

-RRB-

access

for

each

substream

accessed

separately

but

not

for

the

whole

stream

For

instance

for

a

stream

of

stride

-LRB-

x

s

-RRB-

beginning

at

address

and

length

the

mapping

described

in

Figure

results

in

a

period

Px

and

a

CTPx

that

is

repeated

for

each

of

the

four

periods

of

the

stream

Each

period

is

divided

into

two

substreams

and

therefore

the

whole

stream

into

eight

substreams

Considering

the

two

substreams

of

the

first

period

they

contain

elements

-LRB-

-RRB-

and

-LRB-

-RRB-

respectively

which

are

located

in

memory

modules

-LRB-

-RRB-

and

-LRB-

-RRB-

Since

there

are

conflicts

among

the

two

substreams

we

propose

in

the

next

Section

an

additional

reordering

to

provide

conflict-free

access

However

it

is

worth

noticing

that

with

one

buffer

at

the

input

of

each

module

and

one

buffer

at

the

output

the

above

mentioned

ordering

produces

a

latency

which

is

not

greater

than

T+L

cycles

that

is

the

increase

in

latency

due

to

the

non

conflict-free

access

is

at

most

of

cycles

Note

that

the

additional

latency

produced

and

the

number

of

buffers

required

are

significantly

smaller

than

for

in-order

access

with

buffers

This

is

shown

in

the

following

Theorem

Theorem

The

additional

latency

of

a

stream

access

with

the

reordering

proposed

above

is

of

at

most

cycles

Proof

Let

mM

in

this

order

be

the

module

numbers

accessed

by

the

first

substream

Then

we

can

make

the

following

observations

module

mM

begins

operation

in

cycle

M

and

operates

continuosly

thereafter

This

is

so

because

when

it

finishes

with

one

access

the

addresses

of

another

complete

substream

have

been

calculated

and

one

of

the

addresses

has

to

correspond

to

this

module

The

last

element

of

this

module

is

available

to

be

sent

to

the

processor

in

cycle

M

-LRB-

L/M

-RRB-

T

which

is

equal

to

T+L

since

M

T.

module

begins

operation

in

cycle

and

is

idle

at

most

one

cycle

thereafter

This

occurs

when

this

module

appears

as

the

last

module

of

some

substream

Consequently

the

last

element

of

this

module

is

available

to

be

sent

to

the

processor

in

cycle

L

T+L

generalizing

the

process

described

above

module

mi

begins

operation

in

cycle

i

is

idle

at

most

T-i

cycles

thereafter

and

has

the

last

element

available

at

cycle

i

L+T

i

L+T

Consequently

in

the

worst

case

in

cycle

L+T

there

are

M

elements

ready

to

be

sent

to

the

processor

However

since

only

one

element

can

be

sent

per

cycle

the

total

number

of

cycles

is

L+T+T

L

T

Since

a

conflict-free

access

requires

T+L

cycles

the

additional

latency

is

of

cycles

The

waiting

requests

are

stored

in

buffers

associated

to

each

memory

module

Since

each

substream

has

only

one

request

per

module

only

one

buffer

per

module

is

required

q

We

now

consider

the

calculation

of

the

addresses

This

is

divided

into

two

parts

as

shown

in

Figure

i

-RRB-

Calculation

of

addresses

inside

a

substream

Since

the

addresses

of

the

elements

of

the

jth

substream

are

Aj

i?s

-LRB-

i

-RRB-

to

compute

these

addresses

incrementally

it

is

necessary

to

begin

with

the

address

of

the

first

element

of

the

substream

and

add

s

instead

of

s

which

is

the

case

for

the

canonical

order

ii

-RRB-

Calculation

of

the

address

of

the

next

substream

Since

any

sequence

of

substreams

is

suitable

we

choose

to

access

all

substreams

of

one

period

in

sequence

and

then

go

to

the

next

period

In

this

way

the

initial

addresses

of

consecutive

substreams

in

one

period

are

separated

by

s

and

this

is

the

same

increment

required

to

go

from

the

last

element

of

one

period

to

the

first

element

of

the

next

Figure

Two

periods

in

sequence

substreams

and

separation

between

elements

The

control

to

perform

the

address

calculations

is

shown

below

SUB

SUB

is

the

initial

address

of

the

present

substream

A

A

is

the

request

address

for

k

to

-LRB-

s

t-x

-RRB-

k

is

the

period

number

for

j

to

j

is

the

substream

number

for

i

to

the

first

address

of

each

substream

is

obtained

outside

of

the

loop

A

A

s

end

for

if

j

then

-LRB-

SUB

SUB

s

A

SUB

s

-RRB-

in

parallel

end

for

SUB

A

s

A

A

s

end

for

The

hardware

required

is

shown

in

Figure

We

include

also

in

the

Figure

the

calculation

of

the

local

store

address

To

simplify

the

implementation

it

is

convenient

that

the

compiler

issues

Px

s.

Px

s.

s.

elements

of

substream

elements

of

substream

elements

of

substream

instructions

to

load

the

values

s

s

and

If

this

is

done

the

complexity

is

practically

the

same

as

that

for

the

case

in

which

requests

are

performed

in

order

Figure

Hardware

for

address

calculation

and

local

store

address

I.

Conflict-free

Ordering

Even

though

the

additional

latency

associated

with

the

ordering

described

in

the

previous

section

is

low

in

practice

-LRB-

since

L

T

-RRB-

we

now

propose

a

scheme

that

eliminates

this

additional

latency

and

permits

the

access

of

the

whole

stream

in

a

conflict-free

manner

To

achieve

this

it

is

necessary

to

incorporate

a

second

reordering

so

that

the

temporal

distribution

of

all

substreams

is

the

same

However

this

poses

a

problem

with

the

calculation

of

the

addresses

inside

the

substreams

since

to

have

a

simple

incremental

calculation

-LRB-

adding

s

-RRB-

it

is

necessary

to

do

this

in

the

order

described

in

the

previous

section

The

solution

is

to

decouple

the

calculation

of

the

addresses

from

the

actual

requests

This

is

achieved

by

calculating

the

addresses

of

substream

j

while

accessing

substream

j.

For

this

during

the

first

cycles

it

is

necessary

to

K

I

J

K?L

-LRB-

I

J

-RRB-

K?L

s

I

K?L

s

I

I

address

A

load

A

load

SUB

address

A

load

REG

load

SUBREG

K

K

init

I

I

load

dec

J

load

dec

init

J

I

init

I

load

dec

J

I

J

K?L

-LRB-

I

J

-RRB-

K?L

I

I

I

K?L

calculate

the

addresses

of

the

first

substream

-LRB-

which

are

used

immediately

for

memory

access

-RRB-

and

of

the

second

substream

-LRB-

which

are

stored

in

a

set

of

latches

for

access

as

the

next

substream

-RRB-

After

that

for

each

substream

the

addresses

for

access

are

obtained

from

the

latches

and

a

new

address

is

calculated

to

store

Consequently

as

shown

in

Figure

two

address

generators

are

needed

although

one

of

them

is

only

used

in

the

first

cycles

Moreover

it

is

necessary

to

store

the

temporal

distribution

of

the

first

substream

which

is

used

to

control

the

order

of

the

requests

of

the

following

substreams

In

addition

to

the

latches

in

the

processor

neither

buffers

are

needed

in

the

memory

modules

nor

an

arbiter

in

the

return

bus

to

the

processor

Figure

Architecture

model

for

out-of-order

memory

accesses

I.

Choice

of

s

As

shown

previously

the

proposed

scheme

achieves

conflict-free

access

to

the

families

of

strides

s

such

that

s-N

x

s

and

the

choice

of

s

determines

the

window

of

conflict-free

strides

Since

the

family

for

x

includes

all

the

odd

strides

-LRB-

and

in

particular

stride

one

-RRB-

it

is

certainly

convenient

to

include

this

family

by

making

s

l-t

the

largest

window

occurs

when

s

l-t

In

such

a

case

the

conflict-free

strides

belong

to

the

families

with

x

l-t

For

example

for

L

and

m

t

we

choose

s

and

obtain

conflict-free

access

for

the

families

defined

by

x

to

processor

M

memory

modules

S

-LRB-

m

d

-RRB-

processor

mux

q

M

buffers

order

address

generators

arbiter

other

substreams

I.

Unmatched

Memory

One

way

to

increase

the

number

of

families

of

strides

that

produce

conflict-free

access

is

to

increase

the

number

of

memory

modules

-LRB-

m

t

-RRB-

A

possible

mapping

for

the

unmatched

memory

case

is

to

use

the

same

one

as

defined

in

-LRB-

-RRB-

replacing

t

by

m

In

such

a

case

the

period

Px

is

m-x

and

for

ordered

access

any

stream

length

and

any

initial

address

conflict-free

access

is

obtained

for

the

families

defined

by

x

s

s

m-t

-LSB-

-RSB-

This

can

be

combined

with

the

technique

presented

in

Section

I.

for

streams

of

length

L

so

that

conflict-free

access

is

obtained

for

the

families

defined

by

x

s-N

s-N

s

s

m-t

In

this

case

the

streams

with

strides

belonging

to

the

families

x

s

s

m-t

are

accessed

in

order

and

the

rest

are

accessed

out-of-order

using

the

results

of

Section

I.

In

particular

if

s

l-t

the

conflict-free

families

are

x

l

t.

I.

A

Better

Mapping

We

now

consider

a

way

of

increasing

the

number

of

conflict-free

strides

for

streams

of

length

L

and

out-of-order

access

To

simplify

our

discussion

we

consider

the

special

case

where

m

t.

For

this

case

we

use

the

following

mapping

-LRB-

-RRB-

For

the

rest

of

this

section

we

assume

this

mapping

An

example

for

t

m

s

and

y

is

given

in

Figure

This

address

mapping

corresponds

to

a

division

of

the

modules

into

T

sections

of

T

modules

and

of

the

address

space

into

blocks

of

locations

each

block

is

mapped

into

one

section

using

the

mapping

defined

by

the

lower

t

bits

of

b

The

period

for

a

stride

of

the

family

s

is

Px

t-x

We

now

determine

the

set

of

families

of

strides

that

produce

T-matched

streams

and

an

out-oforder

temporal

distribution

to

achieve

conflict-free

access

Lemma

Let

x

y

and

Px

be

the

corresponding

period

Consider

the

grouping

of

these

Px

elements

into

substreams

consisting

of

elements

each

The

jth

substream

-LRB-

j

-RRB-

contains

the

elements

-LRB-

j

i

-RRB-

with

i

For

any

of

these

substreams

all

its

elements

are

located

in

different

memory

sections

b

i

as

i

a

i

ay

i

t

i

t

s

t

t

i

t

y

s

t

Figure

XOR-based

transformation

and

mapping

of

the

address

space

when

m

t

s

and

y

In

italic

the

elements

of

a

stream

with

l

with

initial

address

and

stride

S

are

shown

Proof

The

addresses

of

the

elements

in

the

jth

substream

are

given

by

Aj

i

s

Aj

i?s

with

i

Since

s

is

odd

all

these

addresses

have

different

combinations

in

the

bits

ay

y

so

all

sections

are

visited

q

Lemma

The

families

of

strides

that

produce

T-matched

CTPx

are

those

defined

by

x

y.

Proof

Because

of

Lemma

there

are

elements

of

CTPx

mapped

in

each

section

If

x

s

t

all

elements

are

located

in

the

same

module

inside

the

sections

so

each

one

of

the

T

modules

that

appear

in

CTPx

containsPx

Telements

therefore

CTPx

is

T-matched

If

x

s

t

more

modules

are

visited

in

each

section

so

each

one

contains

less

than

Px/T

elements

CTPx

is

therefore

also

T-matched

On

the

other

hand

if

x

y

just

t-x

modules

are

visited

so

CTPx

is

not

T-matched

q

Theorem

The

families

of

strides

defined

by

x

s-N

s

and

x

y-R

y

where

N

min

-LRB-

l-t

s

-RRB-

and

R

min

-LRB-

l-t

y

-RRB-

produce

T-matched

streams

of

length

L

This

defines

two

windows

of

conflict-free

families

as

shown

in

Figure

module

y

ts

Mi

646 774 902518

512 513 514 515

516 519517

507 506 505 504

511 509 508510

128 129 130 131

132 133 135

123 122 121 120

127 124126

Figure

Two

windows

of

families

that

produce

T-matched

streams

of

length

in

an

unmatched

system

Proof

Let

s-N

x

s

so

l

s

t-x

Consider

a

substream

composed

by

the

elements

-LRB-

j

i

-RRB-

with

i

these

elements

have

different

combinations

in

the

bits

as

s

-LRB-

see

Lemma

-RRB-

and

therefore

are

located

in

T

memory

modules

They

can

be

mapped

in

different

sections

or

not

depending

on

whether

the

bits

ay

y

are

modified

or

not

In

the

stream

there

are

substreams

identified

by

k

t-x

with

-LRB-

t-x

-RRB-

and

If

we

assume

that

the

bits

ay

y

are

not

modified

we

obtain

that

the

stream

elements

are

mapped

into

T

modules

containing

L/T

elements

each

If

the

bits

ay

y

are

modified

then

the

elements

are

distributed

in

more

memory

modules

containing

each

one

less

than

L/T

elements

Therefore

the

stream

is

T-matched

Let

y-R

x

y.

Then

l

y

t-x

so

L

p?Px

for

some

p

Moreover

because

of

Lemma

CTPx

is

T-matched

Therefore

the

stream

is

T-matched

q

To

have

two

non-overlapping

set

of

families

of

strides

that

produce

T-matched

streams

we

make

y-R

s

This

results

in

R

l-t

and

y

s

l-t

Therefore

there

are

two

groups

of

families

of

strides

those

with

x

in

-LSB-

s-N

s

-RSB-

and

those

in

-LSB-

y-R

y

-RSB-

Because

of

Lemmas

and

to

achieve

conflict-free

access

we

divide

the

stream

into

a

sequence

of

substreams

as

follows

i

-RRB-

for

s-N

x

s

we

define

substreams

as

stated

in

Lemma

We

know

that

the

elements

of

each

substream

are

mapped

into

different

modules

because

they

have

different

combinations

in

the

bits

as

s

therefore

the

temporal

distribution

is

conflict

free

For

some

values

of

s

and

also

the

bits

ay

y

of

the

elements

in

one

substream

may

vary

in

this

case

many

sections

are

visited

in

one

substream

which

also

leads

to

a

conflict-free

temporal

distribution

of

the

substream

ii

-RRB-

for

y-R

x

y

we

define

substreams

as

stated

in

Lemma

In

this

case

the

elements

of

each

substream

are

mapped

into

different

sections

so

its

temporal

distribution

is

conflict

free

As

an

example

consider

the

mapping

shown

in

Figure

and

let

x

s

and

L

The

addresses

of

the

elements

in

one

period

Px

are

shown

in

bold

in

Figure

This

period

is

divided

into

eight

substreams

corresponding

to

the

stream

elements

as

follows

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

These

elements

are

stored

in

the

memory

modules

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

s

min

-LRB-

l-t

s

-RRB-

y

min

-LRB-

l-t

y

-RRB-

For

the

calculation

of

the

addresses

we

use

the

same

algorithm

as

in

section

I.

where

the

increment

of

address

in

the

inner

loop

is

either

s

or

s

As

in

section

I.

we

have

obtained

substreams

whose

temporal

distribution

is

conflict

free

however

this

might

not

be

the

case

for

the

whole

stream

For

example

consider

x

s

and

In

this

case

Px

so

there

are

two

substreams

in

one

CTPx

corresponding

to

the

stream

elements

-LRB-

-RRB-

and

-LRB-

-RRB-

These

elements

are

located

in

the

modules

-LRB-

-RRB-

and

-LRB-

-RRB-

respectively

so

if

the

elements

are

accessed

there

are

conflicts

Again

next

we

give

an

additional

reordering

that

provides

conflict-free

access

to

the

whole

stream

I.

Conflict-free

reordering

To

achieve

conflict-free

access

to

the

whole

stream

we

follow

a

similar

strategy

as

that

presented

in

Section

I.

for

matched

memory

However

in

that

case

all

substreams

visit

all

memory

modules

so

it

is

sufficient

to

remember

the

order

in

which

the

first

substream

is

accessed

and

use

it

for

the

remaining

substreams

This

is

not

the

case

for

the

unmatched

memory

case

since

different

substreams

may

visit

different

modules

Because

of

this

we

modify

slightly

the

strategy

as

explained

below

The

following

definition

is

useful

Definition

A

section

is

composed

of

modules

labeled

to

The

SUPERMODULE

i

consists

of

the

ith

module

of

each

section

That

is

the

supermodule

number

of

an

address

is

determined

by

the

bits

as

s.

Two

cases

have

to

be

considered

as

follows

i

-RRB-

s-N

x

s

for

this

case

as

stated

before

the

substreams

used

are

those

defined

in

Lemma

Since

the

elements

in

these

substreams

have

different

combinations

in

the

bits

as

s

all

supermodules

appear

in

each

substream

Therefore

we

can

apply

the

same

strategy

as

in

section

I.

but

at

the

supermodule

level

-LRB-

i.e.

the

supermodule

numbers

of

the

first

substream

must

be

remembered

and

the

elements

of

the

remaining

substreams

should

be

requested

in

the

same

supermodule

order

as

the

first

one

-RRB-

Note

that

this

implies

that

two

latches

are

needed

per

supermodule

not

per

module

This

results

in

latches

-LRB-

not

-RRB-

ii

-RRB-

y-R

x

y

now

the

substreams

are

defined

by

Lemma

Since

its

elements

are

in

different

sections

it

is

sufficient

to

request

each

substream

in

the

section

order

of

the

first

substream

In

summary

i

-RRB-

for

x

s

the

supermodule

order

of

the

first

substream

is

stored

-LRB-

bits

-RRB-

and

the

latches

are

labeled

by

supermodule

ii

-RRB-

for

x

s

the

section

order

of

the

first

substream

is

stored

-LRB-

bits

t

-RRB-

and

the

latches

are

labeled

by

section

I.

Choice

of

s

and

y

For

the

proposed

reordering

we

obtain

conflict-free

access

for

the

families

with

x

s-N

s

and

x

y-R

y.

To

obtain

the

largest

single

window

which

contains

x

we

make

s

l-t

and

y

s

l-t

-LRB-

l-t

-RRB-

This

results

in

conflict-free

access

for

x

-LRB-

l-t

-RRB-

Compared

to

the

mapping

used

at

the

beginning

of

Section

I.

this

provides

l-t

additional

families

For

example

for

L

T

and

M

we

choose

s

and

y

and

obtain

conflict-free

access

for

the

families

defined

by

x

as

shown

in

Figure

Moreover

the

conflict-free

access

for

families

x

and

x

is

obtained

with

access

in

order

without

restrictions

in

the

length

of

the

stream

Other

families

require

out-of-order

access

and

stream

lengths

multiple

of

their

periods

-LRB-

if

x

and

if

x

-RRB-

Note

that

with

this

positioning

of

the

y

field

the

elements

that

compose

the

substreams

are

the

same

for

an

x

of

window

and

the

corresponding

x

of

window

Figure

Mapping

and

structure

of

the

substreams

for

m

t

l

s

and

y

I.

Additional

conflict-free

families

In

this

section

we

show

how

to

obtain

additional

conflict-free

families

of

strides

In

the

previous

section

we

obtained

two

windows

of

conflict-free

families

as

follows

The

first

window

has

the

upper

limit

placed

in

bit

s

-LRB-

families

-LSB-

s-N

s

-RSB-

-RRB-

Conflict-free

access

is

obtained

by

defining

substreams

whose

consecutive

elements

have

addresses

separated

by

s

Consequently

the

elements

of

a

substream

go

to

different

memory

modules

because

their

addresses

produce

different

combinations

in

field

as

s.

Similarly

the

second

window

has

the

upper

limit

placed

in

bit

y

-LRB-

families

-LSB-

y-R

y

-RSB-

-RRB-

Conflict-free

access

is

obtained

by

defining

substreams

whose

consecutive

elements

have

addresses

separated

by

s

The

elements

of

a

substream

go

to

different

memory

modules

because

their

addresses

produce

different

combinations

in

field

ay

y.

y

ts

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

window

-LRB-

-RRB-

We

now

consider

the

case

in

which

the

substream

uses

some

bits

of

field

as

s

and

some

bits

of

field

ay

y

to

produce

the

required

different

combinations

This

is

useful

for

families

with

s

x

s

which

produce

T-matched

streams

as

stated

in

Lemma

Note

that

these

families

could

also

be

made

conflict

free

using

the

window

rooted

in

y

but

this

requires

that

the

corresponding

window

covers

the

field

as

s.

By

following

the

approach

discussed

here

it

is

possible

to

move

the

y-rooted

window

and

increase

the

number

of

conflict-free

families

We

now

find

substreams

whose

elements

are

located

in

different

memory

modules

and

are

therefore

conflict

free

Consider

the

grouping

of

the

stream

elements

into

substreams

so

that

the

addresses

of

the

elements

of

a

substream

have

different

values

in

bits

av

-LRB-

x-s

-RRB-

v

-LRB-

x-s

bits

-RRB-

and

as

x

-LRB-

t

-LRB-

x-s

-RRB-

bits

-RRB-

with

y

v

y

t

-LRB-

x-s

-RRB-

and

y

s

t

as

shown

in

Figure

a.

Because

of

mapping

-LRB-

-RRB-

these

t

bits

affect

different

bits

of

the

memory

module

number

so

that

the

elements

are

located

in

different

memory

modules

Moreover

to

assure

that

the

substream

elements

belong

to

the

stream

v

s

l.

Consequently

s

t

y

s

l

The

elements

of

the

jth

substream

are

with

with

-LRB-

x-s

-RRB-

and

The

rest

of

the

bits

-LRB-

al

v

x-s

and

s

t

-RRB-

specify

the

substream

number

To

have

a

single

field

for

this

purpose

we

select

v

s

l

as

shown

in

Figure

b

In

such

a

case

the

initial

element

of

the

jth

substream

-LRB-

j

-RRB-

is

Note

that

now

the

separation

between

addresses

of

consecutive

substream

elements

is

not

a

constant

in

the

first

part

of

the

substream

the

elements

are

separated

by

s

whereas

in

the

second

part

they

are

separated

by

s

-LRB-

x-s

-RRB-

Figure

Bits

used

in

the

generation

of

substreams

for

the

third

window

i

x

i

j

x

s

-LRB-

-RRB-

y

s

x

t

-LRB-

x-s

-RRB-

bits

v

x-s

bits

l

t

s

t

y

s

x

t

-LRB-

x-s

-RRB-

bits

l

t

s

t

s

l

x-s

bits

-LRB-

a

-RRB-

-LRB-

b

-RRB-

ymin

ymax

vmin

vmax

As

before

we

now

place

the

three

windows

so

as

to

obtain

the

largest

possible

single

window

which

begins

at

x

This

results

in

s

l-t

and

y

s

l

The

total

window

size

is

then

For

example

for

L

T

and

M

we

choose

s

and

y

-LRB-

instead

of

y

as

done

in

section

I.

-RRB-

as

shown

in

Figure

Figure

Three

conflict-free

windows

when

the

y-rooted

window

is

shifted

Mapping

for

m

t

l

s

and

y

and

structure

of

the

substreams

for

the

families

of

the

third

window

The

three

windows

of

conflict-free

families

are

-LSB-

-RSB-

-LSB-

-RSB-

and

-LSB-

-RSB-

In

the

same

Figure

we

also

show

the

stream

elements

that

make

up

the

substreams

for

the

new

conflict-free

families

For

instance

a

stream

of

the

family

x

has

substreams

of

elements

each

as

defined

above

The

existence

of

different

compositions

for

the

substreams

complicate

the

hardware

for

address

generation

and

access

control

This

is

the

reason

why

these

additional

families

have

not

been

considered

in

the

development

of

the

previous

section

I.

Generalization

to

any

degree

of

unmatchness

In

the

previous

section

we

have

used

a

memory

system

where

the

number

of

memory

modules

M

is

equal

to

The

method

can

be

easily

generalized

to

systems

with

any

degree

of

unmatchness

U

M/T

-LRB-

including

the

matched

case

as

a

particular

case

-RRB-

Consider

that

m

k.t

k

-LRB-

i.e.

U

-LRB-

-RRB-

t

k

-RRB-

In

this

case

the

proposed

address

mapping

has

k

bit-fields

the

first

have

t

bits

and

the

last

has

t

k

bits

If

the

method

described

in

section

I.

is

used

the

fields

are

positioned

starting

at

bits

-LRB-

l-t

-RRB-

-LRB-

l-t

-RRB-

i

with

i

In

this

case

k.

-LRB-

l-t

-RRB-

k

conflict-free

families

are

obtained

On

the

other

hand

with

the

method

described

in

section

I.

the

fields

are

positioned

with

the

low

bits

in

-LRB-

l-t

-RRB-

i.l

with

i

In

this

case

k.l

k

t

conflict-free

families

are

obtained

For

example

Figure

a

shows

the

address

mapping

that

is

obtained

for

t

and

m

-LRB-

k

and

k

therefore

a

degree

of

unmatchness

U

-RRB-

and

l

Figure

b

shows

a

comparison

of

the

number

of

conflict-free

families

obtained

for

different

values

of

U

t

and

l

-LRB-

with

in-order

access

out-of-order

access

-LRB-

Section

I.

-RRB-

and

the

maximum

-LRB-

Section

I.

-RRB-

-RRB-

y

ts

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

window

window

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

Figure

-LRB-

a

-RRB-

Mapping

for

t

m

and

l

-LRB-

b

-RRB-

Comparison

of

the

number

of

conflict-free

families

obtained

when

t

and

l

access

in

order

out-of-order

and

the

maximum

I.

Evaluation

In

this

Section

we

present

a

discussion

of

the

effectiveness

of

the

proposed

scheme

in

terms

of

its

efficiency

and

the

cost

of

implementation

Moreover

we

comment

on

the

possibility

of

using

chaining

of

LOAD/STORE

and

EXECUTE

in

important

particular

cases

A

-RRB-

Fraction

of

strides

that

are

conflict

free

We

now

determine

the

fraction

f

of

conflict-free

strides

for

the

choices

of

s

and

y

presented

in

sections

I.

and

I.

Since

the

fraction

of

strides

belonging

to

family

s

is

the

fraction

of

conflict-free

strides

produced

by

a

window

from

x

to

x

w

is

Figure

shows

the

fraction

of

strides

that

are

conflict

free

as

a

function

of

the

degree

of

unmatchness

U

when

t

and

l

The

values

are

shown

when

the

out-of-order

method

of

Section

I.

is

used

B

-RRB-

Access

of

non

conflict-free

strides

and

efficiency

assuming

uniform

distribution

of

strides

The

families

of

strides

included

in

A

-RRB-

are

conflict

free

so

one

element

can

be

obtained

from

memory

per

cycle

after

the

startup

of

T

The

rest

of

families

are

not

conflict

free

because

the

elements

of

streams

of

the

family

with

x

w

i

-LRB-

i

-RRB-

are

located

in

modules

and

one

element

is

accessed

every

cycles

in

average

Consequently

for

uniform

distribution

of

strides

the

efficiency

h

can

be

approximated

by

U

number

of

families

u

k.

-LRB-

l-t

-RRB-

k

k.l-t

k

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

y

ts

y

y

ts

y

-LRB-

a

-RRB-

-LRB-

b

-RRB-

section

I.

section

I.

Conflict-free

families

Conflict-free

families

f

i

w

where

w

is

the

boundary

of

the

conflict-free

window

For

the

matched

example

given

in

Section

I.

the

efficiency

is

h

whereas

for

the

unmatched

memory

in

Section

I.

is

h

In

comparison

for

in

order

access

the

highest

efficiency

is

obtained

for

s

-LRB-

to

have

conflict-free

access

for

odd

strides

-RRB-

This

results

in

h

for

the

matched

memory

case

and

h

for

the

unmatched

one

If

for

a

given

application

the

actual

frequency

of

different

strides

is

known

the

efficiency

can

be

computed

by

weighting

the

corresponding

number

of

accesses

per

cycle

Some

measurements

of

frequency

are

reported

in

-LSB-

-RSB-

Figure

Fraction

of

conclict-free

families

obtained

when

t

and

l

acces

in

order

and

out-of-order

C

-RRB-

Complexity

of

the

hardware

address

calculation

buffers

arbitration

and

register

addressing

As

shown

in

Figure

the

address

calculation

for

conflict-free

access

in

out-of-order

mode

requires

two

address

generators

instead

of

one

for

the

standard

ordered

scheme

Moreover

it

requires

a

buffer

of

size

T

a

queue

to

store

the

temporal

distribution

of

the

first

substream

and

an

arbiter

to

issue

the

remaining

substreams

in

the

same

order

Finally

a

controller

is

required

to

perform

the

address

calculations

in

the

required

order

We

estimate

that

the

hardware

cost

of

these

components

is

a

minor

part

of

the

cost

of

the

whole

processor

To

achieve

the

required

throughput

it

might

be

necessary

to

pipeline

the

adder

-LRB-

this

would

also

be

needed

in

the

standard

ordered

access

and

the

latency

of

the

adder

would

also

have

to

be

added

h

t

U

f

in

order

out-of-order

to

the

latency

of

the

stream

access

unless

several

stream

accesses

are

chained

together

-RRB-

Moreover

the

usual

techniques

have

to

be

used

to

eliminate

the

dependencies

in

the

calculation

of

successive

addresses

In

addition

to

the

buffers

mentioned

before

no

additional

ones

are

needed

This

is

in

contrast

with

other

proposals

that

include

a

significant

number

of

buffers

to

eliminate

the

effect

of

an

unsuitable

temporal

distribution

To

support

the

out-of-order

access

elements

of

the

local

storage

have

to

be

addressed

out-oforder

Consequently

this

local

storage

has

to

be

of

the

random-access

type

whereas

for

ordered

access

a

FIFO

organization

is

adequate

D

-RRB-

Efficiency

of

more

memory

modules

As

has

been

seen

in

Section

I.

the

addition

of

memory

modules

to

make

the

memory

unmatched

increases

the

number

of

families

of

strides

that

are

conflict

free

However

this

is

obtained

at

a

large

expense

because

to

double

the

number

of

conflict-free

families

it

is

necessary

to

square

the

number

of

memory

modules

This

is

aggravated

by

the

fact

that

the

added

families

contain

fewer

strides

and

that

these

strides

are

probably

less

frequently

used

Of

course

the

addition

of

memory

modules

can

be

justified

by

other

reasons

such

as

simultaneous

access

to

several

streams

and

non

regular

data

access

E

-RRB-

Possibility

of

chaining

of

LOAD

and

EXECUTE

As

mentioned

in

the

introduction

the

complicated

timing

produced

when

access

in

order

is

coupled

with

buffers

makes

it

impractical

to

chain

two

instructions

if

one

of

the

operands

of

the

second

is

being

obtained

with

a

LOAD

In

contrast

the

scheme

proposed

produces

one

stream

element

each

cycle

in

a

deterministic

order

-LRB-

for

conflict-free

strides

-RRB-

Consequently

it

is

possible

to

perform

the

chaining

if

the

first

instruction

is

executed

using

the

same

order

of

elements

as

the

LOAD

Note

that

the

sequence

of

local

store

addresses

is

produced

anyhow

as

part

of

the

LOAD

F

-RRB-

Conflict-free

families

and

stream

length

For

unmatched

memory

the

access

in

order

produces

at

most

t

conflict-free

families

for

any

stream

length

-LRB-

for

m

t

-RRB-

In

contrast

the

scheme

we

propose

produces

only

two

conflict-free

families

for

any

stream

length

but

increases

to

-LRB-

l-t

-RRB-

the

number

of

conflict-free

families

for

streams

of

length

L

PART

II

CONFLICT-FREE

ACCESS

TO

POWER-OF-TWO

STRIDES

The

mapping

and

access

mechanism

discussed

in

Part

I

provides

conflict-free

access

to

a

window

of

families

In

some

cases

this

is

not

adequate

because

some

specific

strides

can

not

be

included

This

is

for

example

the

case

when

it

is

desired

to

have

conflict-free

access

to

all

power-of-two

strides

II

Mapping

and

Matched

Memory

We

now

consider

a

mapping

for

conflict-free

access

to

power-of-two

strides

and

present

the

constraints

when

the

access

is

performed

in

order

To

achieve

conflict-free

access

for

power-of-two

strides

we

use

the

following

module

component

of

the

address

mapping

-LRB-

-RRB-

For

example

the

mapping

for

m

and

n

is

shown

in

Figure

Figure

Example

of

the

address

mapping

defined

in

-LRB-

-RRB-

for

m

and

n

This

address

transformation

belongs

to

the

class

defined

in

-LSB-

-RSB-

As

shown

there

for

a

matched

memory

system

it

produces

conflict-free

access

for

substreams

of

length

M

stride

and

such

that

the

address

of

the

first

element

has

value

in

bits

ax

x.

For

example

consider

M

an

initial

address

and

a

stream

access

with

stride

S

In

this

case

the

elements

of

the

substream

are

in

memory

modules

However

there

are

conflicts

-LRB-

the

stream

is

not

T-matched

-RRB-

if

the

beginning

address

has

a

value

not

equal

to

zero

in

bits

ax

x.

For

instance

if

the

initial

address

is

then

the

elements

of

the

substream

are

in

memory

modules

Conflicts

also

appear

if

the

substream

is

longer

For

example

with

length

and

an

initial

address

bi

ai

k

m

ai

j

m

a

i

m

a

i

k

n

m

and

i

m

x

m

x

b

A

010 101 111 000

010 101 111 011

010 101 111 000

the

elements

of

the

substream

are

in

memory

modules

In

this

case

the

stream

is

T-matched

but

conflicts

are

produced

due

to

an

unsuitable

temporal

distribution

In

the

following

sections

we

describe

a

scheme

to

extend

this

conflict-free

access

to

streams

with

any

initial

address

and

lengths

that

are

multiple

of

M.

To

achieve

this

we

use

the

address

mapping

-LRB-

-RRB-

but

make

the

memory

unmatched

and

access

the

elements

out-of-order

II

Unmatched

memory

and

T-matched

substreams

Lemma

For

a

stream

with

stride

S

and

using

the

mapping

defined

by

-LRB-

-RRB-

any

substream

of

M

consecutive

elements

has

at

most

two

elements

in

the

same

memory

module

Proof

Let

W

be

the

initial

address

of

such

a

substream

The

address

of

element

ith

of

the

substream

is

W

i

-LRB-

i

-RRB-

As

shown

in

Figure

consider

the

bits

ax

x

Since

the

stride

is

these

m

bits

will

take

a

different

combination

for

each

of

the

M

elements

of

the

substream

Moreover

it

is

possible

to

divide

the

substream

into

two

parts

of

consecutive

elements

one

part

contains

the

elements

that

in

the

calculation

of

their

address

do

not

produce

a

carry

into

bit

ax

m

-LRB-

we

call

this

part

the

NC

part

-RRB-

and

the

other

contains

the

rest

of

the

elements

-LRB-

this

part

is

called

the

C

part

-RRB-

Any

two

elements

of

one

part

are

mapped

into

different

memory

modules

This

is

because

each

element

of

one

part

has

a

different

combination

for

bits

ax

x

and

the

same

value

for

all

other

bits

in

the

address

Because

of

this

property

of

each

of

the

two

parts

the

whole

substream

is

mapped

so

that

no

more

than

two

elements

go

to

the

same

module

q

For

example

for

the

substream

with

initial

address

Figure

shows

the

sequence

of

requests

performed

for

x

Figure

Value

of

bits

x

x

for

elements

of

a

substream

Theorem

A

stream

with

any

initial

address

stride

S

and

length

L

=p

M

is

T-matched

if

the

memory

is

unmatched

with

M

T.

Proof

Divide

the

stream

of

length

L

into

L/M

substreams

each

of

M

consecutive

elements

Form

Lemma

we

conclude

that

each

memory

module

has

at

most

two

elements

per

substream

010 101 111 011

C

part

NC

part

carry

in

bit

ax

m

bits

ax

m

x

memory

module

resulting

in

at

most

L/M

elements

for

the

whole

stream

Therefore

the

stream

is

T-matched

for

L/M

L/T

that

is

M

T.

q

Consequently

if

the

memory

is

unmatched

with

M

T

the

stream

is

T-matched

and

a

conflictfree

access

can

be

achieved

On

the

other

hand

when

the

system

is

matched

-LRB-

M

T

-RRB-

streams

are

not

T-matched

so

it

is

not

possible

to

obtain

a

conflict-free

temporal

distribution

Since

in

this

paper

we

are

interested

in

conflict-free

access

from

now

on

we

consider

M

T

because

this

is

the

smallest

number

of

modules

required

II

Conflict-Free

substreams

and

reordering

At

this

point

we

have

T-matched

streams

but

not

conflict-free

access

This

is

due

to

the

memory

module

repetitions

that

might

exist

between

the

two

parts

of

each

substream

For

instance

consider

that

M

T

x

and

an

initial

address

W

for

the

substream

In

this

case

the

substream

will

produce

the

following

sequence

of

module

numbers

Observe

that

in

the

and

cycles

it

is

necessary

to

access

memory

modules

that

are

still

busy

As

shown

in

-LSB-

-RSB-

it

is

possible

to

reorder

the

requests

of

a

substream

so

as

to

have

conflict-free

access

However

this

requires

a

somewhat

complicated

control

Because

of

this

in

this

section

we

define

a

different

decomposition

of

the

stream

into

substreams

and

show

an

out-of-order

conflictfree

access

Theorem

Consider

a

stream

with

stride

S

Any

substream

of

M

elements

whichs

begins

in

an

address

with

in

bits

ax

x

is

conflict-free

Proof

For

this

substream

no

carry

into

bit

ax

m

is

produced

in

the

calculation

of

the

addresses

As

a

consequence

all

the

other

bits

of

the

address

have

the

same

value

for

all

the

elements

of

the

substream

and

therefore

each

element

is

mapped

into

a

different

memory

module

The

substream

is

conflict

free

q

These

substreams

are

named

CF-substreams

Figure

a

shows

a

stream

of

length

L

M

with

four

CF-substreams

of

length

M

labeled

-LRB-

-RRB-

to

-LRB-

-RRB-

and

two

pieces

at

the

beginning

-LRB-

-RRB-

and

end

-LRB-

-RRB-

The

initial

address

of

the

first

CF-substream

is

101 010 011 101

W

cycle

C

part

NC

part

memory

module

carry

in

bit

ax

m

element

and

the

initial

address

of

the

jth

CF-substream

is

Bj

-LRB-

-RRB-

m

Since

each

of

these

CF-substreams

is

conflict-free

if

the

stream

consisted

only

of

these

substreams

we

would

achieve

a

conflict-free

access

to

the

whole

stream

with

a

matched

memory

system

-LRB-

M

T

-RRB-

However

the

stream

includes

also

the

pieces

-LRB-

-RRB-

and

-LRB-

-RRB-

in

each

of

them

there

is

no

module

repetition

-LRB-

because

there

is

no

carry

into

bit

ax

m

-RRB-

but

there

might

be

repetitions

for

the

substream

composed

of

both

pieces

Consequently

it

is

necessary

that

M

T.

Figure

Temporal

distribution

that

leads

to

conflict-free

access

Addresses

have

bits

x

x

equal

to

The

temporal

distribution

we

propose

for

conflict-free

access

is

out-of-order

in

two

respects

first

the

parts

-LRB-

-RRB-

and

-LRB-

-RRB-

in

the

example

of

Figure

a

are

combined

and

are

postponed

until

the

end

of

the

stream

so

that

each

of

the

other

substreams

is

conflict

free

However

this

still

leaves

possible

conflicts

between

the

last

T

elements

of

one

substream

and

the

first

T

elements

of

the

next

substream

as

well

as

possible

conflicts

in

the

last

substream

-LRB-

parts

-LRB-

-RRB-

-LRB-

-RRB-

-RRB-

These

conflicts

are

eliminated

by

the

second

level

of

out-of-order

access

as

illustrated

in

Figure

b

We

now

describe

this

conflict-free

temporal

distribution

and

show

how

to

achieve

it

In

the

first

M

T

cycles

the

substream

beginning

at

is

accessed

in

order

In

the

next

M

cycles

the

second

substream

is

accessed

out-of

order

to

satisfy

the

requirements

of

conflict-free

access

And

so

on

for

the

rest

of

the

substreams

Since

the

module

latency

is

T

and

all

M

elements

in

a

CF-substream

go

to

different

modules

only

the

T

first

elements

of

the

substeam

have

to

be

accessed

out

of

order

To

obtain

this

conflict-free

access

the

T

first

addresses

of

the

conflicting

substream

are

precomputed

and

placed

in

buffers

corresponding

to

the

memory

modules

-LRB-

one

address

per

buffer

-RRB-

Associated

with

each

module

buffer

there

is

a

down

counter

which

is

set

to

T

when

an

access

is

sent

to

the

module

and

decremented

each

processor

cycle

For

accesses

from

the

conflicting

substream

it

is

necessary

to

select

one

address

from

those

buffers

for

which

the

corresponding

counter

has

already

reached

In

the

last

M

cycles

the

elements

belonging

to

the

first

piece

and

ending

piece

are

accessed

A

similar

approach

is

used

for

the

control

of

this

part

however

in

this

case

m

m

mod

-LRB-

-RRB-

in

order

out-of-order

out-of-order

stream

temporal

distribution

-LRB-

a

-RRB-

-LRB-

b

-RRB-

in

order

out-of-order

in

order

out-of-order

in

order

up

to

two

addresses

can

be

stored

in

a

module

buffer

so

that

for

conflict-free

access

it

is

necessary

to

give

higher

priority

to

those

buffers

that

contain

more

than

one

address

II

Implementation

To

produce

this

conflict-free

temporal

distribution

two

address

generators

-LRB-

named

and

-RRB-

a

set

of

buffers

and

an

arbiter

are

used

Figure

shows

how

they

are

used

to

obtain

a

conflict-free

access

for

the

example

of

Figure

Moreover

a

mapper

is

required

to

obtain

the

memory-module

number

We

now

comment

on

a

possible

implementation

of

these

components

Figure

Address

generators

buffers

and

arbiter

a

-RRB-

Address

generators

With

respect

to

the

address

generators

we

observe

the

following

generates

all

the

addresses

of

the

substreams

of

length

T

that

are

accessed

in

order

-LRB-

-RRB-

Consequently

these

can

be

placed

directly

on

the

bus

without

going

through

the

buffer

generates

the

substreams

of

length

T

that

are

accessed

out-of-order

-LRB-

-RRB-

These

addresses

are

stored

in

the

buffer

In

the

generation

of

the

addresses

of

each

substream

only

the

bits

ax

x

change

-LRB-

without

propagating

carry

into

bit

ax

m

-RRB-

We

assume

that

this

calculation

takes

one

cycle

The

compiler

loads

the

values

and

as

initial

values

for

and

respectively

Since

the

initial

address

of

consecutive

substreams

are

related

by

Bj

Bj

x

m

the

initial

value

for

each

sequence

generated

by

is

obtained

by

a

counter

with

n-x-m

bits

-LRB-

where

n

is

the

number

of

bits

of

the

memory

address

-RRB-

Since

x

can

be

equal

to

in

general

a

counter

of

n-m

bits

is

required

However

as

illustrated

by

the

dotted

lines

in

Figure

at

least

T

cycles

are

available

for

the

computation

of

this

address

Ta

Tb

Ta

Tb

buffer

bus

The

initial

address

for

each

sequence

generated

by

is

obtained

as

a

continuation

of

the

last

sequence

generated

by

The

address

is

provided

by

the

compiler

and

is

loaded

at

the

right

cycle

as

initial

condition

of

This

results

in

the

implementation

shown

in

Figure

Figure

Hardware

for

address

calculation

interconnection

of

generators

and

The

addresses

generated

by

goes

directly

to

the

bus

The

addresses

generated

by

are

stored

in

the

buffer

b

-RRB-

Address

buffering

and

arbiter

The

characteristics

of

the

buffers

are

as

follows

Each

cycle

the

element

whose

address

is

generated

by

is

placed

in

the

corresponding

buffer

The

maximum

size

of

each

buffer

is

three

In

the

example

of

Figure

it

could

contain

one

element

from

one

from

Ta

and

one

from

Tb

Finally

the

arbiter

can

be

pipelined

because

there

are

at

least

T

cycles

between

the

time

an

element

is

placed

in

the

buffer

and

the

time

in

which

it

is

sent

to

the

bus

Moreover

when

sending

elements

of

-LRB-

-RRB-

or

-LRB-

-RRB-

the

arbitration

policy

gives

priority

to

buffers

with

size

two

To

use

the

scheme

presented

here

the

length

of

the

stream

has

to

be

at

least

T

That

it

works

for

this

length

can

be

seen

from

Figure

by

eliminating

and

On

the

other

hand

for

shorter

streams

for

example

T

the

latency

of

the

pipelined

arbiter

can

not

be

masked

c

-RRB-

Address-mapping

unit

The

address-mapping

unit

-LRB-

mapper

-RRB-

determines

the

module

number

in

accordance

to

the

mapping

defined

in

Section

II

A

direct

implementation

of

this

mapping

requires

m

networks

each

one

producing

the

XOR

function

of

n/m

variables

If

the

network

is

implemented

with

XOR

gates

of

two

inputs

a

tree

of

depth

log

-LRB-

n/m

-RRB-

is

required

Since

the

delay

of

this

tree

might

be

large

we

now

describe

a

scheme

that

performs

the

calculation

faster

for

a

sequence

of

elements

for

which

the

address

calculations

do

not

produce

a

carry

into

bit

x

m

-LRB-

this

is

true

for

a

CF-substream

as

well

for

the

initial

and

final

pieces

taken

individually

-RRB-

As

shown

in

Figure

in

this

case

the

module

adder

m

reg

mux

mux

reg

@initial

@initial

x

x

to

bus

to

buffer

number

can

be

computed

incrementally

This

incremental

calculation

is

based

on

the

following

the

bits

outside

of

the

window

x

x

do

not

change

the

value

of

the

window

is

incremented

by

one

for

each

consecutive

element

each

bit

of

the

window

affects

a

different

bit

of

the

module

number

the

window

is

not

aligned

with

the

grouping

of

bits

defined

in

the

mapping

A

possibility

of

alignment

would

be

to

shift

the

incrementer

output

circularly

to

the

left

by

-LRB-

x

mod

m

-RRB-

positions

Instead

the

same

result

is

obtained

by

having

a

circular

adder

and

adding

mod

m

instead

of

This

calculation

can

be

performed

concurrently

with

the

address

generation

so

no

additional

time

is

required

for

the

mapping

As

with

the

addresses

the

complete

mapping

has

to

be

performed

to

obtain

the

module

number

of

the

first

element

in

a

sequence

however

as

discussed

before

T

cycles

are

available

for

this

Figure

Correspondence

between

bits

in

the

address

window

and

in

the

memory

module

number

This

scheme

is

implemented

by

the

hardware

of

Figure

The

module

number

of

the

first

element

of

the

CF-substream

is

preloaded

in

register

INIT

and

register

INC

is

initialized

to

The

circular

adder

produces

the

bits

in

the

window

-LRB-

aligned

with

the

boundary

of

the

m-bit

group

-RRB-

Finally

the

two-input

XOR

gates

produce

the

memory

module

number

Figure

Implementation

of

an

address

generator

and

mapper

xx

A

b

z

x

mod

m

adder

mux

reg

mod

m

adder

mux

module

XOR

INIT

INC

XOR

-LRB-

-RRB-

displacement

am

initial

Cin

Cout

XOR

-LRB-

-RRB-

mapperAddress

generator

Conclusions

In

this

paper

we

have

considered

the

access

of

streams

of

fixed

length

equal

to

the

length

of

a

vector

register

in

the

case

of

a

vector

processor

or

of

a

fixed

portion

of

the

local

memory

-LRB-

for

block

transfers

-RRB-

The

access

patterns

correspond

to

constant

strides

and

the

stream

can

begin

in

any

address

The

basic

idea

we

propose

is

an

out-of-order

access

of

the

elements

of

the

stream

to

achieve

conflict-free

access

for

all

strides

that

produce

T-matched

streams

In

the

first

part

of

the

paper

we

present

an

address

mapping

and

an

out-of-order

access

method

to

achieve

conflict-free

access

for

a

large

number

of

families

We

first

consider

the

matched

memory

case

where

M

T

In

this

case

we

obtain

a

window

of

l-t

families

of

strides

that

are

conflict

free

whereas

previous

schemes

that

perform

the

access

in

order

result

in

a

single

conflictfree

family

-LRB-

for

streams

of

any

length

-RRB-

To

achieve

this

we

divide

the

stream

in

substreams

which

are

accessed

in

a

conflict-free

manner

This

by

itself

does

not

produce

conflict-free

access

to

the

whole

stream

although

the

added

latency

is

low

To

achieve

the

conflict-free

access

to

the

whole

stream

we

propose

that

an

additional

set

of

T

addresses

be

precalculated

and

latched

so

that

the

temporal

distribution

of

all

substreams

is

the

same

We

present

then

an

extension

of

this

scheme

for

the

unmatched

memory

system

The

case

for

M

-LRB-

m

t

-RRB-

has

been

considered

resulting

in

-LRB-

l-t

-RRB-

conflict-free

families

this

compares

favorably

to

the

t

conflict-free

families

obtained

using

access

in

order

The

method

can

be

applied

to

memory

systems

with

any

degree

of

unmatchness

-LRB-

m

k.t

k

-RRB-

As

stated

in

section

I.

more

conflict-free

families

can

be

obtained

at

a

cost

of

complicating

somewhat

the

substream

generation

for

the

additional

families

The

ideas

have

been

presented

using

an

address

mapping

based

on

linear

transformations

However

the

same

results

can

be

achieved

with

interleaving

or

with

skewing

In

these

cases

we

have

to

select

in

a

suitable

manner

the

bits

that

determine

the

module

number

in

the

interleaved

case

and

the

number

of

rows

to

rotate

for

skewing

The

difference

between

these

schemes

is

the

behavior

for

streams

of

length

smaller

than

L

and

for

streams

that

can

not

be

accessed

in

a

conflictfree

way

In

the

second

part

of

the

paper

we

present

an

address

mapping

and

an

out-of-order

access

method

to

achieve

conflict-free

stream

access

for

any

power-of-two

stride

which

includes

stride

one

and

any

initial

address

It

utilizes

a

linear

transformation

of

the

address

requires

an

unmatched

memory

system

with

M

T

memory

modules

and

performs

the

stream

access

out

of

order

In

both

cases

we

propose

the

use

of

two

address

generators

Moreover

for

the

case

of

powerof-two

strides

we

show

an

accessing

order

that

masks

the

time

for

address

calculation

and

mapping

and

the

bus

arbitration

This

allows

us

to

achieve

the

minimum

latency

of

T+L

cycles

for

a

stream

read

of

length

L

In

general

the

required

hardware

for

address

calculation

and

request

control

are

somewhat

more

complex

than

that

required

for

in-order

access

however

this

added

complexity

should

be

negligible

compared

to

that

of

the

whole

processor

We

plan

to

extend

this

work

to

the

case

in

which

several

streams

are

accessed

simultaneously

either

in

a

single

processor

with

several

memory

ports

or

in

a

SIMD

vector

multiprocessor

We

also

will

explore

further

the

use

of

these

techniques

for

scalar

processors

with

decoupled

memory

access

and

execute

units

Acknowledgements

This

work

has

been

supported

by

the

ESPRIT

III

Basic

Research

Action

-LRB-

APPARC

-RRB-

the

Ministry

of

Education

of

Spain

under

contract

and

by

the

CEPBA

-LRB-

European

Center

for

Parallelism

of

Barcelona

-RRB-

References

P.

Budnik

and

D.

J.

Kuck

The

Organization

and

Use

of

Parallel

Memories

IEEE

Trans

on

Computers

vol

no.

pp.

C-L

Chen

and

C-K

Liao

Analysis

of

Vector

Access

Performance

on

Skewed

Interleaved

Memory

Int?l

Symposium

on

Computer

Architecture

pp.

J.

Frailong

W.

Jalby

and

J.

Lenfant

XOR-schemes

A

Flexible

Data

Organization

in

Parallel

Memories

Int?l

Conference

on

Parallel

Processing

pp.

J.W.C.

Fu

and

J.H.

Patel

Data

Prefetching

in

Multiprocessor

Vector

Cache

Memories

Int

Symp

on

Computer

Architecture

pp

D.T.

Harper

III

and

J.R.

Jump

Performance

Evaluation

of

Vector

Accesses

in

Parallel

Memories

Using

a

Skewed

Storage

Scheme

Int?l

Symposium

on

Computer

Architecture

pp.

D.T.

Harper

III

Block

Multistride

Vector

and

FFT

Accesses

in

Parallel

Memory

Systems

IEEE

Trans

on

Parallel

and

Distributed

Systems

vol

no

pp.

D.T.

Harper

III

and

Y.

Costa

Analytical

Estimation

of

Vector

Access

Performance

in

Parallel

Memory

Architectures

Internal

Report

Dept.

of

Electrical

Engineering

The

University

of

Texas

at

Dallas

D.T.

Harper

III

and

D.

A.

Linebarger

Conflict-Free

Vector

Access

Using

a

Dynamic

Storage

Scheme

IEEE

Trans

on

Computers

vol

no.

pp.

D.H.

Lawrie

Access

and

Alignment

of

Data

in

an

Array

Processor

IEEE

Trans

on

Computers

vol

no.

pp.

Dec.

A.

Norton

and

E.

Melton

A

Class

of

Boolean

Linear

Transformations

for

Conflict-Free

Power-of-Two

Stride

Access

Int?l

Conference

on

Parallel

Processing

pp.

W.

Oed

and

O.

Lange

On

the

Effective

Bandwidth

of

Interleaved

Memories

in

Vector

Processing

Systems

IEEE

Trans

on

Computers

vol

no.

pp.

October

B.

R.

Rau

M.

S.

Schlansker

and

D.

W.

L.

Yen

The

Cydra

Stride-Insensitive

Memory

System

Int?l

Conference

on

Parallel

Processing

pp.

B.R.

Rau

Pseudo-Randomly

Interleaved

Memory

Int?l

Symposium

on

Computer

Architecture

pp

M.

Valero

T.

Lang

and

E.

Ayguad

Conflict-free

Access

of

Vectors

with

Power-of-two

Strides

Int?l

Conference

on

Supercomputing

pp.

M.

Valero

T.

Lang

J.M.

Llaberia

M.

Peiron

E.

Ayguad

and

J.J.

Navarro

Increasing

the

Number

of

Strides

for

Conflict-Free

Vector

Access

Int

Symp

on

Computer

Architecture

pp.

S.

Weiss

An

Aperiodic

Storage

Scheme

to

Reduce

Memory

Conflicts

in

Vector

Processors

Int?l

Symposium

on

Computer

Architecture

pp.

H.A.G.

Wijshoff

and

J.

van

Leeuwen

The

Structure

of

Periodic

Storage

Schemes

for

Parallel

Memories

IEEE

Trans

on

Computers

vol

pp.

June

decoupled access
storage schemes
vector processors

conflict-free access
multi-module memories
out-of-order access
streams with constant strides