Laboratoire

de

l'Informatique

du

Parall?lisme

Ecole

Normale

Sup?rieure

de

Lyon

Unit

de

recherche

associ?e

au

CNRS

n

Combining

retiming

and

scheduling

techniques

for

loop

parallelization

and

loop

tiling

Alain

Darte

Georges-Andr

e

Silber

Fr?ed?eric

Vivien

November

Research

Report

No

Ecole

Normale

Sup?rieure

de

Lyon

Adresse

lectronique

lip@lip.ens-lyon.fr

T?l?phone

-LRB-

-RRB-

T?l?copieur

-LRB-

-RRB-

All?e

d'Italie

Lyon

Cedex

France

Combining

retiming

and

scheduling

techniques

for

loop

parallelization

and

loop

tiling

Alain

Darte

Georges-Andr

e

Silber

Fr?ed?eric

Vivien

November

Abstract

Tiling

is

a

technique

used

for

exploiting

medium-grain

parallelism

in

nested

loops

It

relies

on

a

first

step

that

detects

sets

of

permutable

nested

loops

All

algorithms

developed

so

far

consider

the

statements

of

the

loop

body

as

a

single

block

in

other

words

they

are

not

able

to

take

advantage

of

the

structure

of

dependences

between

different

statements

In

this

report

we

overcome

this

limitation

by

showing

how

the

structure

of

the

reduced

dependence

graph

can

be

taken

into

account

for

detecting

more

permutable

loops

Our

method

combines

graph

retiming

techniques

and

graph

scheduling

techniques

It

can

be

viewed

as

an

extension

of

Wolf

and

Lam

's

algorithm

to

the

case

of

loops

with

multiple

statements

Loop

independent

dependences

play

a

particular

role

in

our

study

and

we

show

how

the

way

we

handle

them

can

be

useful

for

fine-grain

loop

parallelization

as

well

R?esum?e

Loop

tiling

est

une

technique

utilis?ee

pour

exploiter

du

parall?elisme

a

grain

moyen

dans

les

boucles

imbriqu?ees

Elle

repose

sur

une

premi?ere

etape

de

d?etection

de

boucles

permutables

Tous

les

algorithmes

d?evelopp?es

jusqu

a

maintenant

consid?eraient

les

instructions

du

corps

du

nid

de

boucles

comme

un

bloc

indissociable

En

d'autres

termes

ils

ne

pouvaient

pas

tirer

profit

de

la

structure

des

d?ependances

entre

diff?erentes

instructions

Dans

ce

rapport

nous

surmontons

cette

limitation

en

montrant

comment

la

structure

du

graphe

de

d?ependance

r?eduit

peut

etre

prise

en

compte

pour

d?etecter

plus

de

boucles

permutables

Notre

m?ethode

combine

des

techniques

de

synchronisation

et

d'ordonnancement

de

graphes

Elle

peut

etre

vue

comme

une

extension

de

l'algorithme

de

Wolf

et

Lam

au

cas

de

boucles

comportant

plusieurs

instructions

Les

d?ependances

qui

ne

sont

pas

port?ees

par

une

boucle

-LRB-

loop

independent

dependences

-RRB-

jouent

un

r

ole

particulier

dans

notre

etude

et

nous

montrons

comment

la

fa?con

particuli?ere

dont

nous

les

traitons

peut

etre

utile

egalement

pour

la

parall?elisation

a

grain

fin

Mots-cl

es

Parall?elisation

automatique

nids

de

boucles

boucles

permutables

tiling

grain

moyen

Combining

retiming

and

scheduling

techniques

for

loop

parallelization

and

loop

tiling

A.

Darte

G.-A

Silber

and

F.

Vivien

Laboratoire

LIP

URA

CNRS

Ecole

Normale

Sup?erieure

de

Lyon

LYON

Cedex

FRANCE

-LSB-

darte,gsilber,fvivien]@lip.ens-lyon.fr

Abstract

Tiling

is

a

technique

used

for

exploiting

mediumgrain

parallelism

in

nested

loops

It

relies

on

a

first

step

that

detects

sets

of

permutable

nested

loops

All

algorithms

developed

so

far

consider

the

statements

of

the

loop

body

as

a

single

block

in

other

words

they

are

not

able

to

take

advantage

of

the

structure

of

dependences

between

different

statements

In

this

paper

we

overcome

this

limitation

by

showing

how

the

structure

of

the

reduced

dependence

graph

can

be

taken

into

account

for

detecting

more

permutable

loops

Our

method

combines

graph

retiming

techniques

and

graph

scheduling

techniques

It

can

be

viewed

as

an

extension

of

Wolf

and

Lam

's

algorithm

to

the

case

of

loops

with

multiple

statements

Loop

independent

dependences

play

a

particular

role

in

our

study

and

we

show

how

the

way

we

handle

them

can

be

useful

for

fine-grain

loop

parallelization

as

well

Introduction

Affine

scheduling

techniques

from

the

simplest

and

earliest

one

Lamport

's

hyperplane

method

-LSB-

-RSB-

to

the

most

sophisticated

one

Feautrier

's

multi-dimensional

affine

scheduling

-LSB-

-RSB-

are

used

to

transform

a

set

of

nested

loops

into

a

semantically

equivalent

code

consisting

in

parallel

loops

surrounded

by

sequential

loops

Lamport

's

method

and

its

extension

the

linear

scheduling

transform

n

perfectly

nested

loops

into

n

nested

parallel

loops

surrounded

by

a

single

sequential

loop

When

this

is

not

feasible

multi-dimensional

scheduling

can

be

used

to

transform

the

original

loops

into

n

r

sequential

loops

surrounding

r

innermost

parallel

loops

with

n

r

The

goal

is

to

make

r

Supported

by

the

CNRS-INRIA

project

ReMaP

-LRB-

roughly

speaking

the

degree

of

parallelism

-RRB-

as

large

as

possible

The

underlying

computational

model

in

which

these

techniques

are

developed

is

nothing

but

a

PRAM

Additional

constraints

such

as

the

cost

of

communications

the

cost

of

synchronizations

the

number

of

processors

the

ratio

communications/computations

are

not

taken

into

account

The

claim

-LRB-

the

hope

-RRB-

is

that

they

can

be

optimized

a

posteriori

for

example

by

merging

virtual

PRAM

processors

into

fewer

physical

processors

However

especially

when

r

is

small

the

granularity

of

computations

can

be

too

fine

leading

to

poor

performances

especially

on

distributed

memory

systems

To

circumvent

this

problem

the

granularity

of

computations

has

to

be

increased

This

can

be

achieved

by

a

technique

called

tiling

introduced

by

Irigoin

and

Triolet

-LSB-

-RSB-

as

supernode

partitioning

Tiling

consists

in

aggregating

several

loop

iterations

that

will

be

considered

as

an

elemental

computation

The

size

and

shape

of

a

tile

are

chosen

following

various

criteria

for

achieving

better

vectorization

of

communications

and/or

computations

for

improving

cache-reuse

reducing

communications

etc.

All

these

criteria

are

very

machine-dependent

and

despite

the

large

amount

of

different

optimization

strategies

-LSB-

-RSB-

choosing

a

good

tiling

remains

an

open

problem

However

before

even

defining

the

size

and

shape

of

the

tiles

one

has

to

make

sure

that

they

will

be

atomic

i.e.

that

they

can

be

computed

with

no

intervening

synchronization

or

communication

This

atomicity

property

is

fulfilled

if

the

dependence

graph

between

tiles

is

acyclic

which

is

guaranteed

if

the

tiles

partition

the

iteration

domain

into

identical

rectangles

and

if

the

iteration

domain

is

described

by

permutable

loops

Until

now

all

algorithms

proposed

for

detecting

permutable

loops

have

the

following

restrictions

ffl

The

original

loops

are

perfectly

nested

ffl

The

dependences

are

uniform

except

for

Wolf

and

Lam

's

algorithm

-LSB-

-RSB-

where

dependences

can

be

approximated

by

direction

vectors

ffl

The

statements

of

the

loop

body

are

considered

as

a

single

block

This

may

enforce

complicated

skews

even

if

a

simple

shift

between

statements

is

sufficient

to

make

the

loops

permutable

Taking

into

account

the

structure

of

the

reduced

dependence

graph

has

been

proved

very

useful

for

the

detection

of

parallel

loops

see

for

example

the

algorithms

of

Allen

and

Kennedy

-LSB-

-RSB-

Darte

and

Vivien

-LSB-

-RSB-

or

Feautrier

-LSB-

-RSB-

In

this

paper

we

show

that

it

can

also

be

useful

for

the

detection

of

permutable

loops

Our

method

combines

graph

retiming

and

graph

scheduling

techniques

We

do

not

overcome

all

restrictions

listed

above

as

we

still

consider

only

perfectly

nested

loops

However

our

algorithm

can

be

applied

even

if

the

dependences

are

described

by

a

polyhedral

approximation

of

distance

vectors

-LRB-

which

is

more

general

than

direction

vectors

-RRB-

and

we

do

exploit

the

fact

that

the

loop

body

may

have

more

than

one

statement

i.e.

that

the

reduced

dependence

graph

may

have

more

than

one

vertex

The

paper

is

organized

as

follows

In

Section

we

explain

why

some

particular

structures

of

codes

can

not

be

obtained

by

standard

linear

scheduling

techniques

although

they

correspond

to

useful

optimizations

These

are

codes

containing

loop

independent

dependences

i.e.

codes

that

express

sequentiality

in

parallel

loops

In

Section

we

show

how

such

codes

can

be

generated

for

exploiting

fine-grain

parallelism

The

technique

is

to

modify

standard

scheduling

techniques

while

introducing

graph

retiming

techniques

In

Section

we

use

a

similar

combination

for

extending

Darte

and

Vivien

's

algorithm

-LSB-

-RSB-

-LRB-

first

designed

for

detecting

innermost

parallel

loops

i.e.

fine-grain

parallelism

-RRB-

to

the

detection

of

maximal

sets

of

permutable

loops

-LRB-

i.e.

medium-grain

parallelism

-RRB-

Finally

in

Section

we

summary

our

main

results

and

we

point

out

some

open

problems

Sequentiality

in

parallel

loops

Loops

parallelized

by

scheduling

techniques

have

a

particular

structure

each

statement

in

the

parallelized

code

is

surrounded

by

a

set

of

nested

parallel

loops

surrounded

by

a

set

of

sequential

loops

The

term

loop

is

said

parallel

if

it

carries

no

dependences

i.e.

if

there

is

no

dependences

between

different

iterations

of

the

loop

scheduling

comes

from

the

fact

that

the

outermost

sequential

loops

can

be

interpreted

as

a

description

of

the

time

steps

or

synchronization

steps

needed

for

computing

the

loops

in

a

PRAM

manner

The

innermost

parallel

loops

describe

the

set

of

computations

carried

at

a

given

time

step

By

construction

these

computations

are

completely

independent

each

dependence

is

carried

by

one

of

the

sequential

loops

Indeed

the

general

principle

is

to

transform

all

dependences

into

dependences

carried

by

the

outermost

loop

-LRB-

level

dependences

-RRB-

If

this

is

not

possible

as

many

dependences

as

possible

are

transformed

into

level

dependences

then

as

many

as

possible

into

level

dependences

and

so

on

until

all

dependences

are

carried

by

one

of

the

constructed

loops

-LRB-

which

are

therefore

sequential

-RRB-

The

remaining

dimensions

are

completely

independent

With

such

a

principle

the

final

code

never

contains

a

loop

independent

dependence

-LRB-

dependence

distance

-RRB-

A

consequence

of

this

restriction

is

that

some

code

structures

that

also

describe

fine-grain

parallelism

can

not

be

generated

We

illustrate

this

fact

on

the

following

code

structure

for

i

to

n

for

j

to

n

endfor

endfor

Suppose

that

we

succeeded

to

parallelize

the

above

code

with

the

scheduling

technique

called

shifted-linear

scheduling

This

means

that

we

have

found

an

integral

vector

X

-LRB-

a

b

-RRB-

and

two

constants

and

such

that

iteration

I

-LRB-

i

j

-RRB-

of

statement

-LRB-

resp

-RRB-

is

carried

-LRB-

in

the

PRAM

model

-RRB-

at

logical

time

X

I

ai

bj

-LRB-

resp

X

I

-RRB-

Forgetting

the

time

interpretation

this

simply

means

that

we

apply

a

loop

transformation

for

which

ai

bj

-LRB-

resp

ai

bj

-RRB-

is

the

new

loop

counter

for

the

first

loop

surrounding

-LRB-

resp

-RRB-

Now

two

main

cases

can

occur

ffl

The

components

of

X

are

relatively

prime

for

each

iteration

of

the

outermost

loop

-LRB-

corresponding

to

X

-RRB-

a

hyperplane

of

computations

can

be

carried

out

in

parallel

for

and

for

The

resulting

parallel

code

looks

like

X

may

be

chosen

with

rational

components

in

this

case

the

logical

time

is

b

X

I

aec

and

code

generation

may

involve

loop

unrolling

We

will

not

discuss

this

here

We

assume

all

along

the

paper

that

timing

vectors

such

as

X

are

integral

vectors

Code

of

type

-LRB-

a

-RRB-

forseq

forpar

endforpar

endforseq

possibly

with

some

guards

This

is

typically

the

case

if

X

-LRB-

-RRB-

and

for

any

and

All

dependences

are

carried

by

the

first

loop

and

potential

parallelism

between

and

is

exploited

ffl

The

components

of

X

are

not

relatively

prime

A

typical

example

is

X

-LRB-

-RRB-

odd

and

even

In

this

case

the

even

iterations

of

the

outermost

loop

correspond

to

iterations

of

and

the

odd

iterations

to

iterations

of

This

can

be

written

into

a

parallel

code

with

the

following

structure

Code

of

type

-LRB-

b

-RRB-

forseq

forpar

endforpar

forpar

endforpar

endforseq

possibly

with

some

guards

The

dependences

are

either

carried

by

the

first

loop

or

occur

between

the

first

and

the

second

parallel

loop

On

the

other

hand

with

standard

scheduling

techniques

it

is

not

possible

to

obtain

a

code

such

as

Code

of

type

-LRB-

c

-RRB-

forseq

forpar

endforpar

endforseq

which

may

contain

a

loop

independent

dependence

-LRB-

from

to

here

-RRB-

Yet

it

can

be

interesting

to

generate

such

a

code

for

several

reasons

ffl

If

parallelism

between

and

can

not

be

exploited

anyway

because

of

the

machine

programming

model

a

code

of

type

-LRB-

a

-RRB-

reveals

too

much

parallelism

This

is

the

case

for

example

for

a

parallelizer

that

generates

parallel

code

in

an

intermediate

language

such

as

HPF

and

expresses

parallel

loops

as

hpf

independent

directives

the

potential

parallelism

can

not

be

exploited

Instances

of

and

will

be

sequentialized

even

if

they

can

be

carried

out

in

parallel

In

this

case

a

code

of

type

-LRB-

c

-RRB-

is

sufficient

Of

course

any

code

of

type

-LRB-

a

-RRB-

can

be

sequentialized

into

a

code

of

type

-LRB-

c

-RRB-

However

all

codes

of

type

-LRB-

c

-RRB-

can

not

be

obtained

this

way

-LRB-

see

our

example

in

Section

-RRB-

ffl

A

code

of

type

-LRB-

c

-RRB-

can

lead

to

better

performance

than

a

code

of

type

-LRB-

b

-RRB-

when

the

minimization

of

communications

and/or

synchronizations

is

important

Indeed

for

a

code

of

type

-LRB-

b

-RRB-

a

synchronization

-LRB-

or

a

phase

of

communications

-RRB-

is

needed

between

the

two

parallel

loops

In

a

code

of

type

-LRB-

c

-RRB-

all

iterations

can

be

carried

in

parallel

and

possible

communications

from

to

take

place

inside

a

given

iteration

of

the

parallel

loop

This

principle

is

similar

to

the

one

used

in

Allen

and

Kennedy

's

algorithm

where

loop

fusion

-LRB-

more

precisely

partial

loop

distribution

-RRB-

is

shown

useful

to

minimize

synchronizations

ffl

Defining

loop

transformations

that

lead

to

codes

of

type

-LRB-

c

-RRB-

can

also

be

useful

for

enlarging

the

set

of

valid

schedules

and

having

more

flexibility

This

freedom

gives

us

a

better

control

on

the

code

shape

We

can

use

it

to

avoid

loop

skewing

when

it

is

not

necessary

to

keep

loops

perfectly

nested

if

possible

-LRB-

which

can

be

useful

for

tiling

-RRB-

to

impose

loop

transformations

to

be

unimodular

if

loop

strides

are

not

desirable

etc.

To

conclude

this

short

study

let

us

point

out

that

codes

of

type

-LRB-

c

-RRB-

can

be

obtained

simply

by

allowing

loop

independent

dependences

in

the

transformed

codes

We

now

show

that

this

can

be

done

by

combining

standard

scheduling

techniques

with

graph

retiming

techniques

linked

to

Bellman-Ford

's

algorithm

for

fine-grain

parallelism

detection

-LRB-

Section

-RRB-

as

well

as

for

medium-grain

parallelism

detection

-LRB-

Section

-RRB-

Notations

and

hypotheses

In

the

rest

of

the

paper

we

consider

n

perfectly

nested

loops

whose

dependences

are

represented

by

a

polyhedral

reduced

dependence

graph

-LRB-

PRDG

-RRB-

i.e.

where

set

of

distance

vectors

are

approximated

by

non

parameterized

polyhedra

As

shown

in

-LSB-

-RSB-

we

can

capture

such

dependences

through

a

modified

reduced

dependence

graph

with

edges

labeled

by

n-dimensional

integral

vectors

i.e.

a

uniform

dependence

graph

This

graph

has

particular

vertices

-LRB-

called

virtual

vertices

-RRB-

which

are

handled

in

a

special

way

However

to

make

the

discussion

simpler

we

will

forget

about

these

vertices

Taking

them

into

account

is

indeed

mainly

technical

but

does

not

bring

any

fundamental

difficulty

Going

back

from

the

modified

dependence

graph

to

the

original

dependence

graph

is

also

conceptually

not

difficult

See

-LSB-

-RSB-

for

a

complete

explanation

of

this

uniformization

process

The

only

important

point

is

that

the

vectors

that

label

the

edges

are

not

necessarily

lexicographically

positive

Therefore

in

the

rest

of

our

study

we

will

make

the

following

assumptions

ffl

The

reduced

dependence

graph

is

uniform

ffl

There

is

no

cycle

of

weight

ffl

Dependence

vectors

are

not

necessarily

lexicographically

positive

We

will

use

the

following

notations

G

is

the

reduced

dependence

graph

-LRB-

RDG

-RRB-

V

the

set

of

vertices

and

E

the

set

of

edges

V

-LRB-

resp

E

-RRB-

is

the

number

of

vertices

-LRB-

resp

edges

-RRB-

e

-LRB-

x

y

-RRB-

is

an

edge

of

G

from

vertex

x

to

vertex

y.

we

is

the

weight

of

e.

wC

denotes

the

weight

of

a

cycle

C

-LRB-

sum

of

the

weights

of

its

edges

-RRB-

and

lC

denotes

its

length

-LRB-

number

of

edges

-RRB-

Application

to

the

detection

of

fine

grain

parallelism

In

this

section

we

explain

how

a

technique

based

on

shifted-linear

scheduling

can

be

modified

so

as

to

allow

the

generation

of

codes

of

type

-LRB-

c

-RRB-

i.e.

codes

with

parallel

loops

and

sequential

bodies

As

recalled

in

Section

shifted-linear

scheduling

consists

in

defining

a

logical

time

for

computing

the

iterations

of

each

statement

S

iteration

I

of

S

is

scheduled

at

time

X

I

aeS

X

will

be

used

for

building

the

outermost

loop

of

the

parallelized

code

All

dependences

are

carried

by

this

loop

-LRB-

if

possible

-RRB-

The

two

following

lemmas

show

the

differences

between

the

pure

shifted-linear

approach

and

our

modified

approach

which

does

not

require

all

dependences

to

be

loop

carried

in

the

transformed

code

With

this

technique

we

can

find

as

much

parallelism

and

with

more

freedom

on

the

choice

of

X

The

constraints

on

X

given

by

Lemma

-LRB-

resp

Lemma

-RRB-

are

the

constraints

imposed

by

the

pure

shifted-linear

approach

-LRB-

resp

the

modified

one

-RRB-

Lemma

Let

G

be

a

RDG

Let

X

be

a

vector

which

induces

on

each

cycle

C

of

G

a

delay

greater

than

the

length

of

C

cycle

of

G

X

wC

lC

Then

for

each

vertex

v

V

there

exists

a

constant

aev

such

that

the

shifted

linear

schedule

built

from

X

and

the

constants

aev

is

valid

In

other

words

-LRB-

x

y

-RRB-

E

X

we

aey

aex

Proof

Let

F

be

a

copy

of

G

except

that

the

weight

of

an

edge

e

is

set

to

X

we

instead

of

we

Add

to

F

a

new

vertex

s

-LRB-

the

source

-RRB-

and

a

weight

edge

from

s

to

any

other

vertex

Let

C

be

a

cycle

of

F

By

hypothesis

the

weight

of

C

is

non

positive

P

-LRB-

X

we

-RRB-

lC

X

wC

Thus

we

can

successfully

apply

on

F

an

algorithm

to

find

the

longest

paths

from

s

-LRB-

e.g.

Bellman-Ford

's

algorithm

-LSB-

-RSB-

-RRB-

For

each

vertex

x

in

F

let

aex

be

the

length

of

the

longest

path

from

s

to

x.

For

each

edge

e

-LRB-

x

y

-RRB-

in

G

the

definition

of

the

longest

paths

leads

to

the

following

triangular

inequality

aey

aex

i.e.

X

we

aey

aex

Lemma

Let

G

be

a

RDG

Let

X

be

an

integral

vector

which

induces

on

each

cycle

C

of

G

a

delay

greater

than

one

cycle

of

G

X

wC

Then

for

each

vertex

v

V

there

exists

a

constant

aev

such

that

-LRB-

x

y

-RRB-

E

X

we

aey

aex

Furthermore

the

subgraph

generated

by

the

edges

with

delay

is

acyclic

Proof

Let

F

be

a

copy

of

G

except

that

the

weight

of

the

edge

e

is

set

to

X

we

instead

of

we

Add

to

F

a

new

vertex

s

-LRB-

the

source

-RRB-

and

a

weight

edge

from

s

to

any

other

vertex

Let

C

be

a

cycle

of

F

By

hypothesis

the

weight

of

C

is

non

positive

P

-LRB-

X

we

-RRB-

X

wC

Thus

we

can

successfully

apply

on

F

an

algorithm

to

find

the

longest

paths

from

s.

For

each

vertex

x

of

F

let

aex

be

the

longest

path

from

s

to

x

As

the

edge

weights

and

X

are

integers

so

are

the

longest

paths

For

each

edge

e

-LRB-

x

y

-RRB-

of

G

we

have

by

definition

of

the

longest

paths

aey

aex

i.e.

X

we

aey

aex

Now

if

all

edges

of

a

cycle

C

have

a

delay

i.e.

X

we

aey

aex

then

X

wC

which

contradicts

the

hypothesis

Thus

the

subgraph

generated

by

the

edges

of

delay

is

acyclic

Note

that

the

number

of

elementary

cycles

in

a

graph

can

be

exponential

in

the

number

of

vertices

and

edges

of

the

graph

Therefore

checking

directly

that

X

wC

or

X

wC

lC

for

all

elementary

cycles

can

be

exponential

even

if

in

practice

it

can

be

fast

when

the

number

of

cycles

is

small

However

Lemma

shows

that

finding

a

X

such

that

X

wC

lC

for

all

cycles

is

equivalent

to

solving

E

inequalities

with

V

additional

variables

-LRB-

the

constants

aev

-RRB-

thus

a

polynomial

number

of

inequalities

Expressing

the

constraints

X

wC

with

a

polynomial

number

of

inequalities

and

variables

is

more

tricky

but

feasible

With

the

example

hereafter

we

illustrate

the

differences

between

the

three

following

techniques

linear

schedule

shifted-linear

schedule

and

shifted-linear

schedule

allowing

loop

independent

dependences

for

i

to

N

for

j

to

N

a

-LRB-

i

j

-RRB-

b

-LRB-

j

-RRB-

b

-LRB-

i

j

-RRB-

a

-LRB-

i

-RRB-

endfor

endfor

The

reduced

dependence

graph

of

this

program

is

depicted

on

Figure

S

S

Figure

RDG

of

the

orst

example

Linear

schedule

We

look

for

a

vector

X

such

that

X

w

for

each

dependence

vector

w

in

the

RDG

Because

of

the

values

of

the

two

dependences

both

components

of

X

must

be

greater

than

one

Hence

we

have

to

do

at

least

one

loop

skewing

We

choose

X

-LRB-

-RRB-

and

we

complete

it

into

a

unimodular

matrix

with

the

vector

-LRB-

-RRB-

After

transformation

we

obtain

the

following

code

forseq

i

to

forall

j

max

-LRB-

i-N

-RRB-

to

min

-LRB-

N

-RRB-

a

-LRB-

i-j

j

-RRB-

b

-LRB-

j

-RRB-

b

-LRB-

i-j

j

-RRB-

a

-LRB-

i-j

-RRB-

endforall

endforseq

Shifted

linear

schedule

We

look

here

for

a

vector

X

such

that

X

wC

lC

for

each

cycle

C

in

the

RDG

The

only

cycle

weight

is

-LRB-

-RRB-

of

length

thus

we

can

choose

X

-LRB-

-RRB-

Using

Lemma

we

find

two

constants

and

to

complete

the

schedule

Once

again

we

complete

X

into

a

unimodular

matrix

using

the

vector

-LRB-

-RRB-

After

transformation

we

obtain

the

following

code

of

type

-LRB-

b

-RRB-

forseq

i

to

N

forall

j

to

N

a

-LRB-

i

j

-RRB-

b

-LRB-

j

-RRB-

endforall

forall

j

to

N

b

-LRB-

i

j

-RRB-

a

-LRB-

i

-RRB-

endforall

endforseq

Shifted

linear

schedule

allowing

loop

independent

dependences

We

look

here

for

a

vector

X

such

that

X

wC

for

each

cycle

C

in

the

RDG

Using

Lemma

we

will

find

some

constants

to

complete

the

schedule

This

schedule

is

said

to

be

allowing

loop

independent

dependences

because

it

does

not

induce

a

delay

greater

than

one

on

all

the

edges

as

usually

required

but

only

a

non

negative

delay

Since

all

values

are

integers

a

delay

is

either

greater

than

one

or

If

a

delay

is

equal

to

zero

the

dependence

will

only

be

satisfied

by

the

ordering

of

the

statements

in

the

loop

body

This

can

be

achieved

through

a

topological

ordering

of

the

subgraph

generated

by

the

edges

with

delay

since

this

subgraph

is

acyclic

-LRB-

cf

Lemma

-RRB-

Such

a

dependence

will

finally

be

transformed

into

a

loop

independent

dependence

In

our

example

we

can

choose

X

equal

to

-LRB-

-RRB-

and

both

constants

to

be

But

then

must

precede

in

the

final

loop

body

as

the

delay

on

the

edge

from

to

is

Furthermore

constants

in

the

remaining

dimension

must

be

carefully

chosen

so

that

the

dependences

from

to

is

not

carried

-LRB-

otherwise

the

remaining

loop

will

be

sequential

-RRB-

Once

again

we

complete

X

into

a

unimodular

matrix

using

the

vector

-LRB-

-RRB-

For

the

second

dimension

we

choose

and

After

transformation

we

obtain

the

following

code

of

type

-LRB-

c

-RRB-

forseq

i

to

N

forall

j

to

N

if

-LRB-

j

-RRB-

then

a

-LRB-

i

-RRB-

b

-LRB-

-RRB-

if

-LRB-

j

N

-RRB-

then

b

-LRB-

i

j

-RRB-

a

-LRB-

i

-RRB-

endforall

endforseq

This

technique

completely

solves

the

problem

stated

by

Okuda

in

-LSB-

-RSB-

in

a

uniform

nested

loops

shift

statements

before

searching

a

schedule

so

that

the

latency

of

the

best

linear

schedule

is

minimized

This

can

be

done

simply

by

minimizing

the

latency

induced

by

a

vector

X

subject

to

the

constraints

of

Lemma

Detecting

fully

permutable

loops

Consider

the

following

piece

of

code

whose

dependence

graph

is

depicted

in

Figure

for

i

to

N

for

j

to

N

a

-LRB-

i

j

-RRB-

b

-LRB-

i

-RRB-

a

-LRB-

i

-RRB-

b

-LRB-

i

j

-RRB-

a

-LRB-

j

-RRB-

b

-LRB-

j

-RRB-

endfor

endfor

It

is

a

uniform

program

with

four

dependence

vectors

Fine-grain

parallelism

detection

will

lead

to

a

code

with

one

sequential

and

one

parallel

loop

which

may

not

be

sufficient

for

achieving

good

performance

on

distributed

memorymachines

To

increase

the

granularity

of

computations

we

can

use

the

tiling

technique

introduced

by

Irigoin

and

Triolet

-LSB-

-RSB-

by

first

transforming

the

original

loops

into

permutable

loops

The

condition

of

permutability

is

easy

to

check

two

consecutive

loops

are

permutable

if

and

only

if

all

dependence

vectors

not

carried

by

outermost

loops

have

non

negative

components

in

these

dimensions

S

S

Figure

RDG

of

the

second

example

The

technique

is

as

follows

a

non

singular

integral

transformation

matrix

H

is

generated

such

that

H

we

for

each

dependence

vector

we

In

particular

each

row

X

-LRB-

a

b

-RRB-

of

H

satisfies

X

we

This

leads

to

the

following

constraints

a

b

a

b

b

Here

the

simplest

linear

independent

solutions

are

-LRB-

-RRB-

and

-LRB-

-RRB-

H

is

a

matrix

for

performing

a

loop

skewing

The

corresponding

permutable

code

in

which

tiling

can

be

achieved

is

the

following

for

i

to

N

for

j

i

to

N

i

a

-LRB-

i

j

i

-RRB-

b

-LRB-

i

j

-RRB-

a

-LRB-

i

j

-RRB-

b

-LRB-

i

j

i

-RRB-

a

-LRB-

j

i

-RRB-

b

-LRB-

j

i

-RRB-

endfor

endfor

Now

let

us

use

the

same

technique

as

in

Section

so

as

to

exploit

the

structure

of

the

dependence

graph

Instead

of

transforming

each

iteration

vector

I

-LRB-

i

j

-RRB-

into

I

H

I

for

all

statements

we

allow

statements

to

be

shifted

between

each

other

In

other

words

we

transform

iteration

I

of

statement

S

into

H

I

aeS

where

aeS

is

a

shift

vector

possibly

different

for

each

statement

S.

For

the

new

loops

to

be

permutable

the

constraints

are

now

that

for

each

edge

e

-LRB-

x

y

-RRB-

of

the

graph

H

-LRB-

I

we

-RRB-

aey

H

I

aex

i.e.

H

we

aey

aex

Reasoning

row

by

row

it

means

that

for

each

row

X

-LRB-

a

b

-RRB-

of

H

there

are

constants

and

such

that

a

b

a

b

b

Here

the

simplest

linear

independent

solutions

are

-LRB-

-RRB-

-LRB-

with

-RRB-

and

-LRB-

-RRB-

-LRB-

with

and

-RRB-

H

is

simply

the

identity

matrix

and

is

moved

forward

one

iteration

along

the

j

loop

The

corresponding

permutable

code

in

which

tiling

can

be

achieved

is

the

following

for

i

to

N

for

j

to

N

if

-LRB-

j

-RRB-

then

b

-LRB-

i

-RRB-

a

-LRB-

j

-RRB-

b

-LRB-

-RRB-

if

-LRB-

j

N

-RRB-

then

a

-LRB-

i

j

-RRB-

b

-LRB-

i

-RRB-

a

-LRB-

i

-RRB-

endfor

endfor

Remark

that

we

interchanged

and

in

the

loop

body

This

is

because

after

transformation

all

dependence

vectors

are

now

non

negative

and

some

of

them

can

even

be

-LRB-

loop

independent

dependences

-RRB-

To

keep

the

semantic

of

the

code

we

have

to

order

the

statements

inside

the

loop

body

so

that

loop

independent

dependences

follow

the

textual

order

For

this

to

be

possible

we

have

to

make

sure

that

the

subgraph

of

G

generated

by

loop

independent

dependences

is

acyclic

Once

again

the

technique

is

related

to

Lemma

The

main

difference

with

Section

is

that

for

tiling

we

are

looking

for

a

family

of

independent

vectors

X

-LRB-

and

not

only

for

one

vector

X

-RRB-

that

form

a

matrix

H

of

full

rank

The

condition

on

the

weights

of

the

cycles

wC

given

in

Lemma

is

now

too

strong

What

we

need

is

H

wC

and

H

wC

It

is

now

possible

that

one

of

the

rows

X

of

H

satisfies

X

wC

as

long

as

at

least

one

of

the

other

rows

satisfies

X

wC

We

are

now

ready

to

generalize

this

technique

to

arbitrary

reduced

dependence

graphs

with

uniform

-LRB-

but

not

necessarily

lexicographically

positive

-RRB-

dependences

as

long

as

the

graph

has

no

cycle

of

weight

We

combine

two

ideas

ffl

Wolf

and

Lam

's

idea

-LSB-

-RSB-

that

a

set

of

perfectly

nested

loops

can

be

transformed

by

unimodular

transformations

into

a

canonical

form

consisting

of

nested

blocks

of

fully

permutable

loops

The

technique

is

greedy

and

recursive

First

as

many

outermost

permutable

loops

as

possible

are

generated

All

dependences

have

now

non

negative

components

in

these

dimensions

Some

of

them

have

at

least

one

positive

component

they

are

carried

by

at

least

one

loop

and

are

not

considered

any

longer

The

other

ones

are

taken

into

account

for

building

a

new

block

of

permutable

loops

This

recursive

procedure

ends

when

all

dependences

are

finally

carried

by

at

least

one

of

the

generated

loops

ffl

Darte

and

Vivien

's

idea

-LSB-

-RSB-

that

fine-grain

parallelism

can

be

detected

by

uniformizing

the

polyhedral

reduced

dependence

graph

into

the

dependence

graph

of

a

system

of

uniform

recurrence

equations

which

can

be

scheduled

The

technique

is

also

greedy

and

recursive

First

an

outermost

loop

is

generated

that

carries

as

many

dependences

as

possible

possibly

after

shifting

the

different

statements

between

each

other

Then

all

carried

dependences

are

removed

from

the

graph

The

procedure

keeps

going

on

each

strongly

connected

component

of

the

remaining

graph

-LRB-

called

-RRB-

and

the

recursive

procedure

ends

when

all

dependences

are

finally

carried

by

at

least

one

of

the

generated

loops

We

mixed

these

two

approaches

we

aim

at

finding

a

nested

structure

of

blocks

of

permutable

loops

as

in

Wolf

and

Lam

's

algorithm

but

we

exploit

the

structure

of

the

reduced

dependence

graph

as

in

Darte

and

Vivien

's

algorithm

by

allowing

shifts

between

statements

Each

statement

S

is

transformed

by

a

multidimensional

affine

function

iteration

I

of

S

is

represented

by

the

new

iteration

vector

I

HS

I

aeS

where

HS

is

a

non

singular

n-by-n

matrix

Following

the

technique

used

in

-LSB-

-RSB-

-LRB-

called

shifted-linear

multidimensional

schedules

-RRB-

we

look

for

transformation

matrices

HS

whose

first

rows

-LRB-

as

many

as

possible

-RRB-

are

the

same

for

all

statements

within

a

given

strongly

connected

component

of

G

After

transformation

the

first

common

r

rows

of

the

matrices

HS

correspond

to

r

permutable

loops

if

-LRB-

x

y

-RRB-

G

M

we

aey

aex

-LRB-

-RRB-

loop

independent

dependences

of

the

original

loops

they

are

not

taken

into

account

and

they

remain

unchanged

a

restriction

keeps

optimality

for

maximal

parallel

loops

detection

in

polyhedral

reduced

dependence

graphs

-LRB-

PRDG

-RRB-

we

conjecture

it

is

also

true

for

maximal

permutable

loops

detection

where

M

is

the

r-by-n

matrix

of

full

rank

formed

by

these

row

vectors

Our

goal

is

to

build

such

a

matrix

M

while

maximizing

r.

Of

course

M

defines

only

one

part

of

the

final

transformation

To

be

valid

the

final

transformation

has

to

respect

all

dependences

Some

of

them

are

already

carried

by

the

loops

defined

by

M

The

other

ones

corresponding

to

edges

e

-LRB-

x

y

-RRB-

such

that

M

we

aey

aex

will

be

satisfied

either

by

a

topological

sort

as

in

Section

or

recursively

in

the

subsequent

dimensions

For

the

sake

of

clarity

we

only

focus

on

the

construction

of

the

outermost

block

of

permutable

loops

We

will

explain

briefly

at

the

end

of

the

section

how

to

adapt

this

study

to

the

whole

recursive

construction

Our

problem

is

therefore

the

following

build

a

full

maximal

rank

matrix

M

-LRB-

and

its

corresponding

vectors

aev

-RRB-

that

can

be

extended

to

a

n-dimensional

valid

transformation

Condition

is

a

necessary

condition

expressed

in

terms

of

edges

It

can

be

reformulated

as

a

necessary

condition

on

cycles

Lemma

-LRB-

Condition

on

cycles

-RRB-

Let

M

be

a

matrix

M

satisfies

Condition

for

some

vectors

aev

v

V

if

and

only

if

M

wC

for

each

cycle

C

of

G.

Proof

The

proof

is

similar

to

the

proofs

of

Lemmas

and

by

reasoning

on

each

row

of

M

We

now

show

the

fundamental

role

of

the

subgraph

of

G

generated

by

the

multi-cycles

-LRB-

union

of

cycles

-RRB-

of

weight

i.e.

the

subgraph

generated

by

the

edges

of

G

that

belong

to

a

multi-cycle

of

weight

We

point

out

that

is

also

the

base

of

Karp

Miller

and

Winograd

's

decomposition

-LSB-

-RSB-

for

the

computability

of

systems

of

uniform

recurrence

equations

and

of

Darte

and

Vivien

's

algorithm

-LSB-

-RSB-

for

the

detection

of

parallelism

in

PRDGs

can

be

built

by

rational

linear

programming

with

a

polynomial

number

of

constraints

and

variables

-LRB-

see

-LSB-

-RSB-

-RRB-

Lemma

-LRB-

Condition

on

edges

-RRB-

Condition

is

equivalent

to

-LRB-

i

-RRB-

M

we

aey

aex

-LRB-

ii

-RRB-

M

we

aey

aex

Proof

Let

C

be

a

multi-cycle

of

weight

wC

thus

M

wC

Therefore

X

M

we

aey

aex

M

wC

X

-LRB-

aey

aex

-RRB-

The

left-hand

side

of

the

above

equation

is

a

sum

of

non

negative

terms

-LRB-

M

we

aey

aex

-RRB-

thus

is

a

sum

of

terms

The

matrix

M

is

composed

by

r

row

vectors

Xi

with

i

r

Our

goal

is

to

maximize

r.

Let

U

be

the

vector

space

generated

by

the

weights

of

the

cycles

of

Let

k

be

the

dimension

of

U

Lemma

-LRB-

Xi

U

-RRB-

cycle

C

M

wC

In

other

words

each

Xi

is

in

the

orthogonal

of

U

Thus

r

n

k.

Proof

If

C

is

a

cycle

of

all

its

edges

belong

to

By

Lemma

M

wC

is

a

sum

of

terms

and

thus

is

We

now

show

that

in

fact

r

equals

n

k.

Lemma

-LRB-

Vect

-LRB-

Xi

-RRB-

U

-RRB-

The

rows

of

M

form

a

basis

of

U

i.e.

r

n

k

We

first

give

an

existence

proof

of

the

n

k

vectors

Xi

Then

we

will

discuss

their

construction

from

an

algorithmic

point

of

view

Proof

We

use

a

well-known

property

of

-LRB-

see

-LSB-

-RSB-

-RRB-

There

exists

a

vector

and

some

constants

v

v

V

such

that

-LRB-

i

-RRB-

we

y

x

-LRB-

ii

-RRB-

we

y

x

This

can

be

proved

as

follows

There

is

no

multi-cycle

of

weight

which

contains

an

edge

not

in

This

property

can

be

expressed

by

the

fact

that

some

system

of

linear

equations

has

no

solution

Then

using

Farkas

lemma

-LSB-

-RSB-

we

obtain

the

existence

of

and

of

the

desired

constants

In

particular

is

such

that

wC

if

C

and

wC

otherwise

Now

consider

a

basis

bn?k

of

U

Let

B

be

the

n

-LRB-

n

k

-RRB-

matrix

whose

columns

are

the

bi

We

look

for

vectors

Xi

of

the

form

-LRB-

bi

i

-RRB-

According

to

Lemma

we

now

have

to

determine

the

i

such

that

Xi

wC

for

each

cycle

C

of

G

and

such

that

the

vectors

Xi

are

linearly

independent

We

first

give

a

condition

on

the

i

for

the

Xi

to

be

linearly

independent

is

in

the

orthogonal

of

U

too

it

is

therefore

a

linear

combination

of

the

vectors

bi

Pn?k

i

yi

bi

Let

be

the

matrix

of

size

-LRB-

n?k

-RRB-

with

components

the

i

Y

the

matrix

of

size

-LRB-

n?k

-RRB-

with

components

the

yi

and

write

as

a

matrix

X

of

size

n

Then

X

BY

and

tM

B

X

B

-LRB-

In?k

Y

-RRB-

Since

B

is

of

full

rank

M

is

of

full

rank

if

and

only

if

the

matrix

In?k

Y

which

is

a

square

matrix

of

size

n

k

is

non

singular

Actually

this

matrix

is

the

change

of

basis

from

B

to

tM

We

can

show

that

its

determinant

is

equal

to

Y

i.e.

Pn?k

i

yi

i.

To

summarize

the

vectors

Xi

are

linearly

independent

if

and

only

if

n?k

X

i

yi

i

-LRB-

-RRB-

We

now

check

Condition

using

Lemma

Xi

wC

bi

wC

i

wC

If

C

is

a

cycle

of

then

Xi

wC

whatever

i

If

C

is

an

elementary

cycle

with

at

least

one

edge

not

in

then

wC

Therefore

it

is

sufficient

to

choose

i

sufficiently

large

i.e.

larger

than

-LRB-

bi

wC

-RRB-

=-LRB-

wC

-RRB-

If

C

is

any

cycle

it

is

sum

of

elementary

cycles

and

the

desired

inequality

is

automatically

satisfied

if

already

satisfied

for

all

elementary

cycles

This

proves

the

existence

of

the

i

we

choose

them

large

enough

while

checking

Equation

-LRB-

-RRB-

There

is

an

infinite

number

of

matrices

M

of

rank

n

k

satisfying

M

wC

for

each

cycle

C

of

G.

To

build

one

of

them

we

have

two

possibilities

On

one

hand

if

the

number

of

elementary

cycles

is

small

we

can

directly

work

with

the

cone

generated

by

the

weights

of

the

cycles

of

G

The

corresponding

polar

cone

contains

all

candidate

vectors

X.

Then

to

select

the

matrix

M

optimization

techniques

such

as

in

-LSB-

-RSB-

can

be

used

On

the

other

hand

if

generating

all

the

elementary

cycles

is

too

expensive

we

can

still

build

one

solution

in

polynomial

time

by

choosing

M

as

done

in

the

proof

of

Lemma

First

we

find

a

basis

B

of

U

For

that

we

build

the

weights

of

a

basis

of

cycles

of

which

can

be

done

in

polynomial

time

Since

is

a

union

of

strongly

connected

components

we

can

show

that

these

vectors

span

exactly

the

vector

space

U

Now

using

U

we

build

the

basis

B

of

U

Then

we

build

a

vector

by

linear

programming

techniques

Finally

we

choose

the

smallest

i

as

stated

in

the

proof

of

Lemma

As

already

noticed

no

matter

how

M

is

completed

into

a

square

matrix

of

size

n

each

dependence

that

corresponds

to

an

edge

e

-LRB-

x

y

-RRB-

such

that

M

we

aey

aex

and

M

we

aey

aex

will

be

satisfied

as

already

carried

by

one

of

the

loops

corresponding

to

M

We

still

have

to

consider

the

other

edges

those

such

that

M

we

aey

aex

We

show

how

we

can

satisfy

them

recursively

We

need

the

following

lemmas

Lemma

For

each

cycle

C

of

G

wC

U

C

Proof

-LRB-

is

true

by

definition

of

U

Conversely

let

C

be

a

cycle

not

in

Consider

again

the

vector

introduced

in

the

proof

of

Lemma

belongs

to

U

and

is

such

that

wC

Therefore

wC

is

not

in

U

-LRB-

otherwise

wC

-RRB-

Lemma

If

M

is

such

that

M

wC

for

each

cycle

C

of

G

and

if

M

is

of

full

and

maximal

rank

then

for

each

cycle

C

not

in

we

have

M

wC

and

M

wC

Proof

For

each

cycle

C

of

G

we

have

M

wC

If

M

wC

then

wC

is

orthogonal

to

all

rows

of

M

If

M

is

of

full

and

maximal

rank

its

rows

generate

exactly

U

-LRB-

cf

Lemma

-RRB-

Therefore

wC

-LRB-

U

-RRB-

U

Then

Lemma

shows

that

C

We

are

now

able

to

characterize

precisely

the

matrices

M

which

enable

us

to

build

a

maximal

set

of

fully

permutable

loops

Lemmas

and

show

that

they

are

the

matrices

M

of

full

and

maximal

rank

such

that

-LRB-

i

-RRB-

for

each

cycle

C

M

wC

-LRB-

ii

-RRB-

for

each

cycle

C

M

wC

We

can

also

characterize

the

matrices

M

by

conditions

on

edges

They

are

the

matrices

M

of

full

and

maximal

rank

satisfying

both

following

properties

there

exist

some

vectors

aev

v

V

such

that

-LRB-

i

-RRB-

M

we

aey

aex

-LRB-

ii

-RRB-

M

we

aey

aex

the

subgraph

G

of

G

generated

by

the

edges

e

-LRB-

x

y

-RRB-

for

which

M

we

aey

aex

is

a

forest

of

strongly

connected

components

and

those

with

at

least

one

edge

are

exactly

the

strongly

connected

components

of

The

characterization

above

leads

to

a

recursive

construction

of

the

whole

n-dimensional

transformation

As

said

before

each

edge

not

in

G

is

carried

by

one

of

the

loops

corresponding

to

M

Edges

in

G

but

not

in

can

be

satisfied

by

a

topological

ordering

of

the

strongly

connected

components

of

G

Finally

edges

in

will

be

satisfied

through

the

recursive

processing

of

the

strongly

connected

components

of

which

completes

the

matrix

M

already

built

The

construction

of

the

new

rows

of

M

-LRB-

which

may

be

different

for

each

strongly

connected

component

of

-RRB-

is

done

the

same

way

The

only

difference

is

that

they

must

be

chosen

with

an

additional

constraint

they

have

to

be

linearly

independent

with

the

existing

rows

of

M

As

in

-LSB-

-RSB-

the

correctness

of

this

recursive

algorithm

comes

from

the

fact

that

G

has

no

cycles

of

weight

From

a

practical

point

of

view

we

point

out

that

all

statements

do

not

necessarily

have

the

same

final

n-dimensional

matrix

M

However

we

can

impose

these

matrices

to

be

unimodular

so

as

to

get

simpler

codes

We

illustrate

our

technique

on

the

following

code

for

i

to

N

for

j

to

N

for

k

to

N

a

-LRB-

i

j

k

-RRB-

a

-LRB-

i

k

-RRB-

b

-LRB-

i

j

-RRB-

b

-LRB-

i

j

k

-RRB-

b

-LRB-

j

i

k

-RRB-

a

-LRB-

i

k

j

-RRB-

endfor

endfor

endfor

Figure

shows

the

reduced

dependence

graph

with

direction

vectors

Figure

shows

the

uniformized

dependence

graph

G.

Figure

RDG

for

third

example

Figure

Uniformized

RDG

-LRB-

Example

-RRB-

G

has

five

cycles

three

self-dependences

with

weights

-LRB-

-RRB-

-LRB-

-RRB-

and

-LRB-

-RRB-

and

two

other

cycles

with

weights

-LRB-

-RRB-

and

-LRB-

-RRB-

Therefore

is

the

graph

generated

by

the

cycles

whose

weights

are

-LRB-

-RRB-

and

-LRB-

-RRB-

The

dimension

of

U

is

We

can

thus

build

outermost

permutable

loops

Here

we

see

directly

that

the

two

canonical

vectors

-LRB-

-RRB-

and

-LRB-

-RRB-

belong

to

U

and

that

they

satisfy

Xi

wC

for

all

other

cycles

Thus

we

can

choose

them

as

the

rows

of

M

We

just

have

to

find

the

corresponding

shift

vectors

We

get

-LRB-

-RRB-

and

-LRB-

-RRB-

Here

G

no

topological

sort

is

required

We

consider

the

strongly

connected

component

that

contains

and

and

we

look

for

a

vector

linearly

independent

with

and

such

that

-LRB-

-RRB-

e.g.

-LRB-

-RRB-

No

shift

in

this

dimension

is

required

however

has

to

be

textually

ordered

before

To

rewrite

the

code

we

use

the

function

codegen

of

the

software

Petit

-LSB-

-RSB-

Our

final

transformation

can

be

expressed

in

Petit

's

framework

as

-LRB-

i

j

k

-RRB-

-LRB-

i

k

j

-RRB-

and

-LRB-

i

j

k

-RRB-

-LRB-

i

k

j

-RRB-

and

we

get

the

code

for

i

to

N

for

k

N

to

for

j

to

N

if

-LRB-

k

-RRB-

then

b

-LRB-

i

j

k

-RRB-

b

-LRB-

j

i

k

-RRB-

a

-LRB-

i

k

j

-RRB-

if

-LRB-

k

N

-RRB-

then

a

-LRB-

i

j

k

-RRB-

a

-LRB-

i

k

-RRB-

b

-LRB-

i

j

k

-RRB-

endfor

endfor

endfor

in

which

tiling

can

be

performed

on

the

two

outermost

loops

Note

that

in

this

example

permutable

loops

can

not

be

detected

by

Wolf

and

Lam

's

algorithm

Conclusion

In

this

paper

we

enlarge

the

set

of

codes

that

can

be

generated

by

standard

linear

scheduling

techniques

and

that

expose

either

parallel

loops

or

permutable

loops

Our

method

exploits

the

structure

of

the

dependence

graph

by

combining

graph

retiming

and

scheduling

techniques

For

fine-grain

parallelism

detection

we

are

now

able

to

generate

codes

with

parallel

loops

that

contain

loop

independent

dependences

This

can

be

useful

for

minimizing

communications

and/or

synchronizations

For

medium-grain

parallelism

detection

we

generalize

Wolf

and

Lam

's

algorithm

to

the

case

of

loops

with

multiple

statements

We

generate

maximal

sets

of

fully

permutable

loops

that

are

essential

for

tiling

We

still

have

some

open

problems

how

to

define

a

criterion

of

optimality

for

the

detection

of

permutable

loops

How

to

handle

non

perfectly

nested

loops

How

to

choose

the

size

and

shape

of

a

tile

How

to

map

data

with

respect

to

the

chosen

tiling

Our

future

work

will

address

these

problems

References

-LSB-

-RSB-

J.

Allen

and

K.

Kennedy

Automatic

translations

of

Fortran

programs

to

vector

form

ACM

Toplas

-LCB-

-LSB-

-RSB-

P.

Boulet

A.

Darte

T.

Risset

and

Y.

Robert

-LRB-

pen

-RRB-

ultimate

tiling

Integration

the

VLSI

Journal

-LCB-

-LSB-

-RSB-

T.

H.

Cormen

C.

E.

Leiserson

and

R.

L.

Rivest

Introduction

to

Algorithms

The

MIT

Press

-LSB-

-RSB-

A.

Darte

and

F.

Vivien

Revisiting

the

decomposition

of

Karp

Miller

and

Winograd

Parallel

Processing

Letters

-LRB-

-RRB-

-LCB-

Dec.

-LSB-

-RSB-

A.

Darte

and

F.

Vivien

Optimal

fine

and

medium

grain

parallelism

in

polyhedral

reduced

dependence

graphs

In

Proceedings

of

PACT

Boston

MA

Oct.

IEEE

Computer

Society

Press

To

appear

-LSB-

-RSB-

A.

Darte

and

F.

Vivien

Optimal

fine

and

medium

grain

parallelism

in

polyhedral

reduced

dependence

graphs

Technical

Report

LIP

ENS-Lyon

France

Apr.

-LSB-

-RSB-

P.

Feautrier

Some

efficient

solutions

to

the

affine

scheduling

problem

part

II

multi-dimensional

time

Int

J.

Parallel

Programming

-LRB-

-RRB-

-LCB-

Dec.

-LSB-

-RSB-

F.

Gasperoni

and

U.

Schwiegelshohn

Generating

close

to

optimum

loop

schedules

on

parallel

processors

Parallel

Processing

Letters

-LRB-

-RRB-

-LCB-

-LSB-

-RSB-

F.

Irigoin

and

R.

Triolet

Supernode

partitioning

In

Annual

ACM

Symp

Principles

of

Programming

Languages

pages

-LCB-

San

Diego

CA

Jan.

-LSB-

-RSB-

R.

Karp

R.

Miller

and

S.

Winograd

The

organization

of

computations

for

uniform

recurrence

equations

Journal

of

the

ACM

-LRB-

-RRB-

-LCB-

July

-LSB-

-RSB-

W.

Kelly

V.

Maslov

W.

Pugh

E.

Rosser

T.

Shpeisman

and

D.

Wonnacott

New

user

interface

for

Petit

and

other

interfaces

user

guide

University

of

Maryland

June

-LSB-

-RSB-

M.

Lam

and

M.

E.

Wolf

Automatic

blocking

by

a

compiler

In

SIAM

Conference

on

Parallel

Processing

for

Scientific

Computing

pages

-LCB-

-LSB-

-RSB-

L.

Lamport

The

parallel

execution

of

DO

loops

Communications

of

the

ACM

-LRB-

-RRB-

-LCB-

Feb.

-LSB-

-RSB-

K.

Okuda

Cycle

shrinking

by

dependence

reduction

In

Euro-Par

volume

of

Lecture

Notes

in

Computer

Science

pages

-LCB-

Springer

Verlag

-LSB-

-RSB-

J.

Ramanujam

A

linear

algebraic

view

of

loop

transformations

and

their

interaction

In

SIAM

Conference

on

Parallel

Processing

for

Scientific

Computing

pages

-LCB-

-LSB-

-RSB-

R.

Schreiber

and

J.

J.

Dongarra

Automatic

blocking

of

nested

loops

Technical

Report

The

University

of

Tennessee

Knoxville

TN

Aug.

-LSB-

-RSB-

A.

Schrijver

Theory

of

Linear

and

Integer

Programming

John

Wiley

and

Sons

New

York

-LSB-

-RSB-

B.

Sinharoy

and

N.

Szymanski

Finding

optimum

wavefront

of

parallel

computation

Parallel

algorithms

and

applications

-LCB-

-LSB-

-RSB-

M.

E.

Wolf

and

M.

S.

Lam

A

loop

transformation

theory

and

an

algorithm

to

maximize

parallelism

IEEE

Trans

Parallel

Distributed

Systems

-LRB-

-RRB-

-LCB-

Oct.

medium grain
nested loops
permutable loops
tiling