McGill

University

School

of

Computer

Science

Speculative

Execution

and

Branch

Prediction

on

Parallel

Machines

ACAPS

Technical

Memo

December

Kevin

B.

Theobald

Guang

R.

Gao

Laurie

J.

Hendren

A

version

of

this

paper

appears

in

the

Proceedings

of

the

th

ACM

Intl.

Conf

on

Supercomputing

Tokyo

July

e-mail

author's-surname@cs.mcgill.ca

Advanced

Compilers

Architectures

and

Parallel

Systems

Group

ACAPS

ffl

School

of

Computer

Science

ffl

University

St.

ffl

Montr?eal

ffl

Canada

ffl

Several

recent

studies

on

the

limits

of

parallelism

have

reported

that

the

use

of

speculative

execution

can

lead

to

large

increases

in

the

amount

of

exploitable

parallelism

in

a

program

especially

nonnumerical

programs

This

is

true

even

for

parallel

machines

models

which

allow

multiple

flows

of

control

However

most

architectural

techniques

for

speculative

execution

and

branch

prediction

are

geared

toward

conventional

computers

with

a

single

flow

of

control

and

little

has

been

done

in

studying

speculative

execution

models

and

techniques

which

are

applicable

to

parallel

machines

with

multiple

threads

of

control

This

paper

presents

a

model

of

speculative

execution

for

parallel

machines

We

define

two

different

types

of

speculation

-LRB-

conservative

and

aggressive

-RRB-

and

define

the

level

of

speculation

-LRB-

how

far

ahead

the

speculation

can

go

-RRB-

Conventional

techniques

for

branch

and

jump

prediction

must

be

altered

in

order

to

work

with

this

model

so

we

show

how

this

is

done

for

a

variety

of

the

common

static

and

dynamic

prediction

strategies

This

paper

presents

a

comprehensive

quantitative

study

of

-LRB-

-RRB-

how

parallelism

is

affected

by

the

speculative

execution

and

branch

prediction

techniques

under

a

parallel

model

of

execution

and

-LRB-

-RRB-

what

speculation

depth

is

required

to

get

a

large

portion

of

available

parallelism

We

measure

the

parallelism

limits

of

long

benchmarks

-LRB-

non-numerical

-RRB-

with

different

speculation

models

and

branch

prediction

methods

and

compare

the

results

Contents

Introduction

Speculative

Execution

Dynamic

Control

Dependence

A

Parallel

Execution

Model

Adding

Speculation

to

the

Model

Levels

of

Speculation

Types

of

Speculation

Branch

Prediction

Static

Branch

and

Jump

Prediction

Dynamic

Prediction

Branch

Prediction

Strategies

Jump

Prediction

Strategies

Experiments

Experimental

Framework

The

Range

of

Speculation

Infinite

Depth

Speculation

and

Branch

Prediction

Finite

Speculation

Related

Work

Conclusions

A

Implementation

Details

List

of

Figures

Example

of

Control

Flow

Graph

and

Static

Control

Dependence

A

DCDT

Corresponding

a

Parallel

Execution

of

the

Loop

Branch

Prediction

State

Diagrams

List

of

Tables

Parallelism

Upper

and

Lower

Bounds

Tex

Speech

espresso

eqntott

fpppp

Speculation

Success

Conservative

versus

Aggressive

Summary

of

Static

Prediction

versus

Dynamic

Prediction

Summary

of

Branch

Prediction

Success

Rate

and

Parallelism

Summary

of

Speculation

Depth

and

Parallelism

Speculation

Depth

Required

for

Parallelism

of

Infinite

Depth

Introduction

For

those

designing

highly-parallel

computers

an

important

question

to

ask

is

how

much

parallelism

is

inherent

in

the

applications

independent

of

any

particular

architecture

There

have

been

many

studies

over

the

years

attempting

to

answer

this

question

-LSB-

-RSB-

Though

the

methods

and

models

vary

from

paper

to

paper

all

of

these

studies

simulate

real

benchmark

programs

on

idealized

models

that

attempt

to

remove

specific

architectural

limitations

e.g.

by

allowing

for

unlimited

hardwarerenaming

of

registers

assuming

perfect

alias

analysis

by

the

compiler

etc.

Thus

the

authors

hope

that

the

results

give

upper

bounds

on

the

parallelism

achievable

by

real

architectures

Such

studies

can

give

important

insights

into

what

is

needed

at

both

the

architectural

level

and

the

algorithmic

level

for

future

generation

parallel

supercomputers

One

fact

which

is

shown

in

some

of

the

studies

is

that

in

most

programs

particularly

non-numeric

applications

conditional

branches

and

jumps

significantly

reduce

the

amount

of

parallelism

which

can

be

exploited

in

such

programs

This

is

because

a

conditional

branch

or

jump

creates

uncertainty

about

whether

or

not

a

particular

piece

of

code

will

actually

be

executed

so

that

code

ca

n't

be

executed

until

the

branch

outcome

is

known

Opportunities

for

parallelism

can

be

increased

by

performing

controldependence

analysis

in

which

a

program

trace

is

analyzed

to

decide

on

exactly

which

branch

each

block

of

code

is

dependent

Even

so

many

non-numeric

programs

have

low

parallelism

limits

-LSB-

-RSB-

The

same

programs

usually

have

dramatically

higher

parallelisms

when

the

effects

of

branches

are

completely

ignored

e.g.

by

using

an

oracle

model

-LSB-

-RSB-

Consequently

there

has

been

increasing

interest

in

getting

more

parallelism

from

programs

through

speculative

execution

This

means

executing

the

code

at

one

or

more

destinations

of

a

branch

before

the

branch

outcome

is

known

This

has

primarily

been

used

to

try

to

prevent

branches

from

disrupting

long

execution

pipelines

For

instance

some

machines

issue

instructions

at

the

not-taken

destination

of

a

branch

through

the

pipeline

cancelling

them

if

necessary

Such

architectures

may

also

prefetch

the

instructions

at

the

other

destination

just

in

case

A

more

ambitious

approach

is

to

use

the

past

history

of

branches

to

try

to

predict

the

most

likely

destination

and

to

prefetch

along

that

path

Various

algorithms

for

branch

prediction

are

given

in

-LSB-

-RSB-

There

has

been

a

lot

of

effort

toward

finding

branch

prediction

methods

that

increase

the

rate

of

successful

predictions

Some

are

looking

at

doing

more

than

improving

prefetching

One

superscalar

uses

boosting

-LSB-

-RSB-

to

execute

down

the

most

likely

path

many

instructions

before

a

branch

is

resolved

using

shadow

registers

in

order

to

maintain

two

program

states

which

are

made

consistent

once

the

branch

has

been

resolved

Therefore

some

researchers

on

instruction-level

parallelism

have

incorporated

speculative

execution

into

their

idealized

models

in

order

to

get

an

idea

of

how

much

parallelism

may

be

obtained

through

this

method

The

work

most

relevant

to

this

paper

includes

Wall

-LSB-

-RSB-

Lam

and

Wilson

-LSB-

-RSB-

and

Theobald

Gao

and

Hendren

-LSB-

-RSB-

This

paper

ties

these

two

lines

of

research

-LRB-

speculative

execution

and

branch

prediction

-RRB-

together

We

wish

to

study

how

branch

prediction

and

speculation

interact

and

what

impact

this

has

on

program

parallelism

In

particular

we

would

like

to

know

what

kinds

of

speculative

execution

and

which

branch

prediction

strategies

lead

to

the

highest

potential

amounts

of

parallelism

We

also

want

to

understand

how

branch

prediction

will

work

on

a

parallel

machine

The

existing

dynamic

branch

prediction

algorithms

predict

branches

using

the

previous

history

of

branches

Unfortunately

since

on

a

parallel

machine

many

instructions

may

be

executed

concurrently

the

branch

history

wo

n't

have

the

same

meaning

as

it

does

on

a

sequential

machine

Therefore

we

must

modify

these

algorithms

to

a

form

appropriate

for

multiprocessors

As

well

as

providing

the

framework

for

models

of

speculation

and

branch

prediction

we

also

present

a

quantitative

study

on

the

effects

on

parallelism

by

speculative

execution

under

parallel

machine

models

We

have

defined

a

parallel

model

of

speculative

execution

based

on

the

concept

of

Dynamic

Control

Dependence

Tree

-LRB-

DCDT

-RRB-

In

our

experiments

for

each

branch

the

machine

only

performs

speculation

on

one

path

-LRB-

outcome

-RRB-

we

call

this

the

one-path-per-branch

scheme

-LRB-

Note

of

course

that

there

may

be

many

concurrent

branches

in

such

a

one-path-per-branch

speculative

execution

-RRB-

The

following

is

a

summary

of

the

major

results

ffl

In

all

benchmark

programs

studied

speculative

execution

has

a

significant

effects

on

parallelism

limits

Moreover

a

machine

performing

one-path-per-branch

speculative

execution

could

get

impressive

performance

improvements

over

similar

machines

without

speculation

ffl

In

most

cases

only

a

limited

amount

of

speculation

is

needed

to

get

a

large

fraction

of

the

parallelism

which

would

be

available

if

the

machine

could

speculate

to

an

infinite

depth

-LRB-

around

levels

deep

to

get

and

around

levels

deep

to

get

-RRB-

ffl

In

most

cases

static

branch

prediction

works

as

well

as

dynamic

prediction

-LRB-

based

on

branch

history

-RRB-

In

only

one

case

did

dynamic

prediction

lead

to

substantially

better

performance

ffl

The

studies

show

that

branch

prediction

success

rates

do

not

always

correlate

well

with

overall

parallelism

This

should

not

come

as

a

complete

surprise

as

some

similar

observations

on

the

relation

between

success

rate

and

overall

program

performance

has

been

reported

for

sequential

machines

-LSB-

-RSB-

In

the

next

section

we

present

a

parallel

model

of

speculative

execution

based

on

Dynamic

Control

Dependence

relationships

between

instances

of

basic

code

blocks

using

the

notion

of

a

Dynamic

Control

Dependence

Tree

-LRB-

DCDT

-RRB-

Based

on

this

relationship

we

define

levels

of

speculation

Two

types

of

speculation

called

conservative

and

aggressive

are

presented

In

Section

some

of

the

well-known

static

and

dynamic

methods

of

branch

and

jump

prediction

are

modified

to

work

with

the

execution

model

in

Section

Section

presents

results

of

an

empirical

study

measuring

the

parallelism

limits

of

long

benchmarks

-LRB-

non-numerical

-RRB-

with

different

speculation

models

and

branch

prediction

methods

Related

work

is

discussed

in

Section

The

final

section

Section

summarizes

the

paper

and

discusses

the

conclusions

that

may

be

drawn

from

this

study

Speculative

Execution

The

basic

purpose

of

speculative

execution

is

to

speed

up

program

execution

by

running

some

code

segments

before

it

is

known

whether

or

not

they

are

actually

reached

To

illustrate

the

concept

consider

the

simple

program

represented

in

Figure

Part

-LRB-

a

-RRB-

shows

a

Control

Flow

Graph

-LRB-

CFG

-RRB-

-LSB-

-RSB-

showing

all

the

basic

blocks

of

a

program

as

vertices

and

the

possible

paths

from

one

block

to

another

as

directed

edges

The

diagram

might

represent

for

instance

a

for

loop

Basic

block

-LRB-

-RRB-

is

the

inner

loop

while

is

the

end

of

the

outer

loop

The

nodes

vB

and

vE

represent

the

beginning

and

end

of

the

program

and

do

n't

correspond

to

any

real

code

An

arc

connects

them

to

represent

the

decision

of

whether

or

not

to

run

a

program

Without

any

speculation

a

machine

ca

n't

execute

beyond

until

it

knows

which

way

the

branch

at

is

taken

To

run

faster

the

machine

could

assume

that

the

branch

will

be

taken

-LRB-

to

the

start

of

-RRB-

and

begin

another

iteration

of

before

the

first

branch

is

resolved

-LRB-

assuming

this

is

not

prevented

by

data

dependencies

-RRB-

Since

the

branch

might

be

taken

the

other

way

it

may

be

necessary

to

eliminate

the

vB

vE

-LRB-

a

-RRB-

ffl

ffi

oe

vB

vE

-LRB-

b

-RRB-

ffl

ffi

ffl

ffi

oe

oe

oe

Figure

Example

of

Control

Flow

Graph

and

Static

Control

Dependence

state

changes

caused

by

the

extra

execution

of

This

can

be

done

by

backtracking

or

by

maintaining

two

machine

states

which

are

coalesced

once

the

branch

outcome

is

known

The

latter

is

done

in

an

existing

superscalar

-LSB-

-RSB-

It

might

appear

that

ca

n't

execute

until

all

iterations

of

have

been

completed

However

since

inevitably

leads

to

it

is

possible

to

execute

both

concurrently

assuming

that

all

other

dependencies

are

met

In

fact

the

execution

of

is

certain

as

soon

as

the

program

begins

To

get

more

parallelism

out

of

a

program

a

computer

can

start

executing

and

simultaneously

even

without

any

speculation

These

opportunities

for

additional

parallelism

are

uncovered

through

control-dependence

analysis

covered

in

the

next

section

Speculation

can

be

combined

with

control-dependence

analysis

to

achieve

even

more

parallelism

For

instance

if

and

are

executed

simultaneously

the

machine

could

speculate

along

the

branch

at

and

begin

another

iteration

of

Simultaneously

it

could

also

speculate

along

the

branch

at

and

begin

another

instance

of

In

this

section

we

develop

models

of

speculative

execution

for

parallel

machines

capable

of

simultaneously

following

multiple

flows

of

control

In

Section

we

introduce

the

concept

of

dynamic

control

dependence

Section

develops

a

model

based

on

this

concept

beginning

with

a

parallel

non-speculative

execution

model

Speculation

is

added

in

Section

In

Section

we

define

the

notion

of

levels

of

speculation

based

on

the

dynamic

control

dependence

In

Section

we

describe

two

types

of

speculation

models

conservative

and

aggressive

Dynamic

Control

Dependence

In

the

example

of

the

loop

represented

by

the

CFG

in

Figure

-LRB-

a

-RRB-

it

was

shown

that

does

n't

need

to

wait

until

all

instances

of

have

completed

Control

dependence

-LSB-

-RSB-

explicitly

identifies

this

relationship

This

form

of

dependence

captures

the

idea

that

the

choice

of

whether

or

not

to

execute

a

particular

block

is

decided

by

a

specific

conditional

branch

or

jump

Intuitively

block

vj

is

controldependent

on

vi

if

vi

has

more

than

one

exit

if

some

of

the

exits

from

vi

always

eventually

lead

to

vj

and

if

by

taking

other

exits

from

vi

it

is

possible

to

avoid

vj

The

following

definitions

are

adapted

from

-LSB-

-RSB-

Definition

A

Control

Flow

Graph

-LRB-

CFG

-RRB-

is

a

connected

directed

graph

i.e.

a

pair

-LRB-

V

E

-RRB-

in

which

V

is

a

set

of

vertices

corresponding

to

the

basic

blocks

of

a

program

and

E

V

V

is

a

set

of

edges

Edge

-LRB-

vi

vj

-RRB-

is

a

member

of

E

iff

vj

can

follow

vi

in

the

execution

of

a

program

V

contains

special

nodes

vB

and

vE

where

vB

has

no

predecessors

and

two

successors

one

of

which

is

vE

vE

has

no

successors

and

has

vi

as

a

predecessor

iff

the

program

can

terminate

immediately

after

executing

vi

For

any

vertex

vi

there

is

a

path

from

vB

to

vi

and

a

path

from

vi

to

vE

Definition

A

block

vi

in

G

is

post-dominated

by

a

block

vj

-LRB-

written

postdom

-LRB-

vj

vi

-RRB-

-RRB-

if

every

path

from

vi

to

vE

-LRB-

not

including

vi

-RRB-

includes

vj

Definition

A

block

vj

in

G

is

control-dependent

on

block

vi

iff

There

exists

a

directed

path

P

from

vi

to

vj

such

that

any

vz

in

P

-LRB-

excluding

vi

and

vj

-RRB-

is

post-dominated

by

vj

vi

is

not

post-dominated

by

vj

For

example

in

Figure

-LRB-

b

-RRB-

directed

edges

show

the

control

dependence

relations

among

the

elements

of

the

CFG

in

Figure

-LRB-

a

-RRB-

For

instance

is

control-dependent

on

both

itself

and

vB

The

first

occurrence

of

is

predetermined

as

soon

as

the

program

begins

execution

Subsequently

each

instance

of

is

determined

by

the

execution

of

the

branch

at

the

end

of

the

previous

instance

of

Note

that

only

gets

repeated

if

the

branch

in

is

taken

the

program

terminates

if

the

branch

is

not

taken

Therefore

each

control-dependence

relation

is

associated

with

the

conditions

necessary

for

the

dependent

block

to

be

executed

This

would

be

T

or

F

for

a

conditional

branch

-LRB-

a

transfer

to

a

fixed

destination

which

is

either

taken

or

not

taken

-RRB-

and

one

or

more

destinations

for

a

jump

-LRB-

a

transfer

in

which

the

destination

is

computed

at

-RRB-

We

can

unify

the

two

formats

by

representing

conditions

with

the

actual

destinations

in

the

CFG

Thus

a

conditional

branch

will

have

two

destinations

while

the

jump

will

have

two

or

more

destinations

Therefore

each

edge

is

labeled

with

the

branch

or

jump

outcome

-LRB-

s

-RRB-

which

lead

-LRB-

s

-RRB-

to

the

eventual

execution

of

the

destination

vertex

as

illustrated

in

Figure

-LRB-

b

-RRB-

This

notion

of

control

dependence

with

associated

conditions

can

be

formalized

by

a

dependence

function

DV

E

which

is

specific

to

a

particular

CFG

-LRB-

V

E

-RRB-

Each

edge

in

the

CFG

represents

the

act

of

taking

a

branch

or

jump

to

a

particular

destination

For

each

edge

-LRB-

vi

vj

-RRB-

DV

E

-LRB-

-LRB-

vi

vj

-RRB-

-RRB-

is

the

set

of

all

nodes

whose

execution

becomes

imminent

when

the

conditional

branch

or

jump

at

vi

is

taken

to

destination

block

vj

A

block

is

in

this

set

if

it

does

n't

post-dominate

vi

but

either

post-dominates

vj

or

is

vj

The

statement

that

a

block

is

the

same

as

vj

or

post-dominates

vj

is

equivalent

to

condition

in

Definition

-LRB-

Similar

information

is

contained

in

the

Control

Dependence

Subgraph

-LSB-

-RSB-

where

a

region

node

groups

together

all

nodes

control-dependent

on

the

same

outcome

of

a

particular

branch

or

jump

-RRB-

The

following

is

a

definition

of

the

dependence

function

return

to

a

location

which

was

stored

on

a

stack

is

n't

considered

a

jump

Definition

A

dependence

function

-LRB-

or

D-function

for

short

-RRB-

DV

E

is

a

function

E

V

where

vk

DV

E

-LRB-

-LRB-

vi

vj

-RRB-

-RRB-

-LRB-

-RRB-

-LRB-

vi

vj

-RRB-

E

postdom

-LRB-

vk

vi

-RRB-

-LRB-

vk

vj

postdom

-LRB-

vk

vj

-RRB-

-RRB-

-LRB-

vi

vj

-RRB-

E

-RRB-

DV

E

-LRB-

-LRB-

vi

vj

-RRB-

-RRB-

For

example

in

the

CFG

of

Figure

DV

E

-LRB-

-LRB-

-RRB-

-RRB-

and

DV

E

-LRB-

-LRB-

-RRB-

-RRB-

Control

dependence

as

shown

is

a

static

relationship

among

basic

blocks

in

a

CFG

Each

of

these

blocks

can

correspond

to

many

instances

in

the

actual

execution

of

the

code

In

a

CDG

a

block

may

be

control-dependent

on

more

than

one

other

block

For

instance

block

in

Figure

is

control-dependent

on

itself

and

vB

However

the

execution

of

each

particular

instance

of

is

determined

by

a

different

block

instance

which

is

unique

As

soon

as

the

program

begins

it

is

predetermined

that

the

first

execution

of

will

occur

Each

succeeding

iteration

of

the

inner

loop

is

determined

by

the

previous

instance

of

The

first

iteration

of

the

inner

loop

within

the

second

iteration

of

the

outer

loop

is

dependent

on

the

conditional

branch

at

To

charcterize

such

dynamic

control

dependence

between

block

instances

in

the

execution

we

introduce

the

Dynamic

Control

Dependence

Tree

-LRB-

DCDT

-RRB-

This

generalizes

the

idea

illustrated

above

where

the

execution

of

different

instances

of

a

basic

block

can

be

determined

by

different

conditional

branches

It

does

this

by

explicitly

identifying

the

unique

branch

on

which

each

instance

is

controldependent

Intuitively

the

CFG

is

unrolled

-LRB-

using

its

control-dependence

function

D

-RRB-

into

a

tree

corresponding

to

a

specific

instance

of

a

program

execution

The

following

operational

model

will

produce

a

DCDT

from

a

given

CFG

and

its

function

D

according

to

the

following

control-firing

rules

ffl

Initially

the

start

node

vB

is

control-enabled

and

its

execution

instance

becomes

the

root

of

the

DCDT

ffl

Assume

that

a

node

-LRB-

block

-RRB-

vi

corresponding

to

instance

sj

in

the

DCDT

is

control-enabled

Then

this

block

can

be

fired

-LRB-

executed

-RRB-

provided

that

all

relevant

data

dependencies

have

been

satisfied

If

vi

only

has

one

successor

in

the

CFG

then

sj

is

a

leaf

nothing

is

control-dependent

on

it

-LRB-

it

does

n't

contain

a

branch

or

jump

-RRB-

If

vi

has

more

than

one

successor

in

the

CFG

then

it

is

a

branch

or

jump

Since

a

particular

firing

of

a

node

represents

a

single

instance

of

a

block

the

branch

or

jump

has

a

particular

outcome

If

that

outcome

leads

to

block

vk

then

all

nodes

in

DV

E

-LRB-

-LRB-

vi

vk

-RRB-

-RRB-

become

control-enabled

Corresponding

nodes

are

created

in

the

DCDT

as

the

children

of

sj

and

the

arcs

leading

to

the

children

are

labeled

with

vj

For

instance

Figure

shows

a

DCDT

corresponding

to

a

particular

-LRB-

parallel

-RRB-

execution

of

the

loop

Readers

are

encouraged

to

generate

the

tree

from

the

CFG

following

the

rules

descibed

above

Thus

a

DCDT

represents

the

partial

ordering

of

block

instances

necessitated

by

all

the

true

control

dependencies

for

an

execution

of

the

program

Two

unrelated

blocks

which

happen

to

be

separated

by

a

conditional

branch

can

run

in

parallel

if

neither

is

control-dependent

on

the

other

Note

that

we

only

consider

control

dependence

here

There

may

be

data

anti

and

output

dependencies

as

well

For

instance

may

increment

an

index

variable

which

is

used

by

which

is

an

anti-dependency

so

that

instruction

in

would

have

to

wait

We

assume

that

some

kind

of

interlocks

or

dataflow

scheduling

are

supported

by

the

machine

model

which

can

prevent

hazards

arising

from

data

dependencies

while

register

and

memory

renaming

or

a

single-assignment

rule

can

prevent

hazards

from

anti-dependencies

Just

as

a

dynamic

program

trace

is

a

useful

way

to

describe

the

relations

between

instances

of

instructions

executed

in

a

program

run

a

DCDT

is

conceptually

useful

to

characterize

the

partiallyordered

history

of

branch/jump

operations

of

the

corresponding

program

execution

What

is

the

relation

vB

Level

Level

Level

Level

Figure

A

DCDT

Corresponding

a

Parallel

Execution

of

the

Loop

between

a

DCDT

and

a

dynamic

trace

A

dynamic

trace

of

a

sequential

execution

of

a

program

corresponds

to

a

preorder

depth-first

traversal

of

the

DCDT

assuming

that

corresponding

branch/jump

outcomes

are

the

same

Although

a

formal

proof

is

beyond

the

scope

of

this

paper

there

is

a

simple

intuitive

explanation

of

this

result

If

DV

E

-LRB-

-LRB-

vi

vj

-RRB-

-RRB-

is

non-empty

then

one

of

the

members

of

that

set

must

be

the

destination

vj

That

node

will

obviously

immediately

follow

vi

The

eventual

execution

of

every

other

member

of

that

set

is

also

inevitable

by

definition

and

they

will

occur

in

a

particular

order

-LRB-

since

they

are

being

executed

sequentially

-RRB-

Now

suppose

vx

and

vy

are

members

of

DV

E

-LRB-

-LRB-

vi

vj

-RRB-

-RRB-

and

vy

follows

vx

Then

every

block

which

is

directly

or

indirectly

dynamically

control-dependent

on

that

instance

of

vx

-LRB-

i.e.

a

descendent

of

the

corresponding

instance

in

the

DCDT

-RRB-

must

be

executed

before

the

instance

of

vy

Otherwise

we

would

have

the

absurd

sitation

of

a

block

which

is

dynamically

control-dependent

on

an

instance

of

vx

following

an

instance

of

block

vy

even

though

the

execution

of

that

instance

of

vy

ca

n't

be

affected

by

the

outcome

of

the

branch

or

jump

in

the

instance

of

vx

For

instance

It

is

clear

from

the

drawing

of

the

DCDT

that

the

order

of

blocks

in

a

sequential

trace

of

an

execution

of

the

loop

would

follow

a

preorder

left-to-right

depth-first

traversal

of

the

DCDT

A

Parallel

Execution

Model

The

CFG

and

the

corresponding

DCDT

can

be

used

to

create

a

model

of

parallel

execution

in

which

the

effects

of

control

dependence

are

minimized

In

the

previous

section

as

a

CFG

was

unrolled

into

a

DCDT

vertices

in

the

CFG

were

successively

control-enabled

and

each

new

enabling

of

a

node

corresponded

to

a

node

in

the

DCDT

Time

can

be

added

to

this

model

We

can

say

that

a

node

begins

execution

as

soon

as

it

is

control-enabled

and

that

its

children

-LRB-

if

any

-RRB-

are

control-enabled

some

time

later

after

its

branch

or

jump

has

been

completed

We

say

that

an

instance

of

a

block

is

resolved

at

the

time

that

its

branch

or

jump

has

been

computed

-LRB-

This

could

occur

before

all

instructions

in

that

block

have

completed

if

the

instructions

are

reordered

so

that

the

branch

predicate

is

computed

before

the

end

of

the

block

-RRB-

Definition

Speculation

is

a

form

of

execution

in

which

a

node

in

a

DCDT

does

not

begin

execution

until

its

parent

has

been

resolved

Adding

Speculation

to

the

Model

Suppose

a

machine

begins

running

according

to

the

model

and

starts

executing

and

at

the

same

time

If

the

branch

in

block

is

taken

-LRB-

destination

is

-RRB-

then

another

instance

of

will

be

initiated

A

Speculator

would

have

to

wait

until

the

first

instance

of

is

resolved

before

starting

another

iteration

However

a

speculative

machine

could

assume

that

the

branch

at

will

be

taken

and

start

executing

the

next

iteration

of

the

inner

loop

before

the

first

branch

is

resolved

Simultaneously

it

may

start

new

occurrences

of

and

before

the

first

occurrence

of

has

finished

This

would

be

an

example

of

Speculation

The

speculator

would

use

some

algorithm

for

predicting

the

outcome

of

each

branch

or

jump

-LRB-

these

will

be

covered

in

Section

-RRB-

With

speculation

the

execution

of

the

program

would

correspond

exactly

to

the

DCDT

only

if

all

branches

and

jumps

were

correctly

predicted

That

is

the

DCDT

only

corresponds

to

the

true

dynamic

control

dependencies

of

the

program

If

the

speculator

makes

a

wrong

prediction

and

begins

executing

a

block

at

the

wrong

destination

it

will

have

to

cancel

the

state

changes

made

by

that

block

once

the

true

branch

outcome

is

known

Levels

of

Speculation

It

is

possible

to

speculate

down

more

than

one

level

For

instance

the

machine

could

start

another

occurrence

of

-LRB-

on

the

assumption

that

the

first

speculated

instance

of

ends

with

a

branch

to

repeat

the

loop

-RRB-

and

another

set

of

on

the

assumption

that

the

outer

loop

will

be

repeated

as

well

Figure

the

DCDT

can

be

used

to

view

how

the

machine

may

speculate

down

many

levels

In

this

case

it

is

assumed

in

the

diagram

that

program

execution

has

just

begun

The

labels

at

the

left

show

the

blocks

associated

with

each

level

of

speculation

In

this

instance

each

new

level

of

speculation

involves

starting

another

pass

through

each

active

inner

loop

and

initiating

another

pass

through

the

outer

loop

Note

that

at

each

branch

the

machine

only

speculates

down

one

path

A

fully

speculative

machine

could

speculate

down

both

paths

of

each

branch

but

since

the

program

states

would

grow

exponentially

we

only

consider

machines

that

speculate

down

one

branch

in

this

paper

It

is

important

to

note

that

under

a

parallel

execution

model

even

speculating

down

one

destination

can

lead

to

an

exponential

growth

in

program

states

since

taking

a

single

branch

can

lead

to

the

concurrent

execution

of

several

basic

blocks

each

ending

in

its

own

branch

However

a

higher

percentage

of

the

program

states

will

be

valid

since

only

the

most

likely

paths

are

followed

Clearly

the

further

down

the

machine

speculates

the

less

certain

it

is

about

the

blocks

it

is

executing

The

speculation

depth

represents

the

degree

of

certainty

Definition

The

speculation

depth

of

an

instance

of

a

basic

block

is

the

number

of

unresolved

branches

upon

which

that

block

instance

is

dynamically

control

dependent

is

similar

to

the

CD

-LCB-

MF

model

in

-LSB-

-RSB-

This

is

a

dynamic

quantity

which

decreases

with

time

A

block

with

depth

is

definitely

supposed

to

execute

because

its

parent

-LRB-

in

the

DCDT

-RRB-

has

been

resolved

A

finite

speculator

can

speculatively

execute

all

blocks

within

a

limited

distance

of

a

block

Some

of

these

blocks

will

have

high

speculation

depths

which

will

either

decrease

over

time

as

their

ancestors

are

resolved

one-by-one

or

will

disappear

entirely

when

it

is

discovered

that

an

ancestor

's

branch

was

incorrectly

predicted

Definition

n-level

speculation

means

the

speculative

concurrent

execution

of

all

block

instances

whose

speculation

depth

is

less

than

or

equal

to

n.

Types

of

Speculation

What

happens

when

the

branch

or

jump

at

the

end

of

a

block

is

resolved

This

depends

on

whether

the

actual

outcome

of

the

branch

or

jump

is

the

same

as

the

predicted

outcome

or

different

It

also

depends

on

the

current

speculation

depth

of

the

branch

Suppose

that

the

resolved

branch

has

a

current

speculation

depth

of

This

means

that

there

is

no

question

that

this

block

is

supposed

to

be

executed

Therefore

when

this

branch

is

resolved

whatever

immediately

follows

this

branch

is

determined

-LRB-

speculation

depth

-RRB-

The

machine

should

check

the

actual

outcome

of

the

branch

against

the

prediction

it

made

when

speculating

past

that

branch

If

the

resolved

branch

had

been

correctly

predicted

the

states

which

are

the

children

of

the

branch

in

the

DCDT

which

formerly

had

a

speculation

depth

of

have

their

speculation

depth

changed

to

All

descendents

of

this

branch

have

their

speculation

depths

decremented

Since

the

blocks

with

depth

n

have

their

depths

reduced

to

n

the

speculator

can

now

speculate

one

level

past

those

blocks

If

the

resolved

branch

had

been

incorrectly

predicted

all

states

which

are

descendents

of

the

branch

are

invalid

and

must

be

removed

Computation

may

resume

by

re-traversing

the

CDG

from

the

block

in

which

the

branch

occurs

this

time

applying

the

D

function

to

the

correct

destination

What

happens

if

the

resolved

branch

has

a

current

speculation

depth

greater

than

This

means

that

this

branch

is

dynamically

control

dependent

on

branches

which

have

not

been

resolved

yet

Therefore

all

we

can

say

is

that

this

branch

will

be

taken

provided

all

preceding

branches

were

predicted

correctly

We

define

two

types

of

speculation

which

differ

in

their

response

to

this

case

Conservative

Speculation

disallows

speculating

any

new

blocks

until

the

branches

in

all

ancestors

in

the

DCDT

have

been

resolved

If

the

branch

had

been

mispredicted

all

descendent

blocks

are

immediately

halted

and

their

side

effects

canceled

However

no

new

execution

may

begin

until

the

ancestor

branches

have

been

resolved

If

the

branch

had

been

correctly

predicted

all

speculated

descendents

may

continue

executing

and

their

speculation

depths

are

decreased

but

speculation

ca

n't

progress

beyond

the

current

limit

until

the

ancestor

branches

are

resolved

Aggressive

Speculation

does

n't

make

this

restriction

If

the

branch

prediction

was

wrong

the

machine

traverses

down

the

correct

path

assigning

a

speculation

depth

to

each

child

which

is

the

same

as

the

speculation

depth

of

the

resolved

branch

-LRB-

since

the

branch

is

now

resolved

-RRB-

From

there

speculation

continues

until

depth

n

is

reached

If

the

branch

prediction

was

right

the

speculation

depths

of

all

descendents

are

decremented

and

speculation

progresses

beyond

the

current

limits

by

level

Branch

Prediction

The

previous

section

described

different

models

of

speculative

execution

for

multiprocessor

ma

chines

The

execution

models

assumed

the

existence

of

an

algorithm

for

predicting

branches

This

section

develops

several

algorithms

both

static

and

dynamic

which

are

appropriate

for

the

kind

of

speculative

machine

presented

in

the

previous

section

Static

Branch

and

Jump

Prediction

Static

prediction

assigns

a

most

likely

outcome

to

each

branch

or

jump

instruction

in

the

object

code

Every

time

this

instruction

is

encountered

the

same

prediction

is

made

and

it

never

changes

during

program

execution

In

this

paper

three

kinds

of

static

branch

prediction

are

considered

-LRB-

a

branch

is

a

conditional

transfer

which

is

either

taken

or

not

taken

-RRB-

Take

All

Predict

that

branches

are

always

taken

Take

Back

Predict

that

backward-branches

are

always

taken

while

forward-branches

are

never

taken

-LRB-

This

is

based

on

the

assumption

that

backward-branches

mostly

occur

in

while

and

for

loops

that

are

generally

repeated

more

than

once

-RRB-

Table

For

each

branch

predict

the

outcome

by

looking

up

the

particular

branch

instruction

in

a

table

The

table

contains

the

most

frequent

branch

outcome

of

each

instruction

as

seen

in

a

previous

execution

of

the

same

program

This

is

often

based

on

statistics

gathered

by

profiling

information

For

the

prediction

of

jumps

there

are

no

methods

corresponding

to

the

first

two

branch

prediction

schemes

above

Table-based

prediction

however

can

be

done

for

jump

prediction

as

well

Dynamic

Prediction

Dynamic

prediction

modifies

the

predicted

outcome

according

to

the

history

of

actual

outcomes

of

previous

branches

It

is

hoped

that

the

predictor

can

learn

from

and

adapt

to

changing

branching

patterns

One

general

approach

is

to

record

the

outcomes

of

the

previous

n

occurrences

of

each

branch

instruction

in

the

object

code

and

to

base

the

prediction

of

a

branch

on

the

history

specific

to

that

instruction

-LSB-

-RSB-

These

dynamic

methods

must

be

modified

for

execution

on

the

kind

of

speculative

multiprocessor

envisioned

in

this

paper

In

their

original

form

they

assume

a

uniprocessor

with

a

single

thread

of

control

If

the

program

only

follows

a

single

thread

of

control

then

branches

will

occur

in

a

deterministic

order

which

can

be

recorded

in

the

dynamic

branch

history

states

This

breaks

down

when

processing

is

made

concurrent

For

instance

on

a

uniprocessor

if

a

recursive

procedure

calls

itself

twice

on

different

data

the

outcomes

of

branches

in

the

first

recursive

call

will

update

the

dynamic

branch

history

which

will

affect

the

predictions

made

for

the

same

branches

in

the

second

recursive

call

On

a

parallel

machine

the

two

recursive

calls

may

be

executed

concurrently

With

no

deterministic

ordering

between

the

instructions

in

the

two

procedure

invocations

how

can

the

branch

outcomes

in

one

procedure

call

affect

the

predictions

in

the

other

The

outcomes

in

the

first

invocation

may

be

resolved

at

the

same

time

or

even

later

then

the

time

of

the

predictions

in

the

second

invocation

In

the

speculative

models

we

have

been

discussing

the

only

definite

ordering

of

conditionals

are

the

orderings

created

by

the

dynamic

control

dependence

relation

In

other

words

the

only

deterministic

history

of

branches

and

jumps

available

to

the

prediction

algorithm

when

trying

to

predict

the

conditional

at

a

given

instance

of

a

block

is

that

block

's

direct

ancestors

in

the

DCDT

All

other

branches

may

occur

concurrently

with

these

key

branches

so

the

order

of

their

occurrence

relative

to

each

other

and

to

the

direct

ancestors

is

indeterminite

Therefore

for

the

speculative

machine

described

in

Section

a

dynamic

branch

prediction

algorithm

will

base

its

prediction

for

a

given

instance

of

a

block

only

on

a

subset

of

all

the

block

instances

which

are

ancestors

-LRB-

in

the

DCDT

-RRB-

of

the

block

being

predicted

Most

prediction

strategies

do

n't

look

at

the

entire

history

of

branches

or

jumps

from

the

beginning

of

a

program

but

only

encapsulate

this

history

in

a

fixed

number

of

state

bits

On

a

uniprocessor

following

a

single

thread

of

control

only

a

single

state

need

be

maintained

and

updated

because

of

the

total

ordering

of

all

branches

in

the

program

On

a

parallel

speculative

machine

because

different

instances

of

blocks

will

have

different

sets

of

ancestors

in

the

DCDT

they

ca

n't

all

work

with

the

same

state

Rather

a

separate

state

is

associated

with

each

currently

executing

block

instance

which

contains

a

branch

or

jump

When

the

machine

speculates

past

one

of

these

blocks

and

initiates

one

or

more

dynamically-control-dependent

blocks

it

copies

the

branch

history

state

into

the

program

state

of

each

of

the

dependent

blocks

and

updates

these

copies

according

to

the

way

the

branch

was

predicted

If

that

branch

is

later

resolved

and

a

misprediction

is

detected

all

the

states

corresponding

to

blocks

descendent

from

the

mispredicted

branch

including

the

relevant

copies

of

the

branch

history

state

are

incorrect

and

must

be

removed

If

the

branch

is

resolved

and

correct

prediction

is

verified

then

the

copy

of

the

history

state

in

that

branch

can

be

deleted

as

it

is

now

known

that

all

children

have

valid

states

Branch

Prediction

Strategies

In

this

study

four

basic

types

of

dynamic

branch

prediction

are

studied

These

are

taken

from

existing

methods

used

for

conventional

sequential

machines

Each

assumes

a

state

-LRB-

a

set

of

counters

-RRB-

which

is

updated

by

successive

branches

These

techniques

are

adapted

for

the

speculative

machine

as

described

above

The

methods

are

For

each

branch

a

single

bit

records

the

outcome

of

the

most

recent

occurrence

of

that

branch

which

is

an

ancestor

-LRB-

in

the

DCDT

-RRB-

of

the

current

block

and

that

bit

is

used

to

predict

the

next

branch

For

each

branch

two

bits

record

an

abbreviated

history

of

previous

occurrences

of

that

branch

When

a

branch

must

be

predicted

the

prediction

is

based

on

the

state

The

copied

state

is

updated

according

to

the

predicted

branch

outcome

If

it

is

later

found

that

the

branch

was

mispredicted

then

the

copied

state

is

re-created

according

to

the

actual

branch

outcome

Three

specific

prediction

schemes

are

presented

in

-LSB-

-RSB-

and

shown

in

Figure

In

each

state

diagram

a

state

represents

one

of

the

possible

values

of

the

branch

history

Its

label

represents

the

prediction

for

the

next

branch

based

on

this

state

The

arcs

are

labeled

with

the

transitions

which

represent

either

the

predicted

outcomes

-LRB-

if

branches

are

being

predicted

-RRB-

or

the

actual

branch

outcomes

-LRB-

if

they

are

being

corrected

-RRB-

T

T

N

ffl

ffi

T

ffi

oe

N

T

T

N

ffl

ffi

T

ffi

oe

N

T

T

N

ffl

ffi

T

ffi

oe

N

oe

T

N

oe

T

N

oe

T

N

N

oe

T

N

oeT

N

T

T

N

oe

T

N

oe

T

N

N

N

T

Type

Type

Type

Figure

Branch

Prediction

State

Diagrams

For

each

of

these

algorithms

several

variations

are

possible

First

what

is

the

initial

value

of

each

state

value

In

other

words

what

outcome

should

be

predicted

if

when

a

prediction

must

be

made

for

a

particular

branch

there

are

no

instances

of

that

branch

among

the

ancestors

of

that

branch

in

the

DCDT

We

consider

using

the

three

alternatives

for

static

prediction

presented

in

Section

-LRB-

take

all

take

back

table

-RRB-

as

the

default

prediction

or

initial

value

for

the

state

maintained

for

each

branch

For

the

schemes

there

are

four

possible

choices

for

the

initial

value

but

only

two

possible

predictions

-LRB-

take

or

not

take

-RRB-

so

the

initial

value

chosen

is

the

weakest

value

i.e.

the

value

which

corresponds

to

the

statically-predicted

outcome

yet

is

closest

to

being

overturned

by

a

wrong

prediction

Another

variation

involves

the

scope

of

the

history

states

relative

to

procedure

boundaries

Our

experiments

involve

two

alternatives

Total

Scope

We

assume

that

history

states

are

passed

to

called

procedures

so

that

branch

prediction

within

a

procedure

is

affected

by

the

history

of

branch

outcomes

prior

to

the

procedure

call

When

a

procedure

returns

the

branch

history

state

as

it

was

prior

to

the

procedure

call

is

restored

Procedure

Only

We

can

hypothesize

that

branch

histories

wo

n't

have

any

meaningful

effect

across

procedure

boundaries

and

decide

instead

that

history

states

are

reinitialized

-LRB-

using

one

of

the

three

defaults

-RRB-

whenever

a

new

procedure

is

entered

-LRB-

The

original

states

are

restored

when

the

procedure

returns

-RRB-

Thus

there

are

-LRB-

-RRB-

variations

of

each

of

the

four

basic

dynamic

prediction

techniques

We

performed

experiments

involving

all

variations

plus

the

static

techniques

in

Section

These

are

described

in

Section

For

some

selected

algorithms

we

performed

experiments

involving

an

additional

variation

which

is

based

on

a

recent

paper

-LSB-

-RSB-

which

reported

that

branch

prediction

rates

could

be

improved

by

correlating

the

branch

histories

based

on

specific

branch

address

with

the

history

of

recent

branches

-LRB-

regardless

of

address

-RRB-

Instead

of

associating

a

single

or

state

with

each

branch

address

the

machine

keeps

such

states

for

each

branch

address

and

also

keeps

a

record

of

the

most

recent

n

branches

taken

Then

when

predicting

the

outcome

of

a

particular

branch

the

machine

must

first

look

up

the

n-bit

pattern

representing

to

the

recent

branch

history

then

look

up

the

corresponding

or

history

state

for

the

particular

branch

Jump

Prediction

Strategies

Jump

prediction

is

more

complex

than

branch

prediction

because

there

may

be

more

than

two

destinations

for

a

particular

jump

and

because

it

is

not

generally

possible

to

tell

from

the

object

code

what

the

possible

destinations

are

Therefore

the

only

dynamic

jump

prediction

algorithm

we

consider

in

this

study

is

a

straightforward

one

The

predicted

destination

of

a

jump

instruction

is

the

destination

of

the

previous

occurrence

of

the

same

jump

instruction

Variants

on

this

method

are

similar

to

the

variants

on

branch

prediction

The

default

initial

state

is

either

to

make

no

prediction

at

all

or

to

predict

the

destination

which

was

the

most

frequently

occurring

destination

of

that

jump

instruction

in

a

previous

run

of

the

code

Also

like

the

branch

prediction

the

jump

prediction

can

carry

the

previous

destination

history

state

across

procedure

boundaries

or

else

reset

the

history

state

to

the

default

destination

-LRB-

or

table-lookup

-RRB-

whenever

a

new

procedure

invocation

is

entered

Experiments

Our

experiments

can

be

categorized

into

three

main

groups

The

first

group

of

experiments

measure

the

lower

and

upper

limits

of

parallelism

assuming

no

speculation

for

the

lower

limit

and

assuming

an

oracle

model

for

the

the

upper

limit

These

results

are

given

in

Section

We

then

narrowed

our

focus

to

interesting

cases

where

there

is

a

large

difference

between

the

lower

and

upper

limits

which

presumably

means

a

wide

range

of

possible

results

for

the

speculative

models

The

second

group

of

experiments

measure

the

theoretical

limits

of

parallelism

achievable

by

two

speculation

models

and

the

wide

variety

of

branch

and

jump

prediction

algorithms

presented

in

the

previous

section

The

thrust

of

these

experiments

was

to

determine

which

branch

and

jump

prediction

schemes

work

best

when

infinite

speculation

is

allowed

and

to

compare

conservative

and

aggressive

speculation

These

experimental

results

are

given

in

Section

The

third

group

of

experiments

were

designed

to

study

the

effect

of

a

finite

depth

of

speculation

and

to

measure

what

finite

depth

of

speculation

is

required

to

get

close

to

the

parallelism

obtained

for

the

models

allowing

infinite

depth

speculation

These

experimental

results

are

given

in

Section

Experimental

Framework

To

perform

these

experiments

we

used

a

tool

called

SITA

-LRB-

Sparc

Instruction

Trace

Analyzer

-RRB-

which

we

developed

to

study

program

parallelism

-LSB-

-RSB-

The

tool

executes

a

benchmark

program

on

a

Sun

SparcStation

generates

a

dynamic

trace

of

all

the

operations

performed

in

the

execution

and

then

attempts

to

schedule

these

operations

into

parallel

instructions

according

to

how

an

idealized

parallel

architecture

may

try

to

execute

them

This

basic

approach

is

used

in

many

of

the

other

parallelism

studies

-LRB-

using

different

machines

-RRB-

-LSB-

-RSB-

The

scheduler

makes

certain

ideal

assumptions

when

scheduling

the

operations

In

some

cases

these

assumptions

may

appear

to

correspond

to

unrealistically

ambitious

hardware

implementations

but

the

purpose

of

these

studies

is

to

measure

the

parallelism

inherent

in

the

program

and

algorithm

itself

not

the

parallelism

possible

in

a

given

architecture

All

operations

-LRB-

including

floating-point

calculations

and

memory

accesses

-RRB-

have

latency

Anti

and

output

dependencies

are

ignored

on

the

assumption

that

the

effects

of

these

can

be

eliminated

through

means

such

as

register

renaming

and

adhering

to

a

single-assignment

rule

Operations

from

arbitrary

places

in

the

dynamic

stream

can

be

scheduled

into

the

same

parallel

instruction

and

a

parallel

instruction

can

be

filled

with

an

unbounded

number

of

operations

Further

details

as

to

how

we

configured

SITA

to

perform

the

experiments

are

given

in

Appendix

A.

Basic

implementation

details

are

provided

in

-LSB-

-RSB-

We

ran

our

tool

on

several

benchmarks

each

consisting

of

at

least

million

instructions

We

looked

for

programs

large

enough

to

give

figures

suggestive

of

what

can

be

achieved

with

grand

challenge

problems

without

being

so

large

as

to

overwhelm

our

analyzer

We

ran

all

programs

to

completion

giving

the

analyzer

enough

opportunity

to

extract

parallelism

from

all

parts

of

the

code

Analysis

of

a

program

trace

requires

two

passes

In

the

first

pass

the

instruction

trace

is

broken

down

into

its

constituent

basic

blocks

and

the

frequency

of

each

branch

or

jump

is

tabulated

This

forms

a

CFG

for

the

program

in

which

the

edges

are

labeled

with

their

relative

frequencies

The

control

dependence

relations

are

derived

from

this

CFG

using

an

algorithm

presented

in

-LSB-

-RSB-

The

Range

of

Speculation

To

set

the

stage

for

our

complete

experiments

we

first

ran

some

some

preliminary

tests

on

seven

benchmarks

-LRB-

Table

-RRB-

The

benchmarks

consist

of

three

regular

FP-intensive

scientific

programs

and

four

symbolic

applications

All

are

from

the

SPEC

benchmark

test

suite

except

for

tex

which

is

from

the

DLX

suite

and

a

speech-recognition

program

taken

from

an

actual

industrial

application

Two

preliminary

tests

were

run

to

measure

the

upper

and

lower

bounds

of

parallelism

when

control

dependence

is

a

factor

The

upper

bound

was

provided

by

the

oracle

model

which

ignores

control

dependences

completely

The

oracle

model

measures

only

the

effects

of

true

data

dependences

It

shows

what

would

be

achieved

by

a

machine

that

could

speculate

down

all

destinations

of

a

branch

or

jump

though

an

infinite

number

of

levels

or

alternately

what

would

happen

on

an

ideal

machine

if

all

branches

were

predicted

correctly

Thus

an

imperfect

speculator

will

have

a

lower

resulting

parallelism

The

lower

bound

was

found

by

configuring

SITA

to

model

a

Speculator

-LRB-

a

model

that

does

not

support

any

speculation

at

all

yet

gets

as

much

parallelism

as

possible

through

control

dependence

analysis

-RRB-

Since

the

Speculator

is

constrained

by

all

control

dependences

and

the

Oracle

ignores

these

dependences

these

two

models

represent

the

extremes

of

the

effects

of

control

dependences

An

n-Level

Speculator

-LRB-

n

-RRB-

will

be

affected

by

those

control

dependences

arising

from

branches

and

jumps

which

are

mispredicted

and

those

which

are

deeper

than

the

machine

's

ability

to

speculate

Therefore

the

parallelism

for

a

Speculator

will

lie

somewhere

between

these

two

extremes

The

last

two

benchmarks

dodoc

and

tomcatv

show

almost

no

difference

between

the

Source

DLX

industry

SPEC

Benchmark

tex

speech

espresso

eqntott

fpppp

doduc

tomcatv

Input

draft

bca.in

int

pri

eqn

small

Language

C

FORTRAN

Compiler

Sparc

gcc

gcc

gcc

Sparc

Sparc

Sparc

Dyn

Ops

-LRB-

mil

-RRB-

107 544 427 1124

277 522 3018

Dyn

Blocks

-LRB-

mil

-RRB-

FP

Ops

-LRB-

-RRB-

Loads

-LRB-

-RRB-

Stores

-LRB-

-RRB-

Oracle

189 7991 1363 28661

4974 615 8417

Speculator

1178 566 6014

Table

Parallelism

Upper

and

Lower

Bounds

Speculator

and

the

Oracle

This

is

due

to

the

regularity

of

their

control

structures

In

both

cases

there

are

a

lot

of

loops

in

which

the

loop

predicate

is

not

data-dependent

on

the

body

of

the

loop

-LRB-

e.g.

iteration

across

a

fixed

range

-RRB-

In

such

a

case

a

Speculator

could

iterate

and

test

the

loop

counter

as

soon

as

the

loop

body

begins

so

one

loop

iteration

could

begin

every

few

cycles

Speculating

past

the

test

would

save

only

a

few

cycles

because

the

initiation

of

loop

iterations

would

still

be

constrained

by

the

data-dependences

involved

in

the

incrementing

of

the

loop

counter

Therefore

only

the

first

five

benchmarks

were

selected

for

further

study

Infinite

Depth

Speculation

and

Branch

Prediction

The

next

phase

of

the

experiment

was

to

make

a

complete

sweep

of

the

basic

algorithms

discussed

in

Section

SITA

was

run

on

the

five

interesting

bencmarks

tex

espresso

speech

eqntott

and

fpppp

Tables

-LCB-

summarize

the

parallism

results

for

different

algorithms

on

both

the

conservative

and

aggressive

models

The

tables

are

divided

into

rows

The

rows

are

organized

first

by

speculation

type

-LRB-

conservative

or

aggressive

see

Section

-RRB-

then

by

the

sort

of

scope

used

for

passing

the

history

of

branch

prediction

states

-LRB-

total

or

procedure-only

see

Section

-RRB-

and

then

by

the

sort

of

default

branch

prediction

scheme

used

-LRB-

take-all

take-back

table

see

Section

-RRB-

For

each

row

we

give

the

parallelism

measure

for

the

four

different

dynamic

branch

prediction

schemes

and

the

static

branch

prediction

scheme

For

the

static

schemes

no

jump

prediction

was

used

in

conjunction

with

the

take-all

and

take-back

methods

while

table-based

jump

prediction

was

used

along

with

the

table-based

branch

prediction

Dynamic

jump

prediction

-LRB-

always

predicting

the

previous

destination

-RRB-

is

used

with

all

the

dynamic

branch

prediction

schemes

The

default

jump

prediction

is

if

the

default

branch

prediction

is

take-all

or

take-backward

and

from

the

table

if

the

default

branch

prediction

is

from

the

table

Let

us

first

examine

the

overall

success

of

speculation

Table

compares

the

parallelism

results

for

no

speculation

infinite

speculation

and

the

oracle

machine

For

the

infinite

speculation

we

take

the

maximum

of

all

entries

in

the

table

for

each

benchmark

-LRB-

Tables

-LCB-

-RRB-

From

Table

it

is

clear

that

speculation

results

in

substantial

increases

in

parallelism

over

the

speculation

results

-LRB-

to

increase

-RRB-

However

as

expected

the

parallelism

with

speculation

falls

short

of

the

parallelism

for

the

oracle

machine

-LRB-

the

oracle

has

to

increase

over

the

best

infinite

speculation

-RRB-

This

is

particularly

evident

for

the

first

four

benchmarks

which

exhibit

irregular

control

flow

Speculation

Scope

Default

Dynamic

Static

Type

Pred

Type

Type

Type

Conservative

Total

Take

All

Take

Back

Table

Proc

Take

All

Only

Take

Back

Table

Aggressive

Total

Take

All

Take

Back

Table

Proc

Take

All

Only

Take

Back

Table

Table

Tex

Speculation

Scope

Default

Dynamic

Static

Type

Pred

Type

Type

Type

Conservative

Total

Take

All

Take

Back

Table

Proc

Take

All

Only

Take

Back

Table

Aggressive

Total

Take

All

Take

Back

Table

Proc

Take

All

Only

Take

Back

Table

Table

Speech

Given

that

speculation

seems

like

a

beneficial

idea

let

us

now

examine

the

data

to

determine

which

kind

of

speculation

leads

to

the

best

parallelism

results

First

compare

the

results

for

the

conservative

models

versus

the

results

for

the

agressive

models

It

is

clear

by

examining

the

tables

that

in

all

cases

the

aggressive

results

are

better

than

the

correspoding

conservative

results

This

is

to

be

expected

In

order

to

summarize

the

benefit

we

give

a

comparison

of

the

best

conservative

result

and

the

best

aggressive

result

for

each

benchmark

in

Table

Note

the

added

benefit

of

the

aggressive

approach

is

quite

significant

particularly

for

the

speech

and

eqntott

benchmarks

Now

compare

the

results

for

the

models

using

the

total

scope

for

passing

histories

of

branch

predictions

relative

to

the

equivalent

models

that

consider

on

the

procedure

level

scope

Comparing

the

entries

for

the

total

and

procedure

scopes

in

Tables

-LCB-

it

is

quite

clear

that

there

is

very

little

difference

between

the

results

both

strategies

seem

to

work

equally

well

This

suggests

that

there

is

little

correlation

between

branch

histories

in

different

levels

of

a

recursive

procedure

Given

that

aggressive

speculation

is

significantly

better

than

conservative

speculation

and

that

Speculation

Scope

Default

Dynamic

Static

Type

Pred

Type

Type

Type

Conservative

Total

Take

All

Take

Back

Table

Proc

Take

All

Only

Take

Back

Table

Aggressive

Total

Take

All

Take

Back

Table

Proc

Take

All

Only

Take

Back

Table

Table

espresso

Speculation

Scope

Default

Dynamic

Static

Type

Pred

Type

Type

Type

Conservative

Total

Take

All

Take

Back

Table

Proc

Take

All

Only

Take

Back

Table

Aggressive

Total

Take

All

1124 1124 1123 1124

Take

Back

2624 2624 1125 2624

Table

1149 1149 1125 1149

Proc

Take

All

1124 1124 1123 1124

Only

Take

Back

2625 2625 1125 2625

Table

1149 1149 1125 1149

Table

eqntott

total

scope

is

basically

the

same

as

procedure-only

scope

we

now

focus

on

the

effect

of

different

branch

prediction

schemes

for

the

aggressive/total

cases

-LRB-

rows

in

the

tables

-RRB-

First

let

us

determine

the

best

dynamic

scheme

In

all

benchmarks

the

scheme

performs

about

the

same

as

the

best

scheme

Within

the

schemes

type

and

type

are

the

best

type

does

n't

perform

as

well

We

suspect

that

this

is

because

the

type

scheme

is

more

biased

toward

taking

branches

-LRB-

the

type

scheme

takes

branches

in

of

the

states

while

the

other

two

schemes

take

branches

for

of

the

cases

-RRB-

It

is

likely

that

this

bias

towards

taking

branches

does

not

work

so

well

for

non-numeric

programs

Now

let

us

compare

the

static

strategies

versus

the

dynamic

strategies

In

table

we

give

the

comparison

between

the

best

dyanmic

strategy

and

the

best

static

strategy

In

most

cases

the

best

static

prediction

is

as

good

as

the

best

dynamic

prediction

Only

in

the

case

of

eqntott

is

the

dynamic

strategy

significantly

better

Speculation

Scope

Default

Dynamic

Static

Type

Pred

Type

Type

Type

Conservative

Total

Take

All

1659 1683 1683 2054

Take

Back

2283 2274 2054 2283

Table

2162 2269 1683 2269

Proc

Take

All

1659 1683 1683 2054

Only

Take

Back

2283 2283 2054 2283

Table

2162 2269 1683 2269

Aggressive

Total

Take

All

2120 2120 2015 2120

Take

Back

4166 4166 2120 4166

Table

4024 4024 2069 4024

Proc

Take

All

2120 2120 2069 2120

Only

Take

Back

4166 4166 2120 4166

Table

4024 4024 2069 4024

Table

fpppp

Infinite

Infinite

Oracle

Oracle

Speculator

Speculation

Infinite

tex

espresso

speech

eqntott

2625 800 28661

fpppp

Table

Speculation

Success

This

could

indicate

that

with

this

model

of

parallel

execution

the

branch

histories

are

much

shorter

to

begin

with

Also

the

patterns

that

enable

a

dynamic

predictor

to

learn

are

n't

as

strong

Patterns

that

are

established

on

a

sequential

machine

by

many

repetitions

of

the

same

branch

instruction

wo

n't

appear

on

a

parallel

machine

if

those

repetitions

are

not

control-dependent

on

one

another

If

branches

that

are

related

by

control-dependence

are

essentially

random

then

dynamic

predictions

will

tend

toward

the

most

frequently-occurring

branch

outcome

In

other

words

the

dynamic

predictor

will

tend

to

make

the

same

prediction

as

a

table-based

static

predictor

However

we

must

also

consider

that

our

static

estimates

are

based

on

an

execution

history

for

the

same

data

and

thus

give

only

an

upper

bound

on

how

well

the

static

methods

can

perform

Further

experimentation

will

be

required

to

determine

how

well

the

static

methods

perform

when

the

branch

best

best

conservative

aggressive

conservative

aggressive

aggressive

conservative

tex

espresso

speech

eqntott

fppp

Table

Conservative

versus

Aggressive

best

best

static

static

dynamic

dynamic

tex

espresso

speech

eqntott

fppp

Table

Summary

of

Static

Prediction

versus

Dynamic

Prediction

tex

espresso

speech

eqntott

fpppp

Dynamic

Take

Back

-LRB-

Type

-RRB-

1192 2624 4166

Table

141 1149 4024

Static

Take

Back

1191 1721 4166

Table

141 1149 3363

Table

Summary

of

Branch

Prediction

Success

Rate

and

Parallelism

prediction

estimates

come

from

executions

on

different

data

sets

Now

let

us

look

at

the

branch

prediction

success

rate

and

the

resulting

parallelism

In

Table

we

give

the

results

for

four

different

branch

prediction

schemes

two

dynamic

schemes

and

two

static

schemes

For

each

scheme

we

give

the

percent

of

branches

correctly

predicted

and

the

parallelism

result

The

entries

in

bold

give

the

winning

dynamic

strategy

and

the

winning

static

strategy

Note

that

for

tex

and

espresso

the

table

mechanism

is

superior

to

the

take

back

mechanism

Both

of

these

applications

have

relatively

low

parallelism

and

the

branch

predicition

mechanism

with

a

higher

success

rate

seems

to

do

better

For

speech

eqntott

and

fpppp

the

take

back

mechanism

is

better

even

though

it

has

a

much

lower

branch

prediction

success

rate

This

is

particularly

noticable

in

the

case

of

speech

and

fpppp

For

example

in

the

speech

benchmark

the

take

back

mechanism

gives

only

a

branch

prediction

success

but

led

to

an

impressive

parallelism

of

In

contrast

the

table

mechanism

has

a

much

higher

success

rate

of

but

results

in

a

speedup

of

only

This

illustrates

that

a

better

branch

prediction

rate

does

not

necessarily

give

better

parallelism

results

It

seems

that

for

some

branches

in

these

cases

while

the

branch

is

usually

taken

one

way

the

less-frequent

destination

is

on

a

critical

path

and

shortening

that

critical

path

increases

parallelism

A

few

additional

experiments

were

performed

with

infinite

speculation

In

the

previous

set

of

tests

we

compared

take-back

and

table-based

static

prediction

But

the

former

experiment

also

included

jump

prediction

which

was

not

in

the

latter

experiment

To

separate

these

two

effects

we

ran

a

single

experiment

with

static

take-back

branch

prediction

and

table-based

jump

prediction

The

results

were

virtually

identical

to

take-back

branch

prediction

without

jump

prediction

showing

that

for

those

cases

where

table-based

results

were

higher

the

improvement

was

due

to

the

table-based

branch

prediction

not

the

addition

of

jump

prediction

We

also

ran

some

experiments

to

measure

the

effects

of

branch

correlation

discussed

at

the

end

of

Section

We

found

that

branch

correlation

had

virtually

no

impact

on

parallelism

or

on

prediction

rates

-LRB-

what

little

impact

it

had

was

usually

negative

-RRB-

so

we

did

n't

persue

this

path

any

further

Finite

Speculation

So

far

comparisons

of

prediction

strategies

have

been

based

on

the

values

obtained

from

infinite

speculation

But

this

may

not

be

the

best

way

to

compare

prediction

techniques

since

infinite

speculation

would

be

impossible

to

implement

It

is

possible

that

one

algorithm

is

good

for

deep

speculation

while

another

is

better

when

speculation

is

more

limted

Therefore

we

ran

some

experiments

where

speculation

only

went

to

a

specified

level

We

chose

the

four

prediction

algorithms

listed

in

Table

since

based

on

their

previous

results

they

seemed

to

be

representive

of

the

different

classes

of

algorithms

A

summary

of

the

experimental

results

are

listed

in

Table

We

found

that

in

of

the

cases

the

results

for

infinite

speculation

were

an

accurate

predictor

of

the

relative

performances

of

the

finite

speculation

That

is

the

best

branch

predication

mechanism

for

the

infinite

case

was

also

the

best

branch

prediction

mechanism

for

the

finite

cases

Therefore

in

those

cases

we

only

listed

the

results

for

the

best

speculation

algorithm

There

were

anomolous

results

for

finite

speculation

of

eqntott

so

three

algorithms

are

listed

for

that

case

Each

entry

in

Table

consists

of

two

numbers

The

upper

number

gives

the

parallelism

measured

for

the

specified

model

while

the

bottom

number

gives

the

percent

measurement

as

compared

to

the

parallelism

for

infinite

speculation

depth

For

example

for

the

tex

benchmark

using

a

static

table

branch

predictor

and

a

speculation

depth

of

we

get

a

parallelism

measurement

of

which

is

of

the

parallelism

measurement

for

infinite

depth

speculation

We

note

that

for

all

benchmarks

but

eqntott

a

speculation

depth

of

is

sufficient

to

get

a

large

percentage

of

the

parallelism

availabe

with

infinite

depth

In

the

case

of

eqntott

even

a

speculation

depth

gives

less

than

of

the

infinite

speculation

depth

For

the

case

of

eqntott

we

can

also

notice

that

even

though

the

static

take

back

scheme

was

better

for

the

infinite

depth

speculation

the

static

table

scheme

is

better

for

smaller

speculation

depths

Table

gives

the

speculation

depth

required

to

achieve

and

percent

of

the

parallelism

acheived

by

infinite

depth

speculation

To

acheive

of

the

infinite

case

four

of

the

five

Speculation

Depth

32 64 128 256

tex

static

table

espresso

static

table

speech

static

325 332 402 876

take

back

eqntott

static

105 195 350 570

take

back

static

146 296 529 757

770 790 1149

table

dynamic

147 302 541 822

872 933 2624

take

back

fpppp

static

1178 1695 3137 3724

take

back

Table

Summary

of

Speculation

Depth

and

Parallelism

tex

espresso

speech

eqntott

fpppp

AE

AE

Table

Speculation

Depth

Required

for

Parallelism

of

Infinite

Depth

benchmarks

-LRB-

tex

espresso

speech

and

fpppp

-RRB-

needed

only

or

levels

of

speculation

However

the

benchmark

eqntott

required

levels

of

speculation

to

achieve

even

Similar

results

hold

for

higher

percentages

For

tex

espresso

speech

and

fppp

a

maximum

of

levels

gives

and

a

maximum

of

levels

gives

This

indicates

that

for

many

benchmarks

an

acceptable

portion

of

the

benefit

of

speculation

can

be

realized

with

about

levels

of

speculation

However

as

illustrated

by

the

eqntott

benchmark

some

programs

require

much

deeper

speculation

to

acheive

even

a

small

portion

of

the

potential

benefit

Related

Work

Most

studies

on

the

limits

of

instruction-level

parallelism

have

taken

one

of

two

approaches

One

approach

is

to

analyze

the

selected

benchmark

at

either

the

source-code

or

object-code

level

usually

by

executing

the

program

with

a

special

interpreter

that

is

based

on

a

certain

parallel

machine

model

-LSB-

-RSB-

The

other

approach

-LSB-

-RSB-

also

used

in

our

study

is

to

schedule

machine

instructions

from

a

trace

generated

by

an

actual

execution

of

the

object

code

Wall

-LSB-

-RSB-

developed

a

series

of

models

representing

various

levels

of

optimism

about

what

the

hardware

and

compiler

could

do

Using

trace

driven

simulation

he

analyzed

benchmarks

-LRB-

SPEC

programs

plus

some

real

programs

-RRB-

and

found

that

speculative

execution

is

an

important

factor

in

exploiting

the

parallelism

and

achieving

its

limit

Recently

Wilson

and

Lam

reported

their

study

of

the

way

control

flow

limits

parallelism

They

demonstrated

that

substantially

higher

parallelism

can

be

achieved

by

peforming

control-dependence

analysis

executing

multiple

flows

of

control

simultaneously

and

speculative

execution

-LSB-

-RSB-

Theobald

Gao

and

Hendren

have

also

reported

their

study

of

instructionlevel

parallelism

and

its

smoothability

Their

study

also

found

the

potential

for

much

improvement

using

speculation

of

these

previouse

studies

studied

the

interaction

of

speculation

and

branch

prediction

as

we

did

in

this

paper

Instead

the

speculation

in

each

of

these

studies

corresonds

to

an

isolated

point

in

our

set

of

models

Wall

's

study

did

not

emphasize

the

role

of

multiple

flow

of

control

due

to

a

limited

scheduling

window

In

terms

of

level

speculation

it

appears

that

his

speculation

corresponded

to

our

infinite

aggressive

model

In

terms

of

branch

prediction

he

assumes

the

dynamic

scheme

using

a

state

for

each

branch

address

equivalent

to

our

Type

Lam

and

Wilson

included

control

depedence

and

multiple

flow

of

control

as

an

important

part

of

their

study

However

in

terms

of

speculative

execution

under

multiple

flow

of

contrl

their

study

appears

focused

on

only

two

models

their

CD-MF

model

corresponding

to

our

Speculator

and

their

SP-CD-MF

model

corresponding

to

our

infinite

aggressive

speculation

their

experiments

used

static

branch

prediction

based

on

profile

information

collected

from

running

the

same

benchmark

programs

with

the

same

inputs

used

in

the

simulation

-LRB-

i.e.

static

table-based

prediction

-RRB-

All

three

studies

used

a

model

in

which

a

machine

speculates

down

only

one

branch

Riseman

and

Foster

-LSB-

-RSB-

studied

a

model

in

which

the

machine

could

execute

down

both

paths

through

a

finite

number

of

levels

They

found

that

up

to

a

point

potential

parallelism

was

roughly

proportional

to

the

square

root

of

the

number

of

levels

Conclusions

In

this

paper

we

defined

a

family

of

parallel

models

for

speculative

execution

based

on

the

notion

of

dynamic

control

dependence

-LRB-

DCDT

-RRB-

Based

on

these

models

we

presented

a

comprehensive

experimental

study

of

the

limit

of

parallelism

for

a

number

of

benchmark

programs

assuming

a

variety

speculative

execution

models

and

different

branch

prediction

strategies

Our

results

provide

strong

evidence

that

aggressive

speculative

execution

is

required

to

achieve

reasonable

upper

bounds

on

parallelism

particularly

for

non-numerical

applications

In

studying

the

interaction

between

branch

prediction

and

parallelism

for

the

speculative

models

we

found

that

the

dynamic

and

static

schemes

gave

similiar

results

and

that

a

default

prediction

based

on

the

take

back

and

table

schemes

worked

the

best

Somewhat

suprisingly

we

found

that

branch

prediction

success

rate

did

not

correlate

with

parallelism

results

In

fact

in

some

cases

a

scheme

with

a

much

lower

success

rate

gave

much

better

parallelism

results

In

addition

we

found

that

the

parallelism

measurements

were

relatively

insensitive

to

-LRB-

-RRB-

the

scope

of

branch

prediction

-LRB-

-RRB-

the

method

used

for

jump

prediction

and

-LRB-

-RRB-

branch

correlation

On

the

positive

side

our

experiments

indicate

that

even

though

concurrent

speculation

on

multiple

flow

of

control

is

essential

speculation

down

only

one

path

-LRB-

destination

-RRB-

per

branch

execution

seems

to

be

enough

to

achieve

a

reasonable

degree

of

parallelism

Furthermore

for

the

majority

of

programs

we

studied

the

speculation

depth

needed

to

achieve

a

high

portion

of

available

parallelism

is

bounded

by

a

small

number

-LRB-

around

-RRB-

References

-LSB-

-RSB-

T.

M.

Austin

and

G.

S.

Sohi

Dynamic

dependence

analysis

of

ordinary

programs

In

Proceedings

of

the

Annual

International

Symposium

on

Computer

Architecture

pages

-LCB-

Gold

Coast

Australia

May

-LSB-

-RSB-

M.

Butler

T-Y

Yeh

Y.

Patt

M.

Alsup

H.

Scales

and

M.

Shebanow

Single

instruction

stream

parallelism

is

greater

than

two

In

Proceedings

of

the

Annual

International

Symposium

on

Computer

Architecture

pages

-LCB-

Toronto

Ontario

May

-LSB-

-RSB-

R.

Cytron

J.

Ferrante

B.

K.

Rosen

M.

N.

Wegman

and

F.

K.

Zadeck

An

efficient

method

for

computing

static

single-assignment

form

In

Conference

Record

of

the

Sixteenth

Annual

ACM

Symposium

on

Principles

of

Programming

Languages

pages

-LCB-

Austin

Texas

January

-LSB-

-RSB-

J.

Ferrante

K.

J.

Ottenstein

and

J.

D.

Warren

The

program

dependence

graph

and

its

use

in

optimization

ACM

Transactions

on

Programming

Languages

and

Systems

-LRB-

-RRB-

-LCB-

July

-LSB-

-RSB-

J.

A.

Fisher

and

S.

M.

Freudenberger

Predicting

conditional

branch

directions

from

previous

runs

of

a

program

In

Proceedings

of

the

Fifth

International

Conference

on

Architectural

Support

for

Programming

Languages

and

Operating

Systems

pages

-LCB-

Boston

Massachusetts

October

-LSB-

-RSB-

N.

P.

Jouppi

and

D.

W.

Wall

Available

instruction-level

parallelism

for

superscalar

and

superpipelined

machines

In

Proceedings

of

the

Third

International

Conference

on

Architectural

Support

for

Programming

Languages

and

Operating

Systems

pages

-LCB-

Boston

Massachusetts

April

-LSB-

-RSB-

D.

J.

Kuck

Y.

Muraoka

and

S.

C.

Chen

On

the

number

of

operations

simultaneously

executable

in

FORTRAN-like

programs

and

their

resulting

speed-up

IEEE

Transactions

on

Computers

C

-LRB-

-RRB-

-LCB-

December

-LSB-

-RSB-

M.

Kumar

Measuring

parallelism

in

computation-intensive

scientific/engineering

applications

IEEE

Transactions

on

Computers

-LRB-

-RRB-

-LCB-

September

-LSB-

-RSB-

M.

S.

Lam

and

R.

P.

Wilson

Limits

of

control

flow

on

parallelism

In

Proceedings

of

the

Annual

International

Symposium

on

Computer

Architecture

pages

-LCB-

Gold

Coast

Australia

May

-LSB-

-RSB-

J.

K.

F.

Lee

and

A.

J.

Smith

Branch

prediction

strategies

and

branch

target

buffer

design

IEEE

Computer

pages

-LCB-

January

-LSB-

-RSB-

A.

Nicolau

and

J.

A.

Fisher

Measuring

the

parallelism

available

for

very

long

instruction

word

architectures

IEEE

Transactions

on

Computers

-LRB-

-RRB-

-LCB-

November

-LSB-

-RSB-

S.-T

Pan

K.

So

and

J.

T.

Rahmeh

Improving

the

accuracy

of

dynamic

branch

prediction

using

branch

correlation

In

Proceedings

of

the

Fifth

International

Conference

on

Architectural

Support

for

Programming

Languages

and

Operating

Systems

pages

-LCB-

Boston

Massachusetts

October

-LSB-

-RSB-

E.

M.

Riseman

and

C.

C.

Foster

The

inhibition

of

potential

parallelism

by

conditional

jumps

IEEE

Transactions

on

Computers

-LRB-

-RRB-

-LCB-

December

-LSB-

-RSB-

J.

E.

Smith

A

study

of

branch

prediction

strategies

In

Proceedings

of

the

Annual

Symposium

on

Computer

Architecture

pages

-LCB-

Minneapolis

Minnesota

May

-LSB-

-RSB-

M.

D.

Smith

M.

S.

Lam

and

M.

A.

Horowitz

Boosting

beyond

static

scheduling

in

a

superscalar

processor

In

Proceedings

of

the

Annual

International

Symposium

on

Computer

Architecture

pages

-LCB-

Seattle

Washington

May

-LSB-

-RSB-

K.

B.

Theobald

G.

R.

Gao

and

L.

J.

Hendren

On

the

limits

of

program

parallelism

and

its

smoothability

In

Proceedings

of

the

Annual

International

Symposium

on

Microarchitecture

pages

-LCB-

Portland

Oregon

December

-LSB-

-RSB-

K.

B.

Theobald

G.

R.

Gao

and

L.

J.

Hendren

On

the

limits

of

program

parallelism

and

its

smoothability

ACAPS

Technical

Memo

School

of

Computer

Science

McGill

University

Montr?eal

Qu?ebec

June

-LSB-

-RSB-

G.

S.

Tjaden

and

M.

J.

Flynn

Detection

and

parallel

execution

of

independent

instructions

IEEE

Transactions

on

Computers

-LRB-

-RRB-

-LCB-

October

-LSB-

-RSB-

D.

W.

Wall

Limits

of

instruction-level

parallelism

In

Proceedings

of

the

Fourth

International

Conference

on

Architectural

Support

for

Programming

Languages

and

Operating

Systems

pages

-LCB-

Santa

Clara

California

April

A

Implementation

Details

Since

a

sequential

trace

of

a

program

is

equivalent

to

a

preorder

depth-first

traversal

of

the

corresponding

DCDT

-LRB-

see

Section

-RRB-

and

since

a

block

is

only

dynamically

control-dependent

on

its

ancestors

in

the

DCDT

it

is

only

necessary

to

keep

a

stack

of

basic

block

identities

and

the

number

of

the

parallel

instruction

in

which

each

block

's

branch

is

executed

This

represents

the

time

at

which

a

block

is

resolved

so

we

use

the

term

resolution

time

to

refer

to

the

number

of

the

parallel

instruction

containing

the

branch

When

a

new

block

is

encountered

in

the

trace

SITA

uses

the

D

function

to

find

the

set

of

blocks

on

which

the

new

block

is

control-dependent

and

then

pops

blocks

off

the

stack

until

it

finds

a

block

which

is

in

that

set

-LRB-

The

blocks

which

were

popped

off

the

stack

represent

parts

of

the

DCDT

which

have

already

been

traversed

and

hence

do

n't

need

to

be

saved

anymore

-RRB-

SITA

then

looks

at

the

resolution

time

of

the

block

at

the

new

top

of

stack

and

makes

it

a

floor

for

scheduling

the

current

basic

block

all

operations

in

the

current

block

must

be

scheduled

after

that

time

Of

course

the

floor

is

only

one

constraint

each

operation

in

the

new

block

may

be

further

delayed

by

unsatisfied

data

dependencies

Like

Speculation

Infininte-Level

Speculation

is

simulated

by

maintaining

a

stack

with

both

the

identity

of

each

basic

block

and

its

resolution

time

In

addition

if

dynamic

prediction

is

used

then

each

entry

in

the

stack

contains

the

current

branch

history

state

When

a

new

block

is

encountered

in

the

trace

SITA

pops

blocks

off

the

stack

until

it

finds

a

block

which

is

in

the

set

defined

by

D.

SITA

then

looks

back

through

the

stack

until

finding

the

first

mis-predicted

branch

or

jump

and

uses

the

resolution

time

of

that

branch

or

jump

as

the

floor

for

scheduling

the

current

basic

block

When

SITA

encounters

a

branch

or

jump

it

predicts

the

jump

according

to

the

selected

algorithm

If

prediction

is

static

the

appropriate

method

is

applied

and

the

block

at

the

top

of

the

stack

is

either

marked

predicted

or

mis-predicted

If

prediction

is

dynamic

SITA

looks

back

through

the

stack

to

find

the

most

recent

branch

or

jump

with

the

same

address

-LRB-

which

is

an

ancestor

in

the

DCDT

-RRB-

and

copies

the

history

state

into

the

new

top

of

stack

updating

it

according

to

the

algorithm

chosen

The

current

block

is

either

marked

predicted

or

mis-predicted

and

its

resolution

time

is

saved

If

the

previous

block

on

the

stack

-LRB-

representing

the

parent

in

the

DCD

-RRB-

was

resolved

after

the

current

block

and

the

speculation

is

conservative

then

the

new

resolution

time

recorded

is

the

same

as

the

one

in

the

previous

block

The

method

of

determining

the

floor

must

be

modified

if

speculation

is

only

to

a

depth

of

n

After

popping

back

to

the

appropriate

block

in

the

stack

SITA

then

looks

back

through

the

stack

until

finding

the

first

mis-predicted

branch

So

far

this

is

the

same

as

infinite

speculation

But

whereas

with

infinite

speculation

the

resolution

time

of

that

mis-predicted

branch

becomes

the

floor

for

scheduling

instructions

in

the

current

basic

block

with

finite

speculation

the

current

block

may

have

to

be

scheduled

later

This

is

because

there

may

be

many

unresolved

branches

between

the

current

block

and

the

mispredicted

branch

more

than

the

speculation

limit

Therefore

if

n-level

aggressive

speculation

is

used

the

floor

for

the

current

basic

block

is

the

minumum

of

The

resolution

time

of

the

mis-predicted

branch

and

The

resolution

time

of

the

-LRB-

n

-RRB-

st-most-recently

resolved

block

in

the

stack

-LRB-

is

added

because

Speculation

corresponds

to

the

resolution

of

the

most-recently-resolved

block

-RRB-

branch prediction
control dependence
speculative execution

instruction-level parallelism
parallel architectures


Augmented Reality Enabled Collaborative Work in ? Studierstube ? 1 Zsolt Szalav?ri , Michael Gervautz , Anton Fuhrmann , Dieter Schmalstieg Institute of Computer Graphics - Vienna University of Technology Karlsplatz 13/186/2 , A-1040 Vienna , Austria Tel : +43 -1 -58801 -4579 , Fax : +43 -1 -5874932 szalavari | gervautz | fuhrmann | schmalstieg@cg.tuwien.ac.at Abstract : We introduce a multi-user augmented reality system to support collaboration of participants in research , presentation and education . Our system ? Studierstube ? 2 presents threedimensional stereoscopic images simultaneously to a group of users wearing see-through head mounted displays . The displays do not affect natural communication and interaction , making working together very effective . Users see the same spatially aligned model , but can independently control their viewpoint and different layers of the data to be displayed . 1 . Introduction Highly interdisciplinary fields in science , e.g. scientific visualization frequently require experts with different background to cooperate . In other situations , e.g. presentations and lectures a leading person might want to present information to other people with divers educational background . These discussions involve every social communication channel but also make use of different types of presentation tools , like overheads , flip charts or mock-ups to enhance the process of envisioning information . During the debate shared resources , like a prototype or a blackboard are used simultaneously to investigate a given problem collaboratively . The envisioned information is crucial for the success of a discussion . The method for visual improvement or enrichment of the surrounding environment by overlaying spatially aligned computer-generated information onto a human?s view , called Augmented Reality -LRB- AR -RRB- , has potential to support the mentioned collaborative situations . 2 . Related Work Researchers focusing on AR systems , e.g. -LSB- 1 -RSB- , -LSB- 4 -RSB- , -LSB- 5 -RSB- described systems to support one user in such diverse tasks as maintenance and medical inspection by combining synthetic images with real world information . Other groups investigated multi-user display technologies , like the CAVE-System -LSB- 3 -RSB- and the Responsive Workbench -LSB- 6 -RSB- . Closest to our work on supporting the collaboration of multiple users is the Shared Space approach -LSB- 2 -RSB- . 3 . Augmented Collaboration in the ? Studierstube ? In our system -LSB- 8 -RSB- participants wear see-through head mounted displays to see augmented information , while preserving most social communication channels . Each user gets his own view on the synthetic data , so displayed images can contain information customized to own preferences . Interaction in ? Studierstube ? is supported by the Personal Interaction Panel -LRB- PIP -RRB- -LSB- 7 -RSB- , a combination of an augmented panel and pen for each user . Each user has the option to move freely and independently of the other users . Viewing and examining of objects that are either not accessible directly or that do not exist in the real world can be carried out in this environment . Furthermore real-world objects can be augmented with spatially aligned information and can be annotated -LSB- 4 -RSB- . This allows smooth extension of real objects with virtual properties in design processes , like variations of new parts for an existing 1 This work has been supported by the Austrian Fond for Science and Research FWF Proj . No . P 12074-MAT . 2 For our system we selected the name ? Studierstube ? after the play Faust by Johann Wolfgang von Goethe , in which the leading character uses a study room for performing research and philosophy : the Studierstube . system . Moreover annotations can be created , edited and placed or moved directly into the augmentation with the PIP . We group information into layers , so that data is separated into disjoint sets according to semantic considerations -LRB- e.g. floor plan with walls only - furniture - measurements -RRB- . Display can be turned on and off for every layer individually . This concept is fundamental for allowing individuals to customize their display to their needs . The system takes advantage of the major benefit of AR that sophisticated group-ware mechanisms are not really needed to perform real word communication . Normal human interaction -LRB- verbal , gestures , etc. -RRB- is easily possible in this setup , and at the same time very rich . Consequently , the system does not necessarily require advanced communication support ; for a wide range of applications a setup for joint examination and discussion of static -LRB- maybe simple dynamic -RRB- models is sufficient . Users may see the same model and at the same time not see the same information , as anyone can see a different set of aspects of the same visualization using layers . Aside from personal taste and interest , this is useful if professional people -LRB- e.g. an architect -RRB- talk to inexperienced people -LRB- e.g. customer -RRB- , or if people with different interest -LRB- e.g. designer and engineer -RRB- collaborate . Additionally to social communication and direct modification of the virtual model projection walls provide an efficient tool to share information between participants . These walls are static , virtual objects in the ? Studierstube ? and are logically connected to one user?s PIP . Thereby , a single user interaction tool is transformed into a multi-user presentation spot , where personal ideas can be shared . Another way to share ideas is to allow ? giving ? the content of one?s handheld flat PIP to other users . In this multi-user situation it is crucial , that some information has to be really personal . Layers are used to define sets of three dimensional information public to a group of participants . These ? interest groups ? can be redefined dynamically during discussions to enable or prohibit aspects of the data-set to specific users . Additionally users can protect individual information on the PIP by simply not showing it to others or by using layers . Sharing of information is done by using projection walls or placing information into public layers . In this way we solve the problem of privacy and publicity of visual information in our environment . 4 . Conclusions Shared augmented reality is a powerful tool for collaboration tasks like exploration and investigation of three-dimensional data . Social channels can be enhanced by carefully designed mechanisms like annotations , layers , personalized interfaces or projection walls supporting privacy and publicity . 5 . References -LSB- 1 -RSB- M. Bajura , H. Fuchs , and R. Ohbuchi . Merging Virtual Objects with the Real World : Seeing Ultrasound Imaginary within the Patient . Proceedings of SIGGRAPH ' 92 , -LRB- 2 -RRB- :203 -210 , 1992 . -LSB- 2 -RSB- M. Billinghurst , S. Weghorst , T. III Furness . Shared Space : An Augmented Reality Interface for Computer Supported Collaborative Work . Proceedings of Collaborative Virtual Environments ? 96 , 1996 . -LSB- 3 -RSB- C. Cruz-Neira , D. Sandin , and T. DeFanti . Surround-Screen Projection-Based Virtual Reality : The Design and Implementation of the CAVE . Proceedings of SIGGRAPH ' 93 , -LRB- 3 -RRB- : 135-142 , 1993 . -LSB- 4 -RSB- S. Feiner , B. MacIntyre , and D. Seligmann . Knowledge-Based Augmented Reality . Comm . of the ACM , 36 -LRB- 7 -RRB- :53 -62 , 1993 . -LSB- 5 -RSB- W. Fritzmaurice . Situated Information Spaces and Spatially Aware Palmtop Computers . Communications of the ACM , 36 -LRB- 7 -RRB- :39 -49 , 1993 . -LSB- 6 -RSB- M. Agrawala , A. C. Beers , B. Fr?hlich , I. McDowall P. Hanrahan , and M. Bolas . The Two-User Responsive Workbench : Support for Collaboration Through Individual Views of a Shared Space . Computer Graphics Proceedings , Annual Conference Series SIGGRAPH ' 97 , 327-332 , 1997 . -LSB- 7 -RSB- Zs . Szalav?ri and M. Gervautz . The Personal Interaction Panel - A Two Handed Interface for Augmented Reality . Computer Graphics Forum -LRB- Proceedings of EUROGRAPHICS ' 97 -RRB- , 16 -LRB- 3 -RRB- :335 -346 , 1997 . -LSB- 8 -RSB- Zs . Szalav?ri , D. Schmalstieg , A. Fuhrmann , and M. Gervautz . ? Studierstube ? - An Environment for Collaboration in Augmented Reality . Journal of the Virtual Reality Society - ? Virtual Reality : Research , Development and Application ? , Virtual Press Ltd. , UK , 1997 .

B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O