Efficient

Broadcast

and

Multicast

on

Multistage

Interconnection

Networks

using

Multiport

Encoding

Rajeev

Sivaram

and

Dhabaleswar

K.

Panda

Craig

B.

Stunkel

Dept.

of

Computer

and

Information

Science

I.B.M.

T.

J.

Watson

Research

Center

The

Ohio

State

University

P.

O.

Box

Columbus

OH

Yorktown

Heights

NY

Tel

-LRB-

-RRB-

Fax

-LRB-

-RRB-

Tel

-LRB-

-RRB-

Email

fsivaram,pandag@cis.ohio-state.edu

Email

stunkel@watson.ibm.com

Contact

Author

Dhabaleswar

K.

Panda

Abstract

This

paper

proposes

a

new

approach

for

implementing

fast

multicast

and

broadcast

in

unidirectional

and

bidirectional

multistage

interconnection

networks

-LRB-

MINs

-RRB-

with

multiport

encoded

multidestination

worms

For

a

MIN

with

n

stages

such

worms

use

n

header

flits

each

One

flit

is

used

for

each

stage

of

the

network

and

it

indicates

the

output

ports

to

which

a

multicast

message

needs

to

be

replicated

A

multiport

encoded

worm

with

-LRB-

dn

di

k

-RRB-

degrees

of

replication

for

the

respective

stages

is

capable

of

covering

-LRB-

dn

-RRB-

destinations

with

a

single

communication

start-up

In

this

paper

a

switch

architecture

is

proposed

for

implementing

multidestination

worms

without

deadlock

Three

grouping

algorithms

of

varying

complexity

are

presented

to

derive

the

associated

multiport

encoded

worms

for

a

multicast

to

an

arbitrary

set

of

destinations

Using

these

worms

a

multinomial

tree-based

scheme

is

proposed

to

implement

the

multicast

This

scheme

significantly

reduces

broadcast/multicast

latency

compared

to

schemes

using

unicast

messages

Simulation

studies

for

both

unidirectional

and

bidirectional

MIN

systems

indicate

that

improvement

in

broadcast/multicast

latency

up

to

a

factor

of

is

feasible

using

the

new

approach

Interestingly

this

approach

is

able

to

implement

multicast

with

reduced

latency

as

the

number

of

destinations

increases

beyond

a

certain

number

Compared

to

implementing

unicast

messages

this

approach

requires

little

additional

logic

at

the

switches

Thus

the

scheme

demonstrates

significant

potential

for

implementing

efficient

collective

communication

operations

on

current

and

future

MIN-based

systems

This

research

is

supported

in

part

by

NSF

Grant

NSF

Career

Award

and

an

IBM

Cooperative

Fellowship

A

preliminary

version

of

this

manuscript

has

been

accepted

for

presentation

in

SPDP

Contents

Introduction

Multicasting

using

Unicast

Worms

Basic

Multicasting

Schemes

Encoding

Unicast

Worms

Multicasting

using

Multiport

Encoded

Worms

Multiport

Encoding

Multinomial

Tree

Based

Multicasting

Architectural

Considerations

for

Implementing

Multidestination

Worms

Design

Criteria

A

Switch

Architecture

Reducing

Switch

Complexity

Register-Staircase

Approach

Split-RAM

Approach

Summary

Algorithms

for

Generating

Multiport

Encoded

Worms

Some

Basic

Definitions

and

Properties

Statement

of

the

problem

Approaches

for

solving

the

problem

A

Method

for

Constructing

a

Multiport

Tree

Multiport

Tree?Simple

-LRB-

MTS

-RRB-

Algorithm

A

Greedy

Approach

for

finding

a

SMT

and

a

Method

for

Constructing

Larger

Multiport

Trees

Multiport

Tree?Merge

-LRB-

MTM

-RRB-

Algorithm

Multiport

Tree?Exhaustive

-LRB-

MTE

-RRB-

Algorithm

Building

a

Modified

Karnaugh

Map

Identifying

a

row

of

the

MKM

Forming

a

Grid

Find

Maxima

Find

Enhanced

Group

Comparison

of

the

Algorithms

Application

of

Multiport

Encoding

to

Bidirectional

MINs

Simulation

Results

for

Multicast

on

Unidirectional

MINs

Experiments

Impact

of

Startup

Time

Impact

of

System

Size

Impact

of

Message

Length

Simulation

Results

for

Multicast

on

Bidirectional

MINs

Conclusion

A

The

Sub-procedures

of

the

MTE

Algorithm

Introduction

The

wormhole-routing

switching

technique

is

the

current

trend

in

building

parallel

systems

due

to

inherent

advantages

such

as

low-latency

communication

and

reduced

communication

hardware

overhead

-LSB-

-RSB-

This

switching

technique

is

being

used

with

a

wide

variety

of

topologies

Examples

include

k-ary

n-cube

networks

-LRB-

Cray

-LSB-

-RSB-

Intel

Paragon

-LSB-

-RSB-

Ncube

-LSB-

-RSB-

-RRB-

fat-tree

networks

-LRB-

-LSB-

-RSB-

Meiko

-LSB-

-RSB-

-RRB-

and

multistage

interconnection

networks

-LRB-

IBM

-LSB-

-RSB-

-RRB-

All

of

these

systems

are

being

used

to

support

either

the

distributed-memory

or

distributedshared

memory

programming

paradigms

For

efficient

support

of

either

paradigm

these

systems

require

fast

collective

communication

-LSB-

-RSB-

support

from

the

underlying

communication

subsystem

as

for

example

is

defined

by

the

Message

Passing

Interface

-LRB-

MPI

-RRB-

Standard

-LSB-

-RSB-

Among

the

set

of

collective

communication

operations

broadcast

-LSB-

-RSB-

and

multicast

-LSB-

-RSB-

are

fundamental

and

are

used

in

several

other

operations

like

barrier

synchronization

and

reduction

-LSB-

-RSB-

Thus

it

is

advantageous

to

reduce

latency

of

broadcast

and

multicast

operations

on

these

systems

Many

software

schemes

have

been

recently

proposed

to

efficiently

implement

broadcast

and

multicast

in

wormholerouted

k-ary

n-cube

networks

-LSB-

-RSB-

and

multistage

interconnection

networks

-LSB-

-RSB-

All

of

these

schemes

use

pointto-point

-LRB-

unicast

-RRB-

message

passing

and

require

multiple

contention-free

phases

to

achieve

fast

broadcast

and

multicast

For

multicasting

to

d

destinations

these

schemes

typically

require

-LRB-

d

-RRB-

e

communication

phases

Since

the

ratio

of

communication

start-up

time

to

propagation

time

is

quite

high

on

current

parallel

systems

-LSB-

-RSB-

such

a

software

based

unicast

message-passing

approach

leads

to

very

high

latency

for

broadcast

and

multicast

operations

For

example

with

a

microsec

communication

start-up

time

it

takes

at

least

microsec

-LRB-

ignoring

the

data

propagation

time

-RRB-

to

perform

a

broadcast

operation

on

a

system

This

raises

a

fundamental

question

can

the

latency

of

these

operations

be

reduced

by

using

a

new

and

efficient

mechanism

at

the

hardware

level

A

new

concept?multidestination

wormhole

message

passing?has

been

introduced

recently

-LSB-

-RSB-

Unlike

a

unicast

message

which

has

a

single

destination

such

a

mechanism

allows

a

message

to

have

multiple

destinations

A

multidestination

worm

provides

the

flexibility

to

distribute

data

to

multiple

nodes

or

gather

data

from

multiple

nodes

using

a

single

message

and

a

single

communication

start-up

Using

such

worms

it

has

been

shown

that

broadcast

and

multicast

operations

on

k-ary

n-cube

networks

with

different

routing

schemes

-LRB-

e-cube

adaptive

Hamiltonian

etc.

-RRB-

can

be

implemented

with

significantly

reduced

latency

-LSB-

-RSB-

compared

to

unicast-based

schemes

Since

this

mechanism

and

the

associated

schemes

have

been

specifically

developed

for

direct

networks

with

k-ary

n-cube

topologies

they

can

not

be

applied

to

switch-based

indirect

networks

with

multistage

interconnects

Recently

an

approach

using

multidestination

worms

has

been

proposed

for

MINs

supporting

strict

wormholerouting

-LSB-

-RSB-

It

has

been

shown

that

both

asynchronous

and

synchronous

replication

of

multidestination

worms

lead

to

deadlock

Thus

a

scheme

using

synchronous

replication

of

multidestination

worms

that

uses

a

deadlock

avoidance

scheme

at

each

intermediate

switch

has

been

proposed

However

this

scheme

is

very

complex

and

requires

considerable

hardware

support

Alternatively

multidestination

worms

with

multiport

encoding

has

been

proposed

-LSB-

-RSB-

to

support

broadcast

over

a

contiguous

range

of

output

ports

at

a

given

switch

However

arbitrary

multicast

over

output

ports

is

not

feasible

by

this

scheme

Multiple

broadcast/multicast

using

such

a

scheme

also

leads

to

deadlock

Recently

such

a

multiport

encoding

of

worms

has

also

been

proposed

for

source-based

adaptive

routing

on

MINs

-LSB-

-RSB-

Since

current

MIN-based

parallel

systems

like

the

IBM

and

-LSB-

-RSB-

use

buffered

wormhole

or

virtual

cut-through

techniques

instead

of

wormhole

routing

a

worthwhile

challenge

is

to

devise

a

method

to

do

multicast

on

such

systems

using

multiport

encoding

In

this

paper

we

propose

a

new

approach

to

implement

fast

broadcast

and

multicast

on

both

unidirectional

and

bidirectional

MINs

using

multiport

encoded

multidestination

worms

For

a

MIN

-LSB-

-RSB-

with

n

stages

and

k?k

switches

our

approach

uses

multidestination

worms

with

n

flit

headers

One

header

flit

is

used

for

each

stage

Each

header

flit

carries

information

about

the

ports

to

which

a

message

needs

to

be

replicated

at

the

associated

switches

of

a

given

stage

Architectural

details

for

implementing

such

worms

on

MINs

without

deadlock

are

presented

With

this

encoding

a

single

worm

has

capability

to

cover

a

large

set

of

destinations

with

a

single

communication

start-up

For

example

a

worm

with

the

following

degrees

of

replication

for

each

stage

-LRB-

dn

di

k

-RRB-

can

cover

-LRB-

dn

-RRB-

destinations

incurring

a

single

communication

start-up

We

present

a

theoretical

framework

under

which

we

define

the

problem

of

generating

multiport

encoded

worms

that

cover

given

multicast

destination

sets

We

first

develop

the

framework

considering

unidirectional

MINs

alone

and

then

show

that

the

same

framework

can

be

applied

for

multicast

in

bidirectional

MINs

Three

algorithms

of

differing

complexities

that

solve

the

problem

of

generating

multiport

encoded

worms

to

cover

multicast

sets

are

presented

A

hierarchical

scheme

using

a

multinomial

tree

is

proposed

to

assign

the

generated

multiport

encoded

worms

to

the

source

and

intermediate

destinations

in

order

to

parallelize

communication

startups

and

to

reduce

multicast

latency

With

p

destinations

being

covered

by

a

single

multiport

encoded

worm

on

the

average

our

scheme

uses

approximately

dlogp

-LRB-

d

-RRB-

e

communication

start-ups

to

cover

d

destinations

This

approach

significantly

reduces

broadcast/multicast

latency

compared

to

schemes

using

unicast

messages

-LSB-

-RSB-

For

a

range

of

system

sizes

message

lengths

and

communication

start-up

times

simulation

studies

indicate

that

a

reduction

in

broadcast/multicast

latency

up

to

times

can

be

achieved

by

our

approach

Similar

to

the

results

on

k-ary

n-cube

networks

-LSB-

-RSB-

this

study

also

indicates

that

multiport

encoded

worms

can

reduce

multicast

latency

in

MINs

as

the

number

of

destinations

participating

in

a

multicast

increases

beyond

a

certain

number

This

shows

that

there

is

significant

potential

for

this

new

approach

to

be

used

in

MINs

for

implementing

dense

collective

communication

operations

efficiently

The

paper

is

organized

as

follows

Section

reviews

methods

for

performing

unicast

based

multicast

on

MINs

and

describes

alternative

encodings

for

a

unicast

worm

header

In

Sec.

we

introduce

the

concept

of

multiport

encoding

in

the

context

of

multidestination

worms

and

present

a

multinomial

tree

based

algorithm

for

performing

multicast

using

multiport

encoded

worms

In

Section

we

present

design

criteria

that

must

be

taken

into

account

for

implementing

multidestination

worms

and

present

alternative

implementations

of

a

proposed

architecture

A

theoretical

framework

is

developed

in

Sec.

under

which

the

problem

of

generating

multiport

encoded

worms

to

cover

given

multicast

destination

sets

in

unidirectional

MINs

is

presented

The

section

also

presents

three

algorithms

to

solve

this

problem

Section

shows

that

this

framework

and

the

proposed

algorithms

can

also

be

applied

to

bidirectional

MINs

Simulation

results

for

multicast

using

unicast

and

multiport

encoded

worms

on

unidirectional

and

bidirectional

MINs

are

presented

in

Sec.

and

Sec.

respectively

and

our

conclusions

are

presented

in

Sec.

Multicasting

using

Unicast

Worms

In

this

section

we

briefly

introduce

some

of

the

basic

unicast-based

multicast

algorithms

that

have

been

proposed

in

the

literature

We

also

look

at

different

encoding

schemes

used/proposed

for

MINs

to

specify

the

header

of

a

unicast

worm

Basic

Multicasting

Schemes

The

unicast-based

multicasting

schemes

typically

use

a

multi-phase

approach

The

basic

idea

in

these

schemes

is

to

derive

contention

free

phases

so

that

multiple

communications

can

occur

in

parallel

A

good

algorithm

for

unicast-based

multicast

is

the

U-min

algorithm

proposed

in

-LSB-

-RSB-

This

algorithm

proposed

for

fat-tree

networks

uses

a

destination

that

receives

a

message

in

any

phase

to

retransmit

the

message

in

succeeding

phases

The

coverage

of

the

destinations

therefore

proceeds

in

the

form

of

a

binomial

tree

with

the

number

of

sources

doubling

in

every

phase

Furthermore

the

algorithm

orders

the

destinations

to

which

a

worm

must

transmit

a

message

in

any

phase

so

that

the

worms

do

not

contend

for

communication

links

The

algorithm

thus

implements

a

multicast

to

d

destinations

in

-LRB-

d

-RRB-

e

phases

The

important

feature

of

the

U-min

algorithm

is

that

the

phases

are

contention

free

and

worms

across

phases

also

do

not

contend

for

the

same

links

Therefore

in

the

absence

of

other

network

traffic

one

phase

of

the

multicast

requires

only

as

much

time

as

it

takes

one

message

to

traverse

the

network

Figure

gives

an

example

of

a

unicast

based

multicast

using

the

U-min

algorithm

where

the

source

is

node

and

the

destination

set

is

Recently

a

variant

of

the

U-min

algorithm

-LRB-

called

C-min

-RRB-

has

been

proposed

by

Chiang

and

Ni

-LSB-

-RSB-

Although

this

algorithm

performs

no

better

than

U-min

for

a

single

multicast

it

performs

better

than

U-min

for

multiple

multicasts

Both

U-min

and

C-min

require

-LRB-

d

-RRB-

e

phases

to

cover

a

destination

set

of

size

d

This

raises

an

interesting

question

can

the

number

of

phases

be

reduced

by

allowing

a

worm

to

cover

more

than

one

destination

in

a

phase

The

multiport

encoding

based

approach

proposed

in

the

next

section

achieves

this

objective

and

reduces

the

multicast

-LRB-

a

-RRB-

-LRB-

b

-RRB-

Phase

Worm

Phase

Worm

Phase

Worm

Phase

Worm

Figure

Unicast

based

multicast

on

a

processor

unidirectional

cube

network

using

the

U-min

algorithm

-LRB-

a

-RRB-

the

binomial

tree

used

for

multicast

and

-LRB-

b

-RRB-

the

propagation

of

the

corresponding

unicast

worms

latency

tremendously

Before

we

introduce

multiport

encoding

based

worms

let

us

see

how

the

header

of

a

unicast

worm

is

encoded

in

current

generation

MINs

Encoding

Unicast

Worms

The

organization

of

a

worm

's

header

determines

the

complexity

of

the

routing

decision

to

be

made

at

every

switch

along

a

worm

's

path

A

simple

way

to

encode

a

worm

's

header

is

to

use

the

destination

ID

as

header

Every

switch

in

the

network

is

equipped

with

information

to

decide

which

output

links

lead

to

the

worm

's

destination

In

traditional

unidirectional

MINs

such

as

cube

omega

butterfly

etc.

only

one

of

the

output

ports

at

a

switch

can

lead

to

a

given

destination

However

multiple

choices

are

available

in

bidirectional

MINs

Every

switch

uses

the

information

it

possesses

to

route

the

worm

to

an

appropriate

output

port

This

method

of

routing

is

commonly

referred

to

as

dynamic

routing

since

the

worm

's

path

is

not

predetermined

but

instead

is

determined

by

the

switches

on

the

worm

's

path

as

the

worm

traverses

the

network

Figure

-LRB-

b

-RRB-

shows

this

form

of

encoding

for

the

worms

in

Fig.

-LRB-

a

-RRB-

-LRB-

a

-RRB-

Worm

-LRB-

d

-RRB-

-LRB-

b

-RRB-

-LRB-

c

-RRB-

Worm

Worm

Worm

Worm

Worm

Header

Flit

-LRB-

s

-RRB-

Data

Flits

Figure

Header

encoding

for

unicast

worms

-LRB-

a

-RRB-

two

unicast

worms

in

a

node

cube

network

-LRB-

b

-RRB-

explicit

encoding

of

the

destination

ID

-LRB-

c

-RRB-

encoding

the

port

ID

using

bits

and

-LRB-

d

-RRB-

encoding

the

port

ID

using

bits

Another

form

of

header

encoding

is

to

store

the

entire

path

of

the

worm

in

the

worm

's

header

The

worm

's

path

is

determined

a

priori

and

known

before

the

worm

enters

the

network

Such

a

form

of

routing

is

called

static

routing

Systems

like

the

IBM

-LSB-

-RSB-

use

such

an

encoding

Assuming

k

k

switches

are

used

Figs.

-LRB-

c

-RRB-

and

-LRB-

d

-RRB-

show

examples

of

such

encoding

using

and

k

bits

per

stage

respectively

It

is

to

be

noted

that

in

such

an

encoding

scheme

the

bits

corresponding

to

a

stage

can

be

discarded

once

a

routing

decision

has

been

made

Recently

a

new

scheme

called

source

based

adaptive

routing

has

been

proposed

in

-LSB-

-RSB-

Here

the

header

of

the

unicast

worm

is

organized

as

a

sequence

of

k-bit

strings

similar

to

the

example

in

Fig.

-LRB-

d

-RRB-

However

each

k-bit

string

may

have

more

than

one

bit

Positions

with

bits

represent

alternative

output

ports

to

which

the

unicast

worm

may

be

forwarded

The

decision

as

to

which

output

port

is

to

be

used

can

be

made

at

a

switch

based

on

local

traffic

conditions

Thus

the

source

determines

a

priori

a

set

of

paths

that

lead

to

a

given

destination

The

actual

path

taken

by

the

worm

is

determined

dynamically

This

approach

therefore

represents

a

compromise

between

the

approaches

described

above

Since

traditional

unidirectional

MINs

do

not

offer

multiple

paths

between

a

given

source

and

destination

such

k-bit

strings

can

not

be

used

to

capture

adaptivity

Nonetheless

they

can

encode

the

single

predetermined

path

leading

to

the

worm

's

destination

an

implementation

of

static

routing

Every

k-bit

string

determines

the

single

output

port

to

which

the

worm

is

to

be

forwarded

In

the

next

section

we

propose

a

variant

of

this

scheme

for

encoding

multidestination

worm

headers

and

develop

a

simple

multicasting

scheme

using

multiport

encoded

worms

Multicasting

using

Multiport

Encoded

Worms

In

this

section

we

introduce

the

basic

idea

of

multiport

encoding

of

multidestination

worm

headers

We

also

develop

a

multicasting

scheme

to

perform

efficient

multicast

using

multiport

encoded

worms

Multiport

Encoding

A

multidestination

worm

transfers

a

packet

from

a

source

to

multiple

destinations

-LSB-

-RSB-

A

multidestination

worm

's

header

must

carry

information

that

helps

the

worm

reach

all

its

destinations

In

a

MIN

based

system

a

multidestination

worm

requires

that

switches

possess

the

capability

to

forward

flits

arriving

at

one

input

port

of

the

switch

to

one

or

more

output

ports

of

the

switch

Assuming

k

k

switches

with

k

input

and

k

output

ports

a

k-bit

string

is

sufficient

for

a

switch

to

find

the

output

ports

to

which

an

arriving

multidestination

worm

is

to

be

forwarded

A

in

the

ith

position

of

the

k-bit

string

-LRB-

i

k

-RRB-

denotes

that

output

port

i

is

one

of

the

ports

to

which

the

worm

should

be

forwarded

-LSB-

-RSB-

The

basic

idea

of

multiport

encoding

is

to

use

a

sequence

of

such

k-bit

strings

to

encode

the

multidestination

worm

's

path

The

number

of

such

k-bit

strings

in

the

worm

header

is

equal

to

the

number

of

stages

in

the

network

A

switch

at

any

stage

uses

the

first

k-bit

string

in

the

worm

header

to

make

a

routing

decision

At

a

given

stage

all

switches

that

a

multidestination

worm

traverses

use

the

same

k-bit

string

to

make

their

routing

decisions

Every

switch

deletes

the

first

k-bit

string

of

the

worm

header

before

forwarding

the

remaining

portion

of

the

worm

header

and

the

data

flits

to

the

specified

output

ports

Figure

-LRB-

a

-RRB-

shows

a

multidestination

worm

header

using

multiport

encoding

for

the

node

cube

of

Fig.

-LRB-

b

-RRB-

It

is

assumed

in

this

figure

that

a

flit

is

bits

wide

Since

k

the

more

significant

four

bits

of

every

header

flit

have

been

set

to

zero

Note

that

if

k

flit

size

the

k-bit

string

may

span

multiple

flits

In

this

example

the

degrees

of

replication

of

the

worm

are

as

follows

and

Thus

this

single

worm

is

capable

of

covering

six

destinations

using

a

single

communication

startup

Multiport

encoding

assumes

greater

importance

in

the

light

of

the

fact

that

unicast

worms

can

always

be

encoded

using

a

similar

sequence

of

k-bit

strings

As

described

in

the

previous

section

such

an

approach

is

already

being

considered

-LSB-

-RSB-

to

support

adaptive

routing

in

MINs

with

unicast

worms

Thus

such

encoding

can

provide

dual

functionality

adaptivity

for

unicast

worms

and

a

mechanism

for

efficient

multicast

Siegel

et

al

-LSB-

-RSB-

describe

a

method

of

multicasting

that

is

equivalant

to

multiport

encoding

based

multicast

for

the

special

case

of

k

Multiport

encoding

based

multicast

has

also

been

introduced

in

the

context

of

packet

switched

cache

coherent

systems

-LSB-

-RSB-

Although

-LSB-

-RSB-

shows

how

multiport

encoding

can

be

used

in

a

single

switch

the

paper

does

not

extrapolate

the

use

of

multiport

encoding

for

a

MIN

Furthermore

the

paper

does

not

deal

with

the

question

of

how

multicast

can

be

performed

to

arbitrary

multicast

destination

sets

using

multiport

encoded

worms

Data

Flits

Header

flits

encoded

using

multiport

encoding

-LRB-

a

-RRB-

-LRB-

b

-RRB-

Header

flit

used

to

make

routing

decision

at

switch

Figure

Multiport

encoding

-LRB-

a

-RRB-

an

example

of

a

multidestination

worm

header

that

uses

multiport

encoding

and

-LRB-

b

-RRB-

path

of

the

corresponding

worm

along

with

the

header

flit

used

to

make

routing

decisions

at

a

switch

Multinomial

Tree

Based

Multicasting

All

switches

that

a

multiport

encoded

worm

traverses

at

a

stage

must

use

the

same

k-bit

string

to

make

their

routing

decisions

Thus

it

is

not

possible

to

cover

an

arbitrary

multicast

destination

set

with

a

single

multiport

encoded

worm

However

a

number

of

destination

sets

can

be

covered

by

a

single

multiport

encoded

worm

as

specified

below

Lemma

In

a

kn

node

MIN

system

consisting

of

n

stages

of

k

k

switches

-LRB-

-RRB-

n

different

multicast

destination

sets

can

be

covered

using

exactly

one

multiport

encoded

worm

with

a

single

communication

startup

Proof

At

a

given

switch

the

ports

to

which

an

incoming

message

is

to

be

forwarded

can

be

chosen

in

-LRB-

-RRB-

ways

Since

all

switches

at

a

given

stage

forward

a

multiport

encoded

message

to

the

same

set

of

output

ports

it

is

sufficient

to

consider

one

switch

alone

per

stage

Given

an

n

stage

network

therefore

there

are

-LRB-

-RRB-

n

distinct

ways

ports

can

be

chosen

for

a

multiport

encoded

worm

at

the

intermediate

switch

stages

each

of

which

cover

a

different

destination

set

For

example

by

Lemma

in

a

node

system

with

switches

different

destination

sets

can

be

covered

by

a

single

multiport

encoded

worm

In

a

node

system

with

switches

different

destination

sets

can

be

covered

by

a

single

multiport

encoded

worm

The

total

number

of

different

multicast

sets

in

system

with

N

node

is

The

number

of

destination

sets

that

can

be

covered

by

a

single

multiport

encoded

worm

is

therefore

a

small

fraction

of

the

total

number

of

different

multicast

sets

Multicasting

to

an

arbitrary

set

of

destinations

would

therefore

require

more

than

one

multidestination

worm

We

propose

a

hierarchical

scheme

to

achieve

a

multicast

with

reduced

latency

A

multiport

encoded

worm

that

covers

a

maximal

number

of

destinations

is

sent

from

the

source

node

in

the

first

phase

In

the

second

phase

a

set

of

multiport

encoded

worms

are

assigned

to

every

destination

covered

in

the

first

phase

along

with

the

original

source

Thus

at

the

beginning

of

every

phase

all

destinations

covered

in

previous

phases

plus

the

original

source

act

as

secondary

sources

Using

such

an

approach

the

number

of

phases

required

to

cover

a

given

destination

set

is

drastically

reduced

compared

to

a

unicast

based

scheme

Figure

shows

a

multicast

to

the

same

destination

set

as

that

of

Fig.

It

can

be

seen

that

the

number

of

phases

required

to

perform

the

multicast

reduces

from

phases

for

the

unicast

based

approach

to

phases

This

method

of

covering

destinations

using

multiport

worms

can

be

depicted

in

the

form

of

a

tree

as

shown

in

Fig.

-LRB-

a

-RRB-

Such

a

tree

is

referred

to

as

a

multinomial

tree

and

the

approach

to

performing

multicast

is

referred

to

as

a

multinomial

tree

based

approach

An

important

and

intriguing

problem

is

to

find

an

algorithm

that

given

a

destination

set

and

source

as

input

generates

multiport

encoded

worms

that

cover

all

nodes

in

the

given

destination

set

Section

presents

novel

algorithms

that

attempt

to

achieve

this

objective

-LRB-

a

-RRB-

Phase

worm

Phase

worm

-LRB-

b

-RRB-

Figure

A

multinomial

tree

based

multicast

using

multiport

encoded

multidestination

worms

for

the

example

multicast

set

of

Fig.

-LRB-

a

-RRB-

the

multinomial

tree

and

-LRB-

b

-RRB-

propagation

of

the

worms

Assignment

of

worms

to

the

various

phases

of

the

multicast

algorithm

and

assigning

them

to

different

secondary

sources

in

a

given

phase

are

also

important

problems

Figure

presents

the

multinomial

tree

based

approach

for

assigning

worms

to

various

sources

and

phases

of

the

multicast

Given

a

set

of

multiport

encoded

worms

we

rank

them

in

decreasing

order

of

the

number

of

destinations

being

covered

The

worm

that

covers

the

largest

number

of

destinations

is

assigned

to

the

source

in

the

first

phase

Every

destination

that

this

worm

covers

acts

as

a

secondary

source

in

the

second

phase

Worms

from

the

rank-ordered

list

are

then

assigned

to

the

original

as

well

as

the

set

of

secondary

sources

These

worms

are

transmitted

in

the

second

phase

This

process

continues

until

all

worms

are

taken

care

of

The

set

of

secondary

destinations

grows

with

every

phase

The

set

of

secondary

sources

at

the

end

of

a

phase

consists

of

the

union

of

the

set

of

destinations

covered

in

the

phase

and

the

set

of

secondary

sources

present

at

the

beginning

of

the

phase

Such

an

assignment

scheme

may

cause

the

worms

to

contend

for

links

and

increase

the

transmission

delay

However

this

simple

scheme

delivers

very

good

performance

as

described

in

Section

Currently

we

are

investigating

alternative

assignment

schemes

to

obtain

even

better

performance

It

is

to

be

noted

that

if

p

destinations

are

covered

by

a

single

multidestination

worm

on

the

average

then

this

scheme

is

capable

of

implementing

multicast

to

d

destinations

in

dlogp

-LRB-

d

-RRB-

e

phases

Table

shows

the

average

number

of

worms

required

to

cover

a

given

set

of

destinations

using

the

binomial

tree

based

approached

adopted

by

the

U-min

and

C-min

algorithms

and

the

multinomial

tree

based

approach

outlined

above

that

uses

multiport

worms

The

table

also

presents

the

number

of

phases

required

by

each

of

the

approaches

The

use

of

the

multinomial

tree

based

approach

reduces

the

number

of

worms

and

the

number

of

phases

quite

drastically

when

compared

to

the

binomial

tree

based

approach

Before

looking

at

the

algorithms

for

generating

multiport

encoded

worms

that

cover

a

given

multicast

destination

set

we

examine

the

architectural

details

for

implementing

such

multidestination

worms

Architectural

Considerations

for

Implementing

Multidestination

Worms

In

this

section

we

briefly

discuss

some

of

the

architectural

alternatives

that

are

present

for

implementing

multidestination

worms

on

multistage

interconnection

networks

We

then

describe

in

some

detail

a

switch

architecture

that

enables

the

performance

of

multidestination

based

multicast

Design

Criteria

There

are

two

orthogonal

design

considerations

that

must

be

taken

into

account

The

first

criterion

is

the

replication

mechanism

that

is

used

at

a

switch

The

second

criterion

is

the

flow

control

mechanism

used

There

are

two

main

replication

mechanisms

that

have

been

proposed

in

the

literature

-LSB-

-RSB-

These

mechanisms

are

known

as

synchronous

replication

and

asynchronous

replication

respectively

When

a

multidestination

worm

arrives

at

a

switch

it

must

be

Procedure

Multinomial

Input

A

set

of

multiport

encoded

worms

the

destinations

covered

by

these

worms

the

total

number

of

destinations

covered

and

the

source

Output

An

assignment

of

worms

to

the

source

and

secondary

sources

along

with

the

phase

information

Multinomial

-LRB-

-RRB-

f

SortedList

Sort

worms

in

descending

order

of

destinations

covered

SourceList

Source

phase

WormCount

TmpSrcList

OE

while

-LRB-

WormCount

j

SortedList

j

-RRB-

f

SrcCount

while

-LRB-

SrcCount

j

SourceList

j

-RRB-

f

SourceList

-LSB-

SrcCount

-RSB-

phase

phase

SourceList

-LSB-

SrcCount

-RSB-

worm

SortedList

-LSB-

W

ormCount

-RSB-

if

-LRB-

WormCount

j

SortedList

j

-RRB-

f

return

g

else

f

TmpSrcList

TmpSrcList

-LSB-

SortedList

-LSB-

W

ormCount

-RSB-

DestsCovered

WormCount

SrcCount

g

g

SourceList

SourceList

-LSB-

TmpSrcList

phase

g

g

Figure

The

multinomial

tree

based

algorithm

for

assigning

worms

to

various

sources

and

to

various

phases

of

the

multicast

Table

THE

AVERAGE

NUMBER

OF

WORMS

TAKEN

TO

COVER

A

SET

OF

DESTINATIONS

USING

MULTIPORT

AND

UNICAST

WORMS

GIVEN

IN

PARENTHESIS

IS

THE

NUMBER

OF

PHASES

REQUIRED

TO

COVER

THE

GIVEN

SET

OF

DESTINATIONS

USING

THE

MULTINOMIAL

AND

BINOMIAL

TREE

BASED

APPROACHES

No

System

Size

of

Switch

Size

Switch

Size

Dests

Mport

Unicast

Mport

Unicast

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

No

System

Size

of

Switch

Size

Dests

Mport

Unicast

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

forwarded

to

a

set

of

the

switch

's

output

ports

However

not

all

of

these

ports

may

be

available

when

the

worm

arrives

at

the

switch

Under

synchronous

replication

a

worm

waits

till

all

the

output

ports

it

has

requested

at

a

switch

become

available

before

forwarding

a

copy

to

any

of

the

switch

's

output

ports

Furthermore

all

of

a

worm

's

copies

at

a

stage

must

proceed

together

to

the

next

stage

of

the

network

Synchronous

replication

therefore

requires

some

feedback

mechanism

from

the

switches

to

the

originating

nodes

-LSB-

-RSB-

Under

asynchronous

replication

a

multidestination

worm

arriving

at

a

switch

uses

the

available

ports

to

forward

copies

of

the

flits

while

remaining

blocked

on

the

unavailable

ports

Asynchronous

replication

thus

has

the

advantage

of

doing

away

with

the

feedback

architecture

required

for

synchronous

replication

The

second

criterion

is

the

flow

control

mechanism

used

The

flow

control

technique

determines

the

buffer

space

available

for

the

worm

at

a

switch

Under

strict

wormhole

routing

the

worm

has

a

buffer

space

of

one

flit

available

to

it

Thus

only

one

flit

can

arrive

at

a

time

at

an

input

port

The

next

flit

can

arrive

only

after

the

first

flit

has

been

transmitted

and

the

buffer

is

freed

A

second

approach

is

to

enlarge

the

input

buffer

to

provide

space

for

multiple

flits

However

when

a

worm

is

blocked

a

larger

buffer

still

does

not

guarantee

that

the

entire

worm

can

be

buffered

within

that

switch

A

third

approach

is

to

guarantee

that

an

arriving

worm

will

have

enough

space

for

the

worm

to

be

fully

buffered

One

such

mechanism

is

virtual

cut-through

-LSB-

-RSB-

Note

that

in

both

wormhole

routing

and

virtual

cut-through

in

the

absence

of

contention

for

output

ports

worms

continue

to

propagate

on

a

flit

by

flit

basis

from

one

switch

to

the

next

Chiang

and

Ni

-LSB-

-RSB-

have

shown

that

both

synchronous

and

asynchronous

replication

mechanisms

can

cause

deadlock

when

used

in

conjunction

with

strict

wormhole

routing

Although

a

method

to

avoid

deadlock

in

systems

using

synchronous

replication

is

suggested

in

-LSB-

-RSB-

the

method

is

extremely

restrictive

and

is

inappropriate

for

variations

of

wormhole

routing

that

provide

more

intermediate

buffering

As

shown

in

-LSB-

-RSB-

the

root

of

the

deadlock

problem

is

the

dependence

between

the

multiple

output

ports

involved

in

a

worm

replication

This

dependence

results

from

blocked

flits

within

the

input

buffer

which

deny

the

entrance

of

the

entire

worm

into

this

buffer

An

input

buffer

that

is

larger

than

any

multidestination

worm

could

guarantee

that

the

entire

worm

will

eventually

enter

the

buffer

and

it

would

therefore

provide

a

guarantee

similar

to

virtual

cut-through

for

replicating

a

worm

We

propose

a

switch

architecture

that

uses

an

asynchronous

replication

mechanism

and

provides

buffers

at

least

as

large

as

the

largest

multidestination

packet

thus

ensuring

deadlock

freedom

This

architecture

is

described

next

A

Switch

Architecture

We

now

describe

a

switch

architecture

to

support

asynchronous

replication

and

virtual

cut-through-like

buffering

We

assume

that

worms

have

the

format

shown

in

Fig.

-LRB-

a

-RRB-

The

Tag

field

identifies

whether

the

worm

is

a

unicast

worm

or

a

multidestination

worm

The

Routing

Header

is

the

multiport

header

in

the

case

of

the

multidestination

worm

If

the

worm

is

a

unicast

worm

this

field

is

of

the

form

of

one

of

the

headers

discussed

in

Section

Figure

-LRB-

b

-RRB-

gives

an

organization

of

the

switch

architecture

that

is

assumed

Tag

Data

Flits

-LRB-

a

-RRB-

Routing

Header

Buffer

associated

with

every

input

link

Every

buffer

has

a

size

greater

than

the

maximum

worm

length

The

divisions

represent

chunks

Output

buffer

Size

chunk

Input

Links

Output

Links

SwitchCrossbar

-LRB-

b

-RRB-

Figure

-LRB-

a

-RRB-

A

typical

format

assumed

for

the

worms

in

the

system

and

-LRB-

b

-RRB-

the

switch

architecture

Every

input

link

is

associated

with

an

input

buffer

This

buffer

is

at

least

as

large

as

the

maximum

worm

length

allowed

in

the

system

Every

one

of

the

output

ports

can

access

any

of

the

input

buffers

Furthermore

an

output

buffer

may

read

any

part

of

a

packet

that

is

at

the

head

of

the

input

buffer

However

packets

are

processed

in

FIFO

order

An

output

port

has

access

only

to

the

packet

at

the

head

of

an

input

buffer

Upon

entering

the

input

port

flits

are

aggregated

into

k-flit

chunks

where

k

is

the

number

of

output

ports

Chunks

are

the

primary

unit

of

data

movement

within

the

switch

Each

chunk

is

first

written

into

the

input

buffer

and

is

subsequently

read

by

one

output

port

for

unicast

packets

or

by

multiple

output

ports

for

multidestination

packets

An

arbitration

mechanism

is

used

to

select

among

the

competing

output

ports

for

reading

from

an

input

buffer

Aggregation

of

flits

into

chunks

thus

allows

output

ports

to

read

a

chunk

out

only

every

k

cycles

and

yet

still

maintain

a

flit

per

cycle

bandwidth

Thus

up

to

k

output

ports

can

be

transmitting

up

to

k

different

packet

chunks

from

the

same

input

port

while

requiring

only

a

-LRB-

-RRB-

buffer

This

buffer

may

be

implemented

as

a

RAM

-LRB-

the

more

costly

alternative

would

be

a

write

k-read

RAM

that

buffers

flits

-RRB-

Since

packet

chunks

arrive

at

the

input

buffer

in

sequence

they

may

be

stored

at

successive

locations

in

the

buffer

However

our

assumption

that

a

k-flit

chunk

is

the

unit

of

transfer

across

the

switch

implies

that

a

wider

-LRB-

k-flit

wide

-RRB-

crossbar

has

to

be

used

Assuming

a

MUX

implementation

of

a

crossbar

Fig.

shows

an

organization

of

the

input

and

output

buffers

as

well

as

the

connectivity

between

them

Let

us

define

a

flit-MUX

as

a

flit-wide

k

to

MUX

The

suggested

organization

uses

k

large

MUXs

where

each

large

MUX

has

k

k-flit

wide

inputs

and

one

k-flit

wide

output

Each

of

these

larger

MUXs

is

actually

implemented

using

k

flit-MUXs

Such

an

implementation

incurs

high

wiring

complexity

Later

in

the

section

we

propose

two

alternative

organizations

to

reduce

this

complexity

MUX

Fromi/p

link

Fromi/p

link

Fromi/p

link

Fromi/p

link

Chunk-wide

Too/p

link

Output

Buffer

-LRB-

chunk

-RRB-

MUX

Too/p

link

Output

Buffer

-LRB-

chunk

-RRB-

MUX

Too/p

link

Output

Buffer

-LRB-

chunk

-RRB-

MUX

Too/p

link

Output

Buffer

-LRB-

chunk

-RRB-

Input

Buffer

-LRB-

RAM

-RRB-

Chunk-wideInput

Buffer

-LRB-

RAM

-RRB-

Chunk-wideInput

Buffer

-LRB-

RAM

-RRB-

Chunk-wideInput

Buffer

-LRB-

RAM

-RRB-

Register

-LRB-

chunk-wide

-RRB-

Register

-LRB-

chunk-wide

-RRB-

Register

-LRB-

chunk-wide

-RRB-

Register

-LRB-

chunk-wide

-RRB-

Figure

Switch

organization

to

implement

multidestination

worms

using

larger

MUXs

in

a

switch

The

thick

lines

represent

chunk-wide

-LRB-

k

f

litsize

-RRB-

lines

the

thin

lines

represent

flit-wide

lines

A

counter

is

associated

with

each

chunk

of

the

worm

that

arrives

in

the

input

buffer

The

counter

is

initialized

to

the

number

of

output

ports

that

the

worm

must

be

forwarded

to

at

that

switch

When

a

packet

reaches

the

head

of

the

input

buffer

the

input

port

raises

a

request

to

all

involved

output

ports

Each

request

remains

raised

until

the

associated

output

port

grants

the

request

and

begins

to

read

from

that

input

port

If

any

of

the

required

output

ports

are

free

they

begin

reading

chunks

from

the

input

buffer

and

transmitting

them

During

the

read

the

associated

chunk

counter

is

decremented

and

the

chunk

is

deleted

from

the

buffer

only

if

this

counter

becomes

zero

Multiple

output

ports

may

flit-MUX

may

actually

be

implemented

as

f

litsize

smaller

MUXs

each

of

which

has

k

inputs

and

one

output

read

chunks

from

the

input

buffer

in

sequence

allowing

a

multidestination

packet

to

be

forwarded

to

k

output

ports

while

maintaining

full

bandwidth

through

the

input

buffer

Buffer

space

is

freed

as

chunks

are

deleted

This

method

of

replication

is

deadlock

free

since

the

input

buffer

is

larger

than

any

multidestination

worm

therefore

providing

a

guarantee

similar

to

virtual

cut-through

as

discussed

in

the

previous

section

Reducing

Switch

Complexity

We

propose

two

modified

switch

architectures

in

this

section

that

can

reduce

the

cost

and

wiring

complexity

when

compared

to

the

previously

proposed

design

while

retaining

all

of

the

functionality

and

performance

Both

methods

reduce

the

width

of

the

MUXs

in

the

multicasting

circuit

Register-Staircase

Approach

This

method

uses

a

set

of

k

registers

associated

with

every

input

buffer

in

an

effort

to

reduce

the

MUX

width

A

schematic

of

the

proposed

architecture

for

a

switch

is

shown

in

Fig.

This

approach

makes

use

of

the

following

observations

to

achieve

performance

similar

to

that

achieved

by

the

model

specified

in

the

previous

section

-LRB-

i

-RRB-

all

output

ports

transmit

the

chunk

that

they

read

from

an

input

port

on

a

flit

by

flit

basis

to

their

corresponding

output

link

and

-LRB-

ii

-RRB-

output

ports

can

proceed

in

parallel

if

they

read

from

different

registers

MUX

Fromi/p

link

Chunk-wide

Input

Buffer

-LRB-

RAM

-RRB-

Fromi/p

link

Chunk-wide

Input

Buffer

-LRB-

RAM

-RRB-

Fromi/p

link

Chunk-wide

Input

Buffer

-LRB-

RAM

-RRB-

Fromi/p

link

Chunk-wide

Input

Buffer

-LRB-

RAM

-RRB-

MUX

MUX

MUX

MUX

Too/p

link

MUX

MUX

MUX

Output

Buffer

-LRB-

flit

-RRB-

Too/p

link

Output

Buffer

-LRB-

flit

-RRB-

Too/p

link

Output

Buffer

-LRB-

flit

-RRB-

Too/p

link

Output

Buffer

-LRB-

flit

-RRB-

Figure

The

switch

organization

under

the

Register-Staircase

Approach

The

basic

idea

is

as

follows

Let

us

assume

that

all

k

output

ports

want

to

read

from

the

same

input

buffer

The

output

ports

that

are

ready

take

turns

reading

from

the

input

buffer

Note

that

there

is

no

restriction

that

all

output

ports

have

to

be

ready

at

the

same

time

An

output

port

may

begin

contending

with

other

output

ports

for

the

input

buffer

whenever

it

is

ready

This

flexibility

allows

output

ports

to

be

reading

chunks

from

possibly

different

points

in

the

input

buffer

Flits

are

transferred

in

a

pipelined

fashion

as

depicted

in

Fig.

In

a

given

cycle

one

of

the

contending

output

ports

-LRB-

say

-RRB-

succeeds

in

reading

a

chunk

from

the

input

buffer

The

chunk

that

is

read

is

transferred

to

the

first

of

the

k

registers

-LRB-

-RRB-

in

this

cycle

In

the

next

cycle

the

next

available

output

port

-LRB-

say

-RRB-

reads

a

chunk

from

the

input

buffer

and

transfers

it

to

In

the

same

cycle

reads

the

first

flit

from

and

transfers

the

remaining

flits

of

to

In

the

third

cycle

an

output

port

may

read

from

the

input

buffer

and

transfer

a

chunk

to

and

reads

the

first

flit

from

and

transfers

the

remaining

flits

of

to

In

the

same

cycle

reads

the

first

flit

from

and

transfers

the

remaining

flits

of

to

while

putting

the

flit

it

read

in

the

previous

cycle

onto

the

output

link

This

pipelined

transfer

continues

with

each

output

port

transferring

a

flit

to

its

corresponding

output

link

once

every

cycle

reads

chunk

into

reads

first

flit

from

Remaining

reads

chunk

into

puts

flit

on

output

link

reads

next

flit

from

Remaining

flits

from

flits

from

reads

first

flit

from

Remaining

flits

from

reads

chunk

into

puts

flit

on

output

link

reads

next

flit

from

Remaining

flits

from

puts

flit

on

output

link

reads

next

flit

from

Remaining

flits

from

reads

chunk

into

reads

first

flit

from

Remaining

flits

from

puts

flit

on

output

link

reads

next

flit

from

and

next

chunk

into

puts

flit

on

output

link

reads

next

flit

from

Remaining

flits

from

puts

flit

on

output

link

reads

next

flit

from

Remaining

flits

from

reads

first

flit

from

Remaining

flits

from

CYCLE

CYCLE

CYCLE

CYCLE

CYCLE

Output

Ports

Figure

The

pipelined

transfer

of

flits

in

a

switch

under

the

register

staircase

approach

in

a

scenario

where

all

output

ports

read

concurrently

from

the

same

input

buffer

It

is

to

be

observed

that

such

an

implementation

reduces

the

size

of

the

k

output

port

MUXs

in

the

circuit

of

Fig.

although

the

total

number

of

flit-MUXs

has

increased

-LRB-

a

total

of

-LRB-

k

-RRB-

flit-MUXs

are

used

in

a

k

k

switch

-RRB-

Furthermore

there

is

no

loss

in

performance

when

compared

to

the

previous

approach

Most

importantly

we

have

moved

much

of

the

dense

wiring

complexity

into

the

separate

input

port

modules

avoiding

huge

multi-drop

buses

from

crossing

the

chip

to

the

output

ports

Split-RAM

Approach

We

now

present

an

alternative

organization

for

reducing

the

complexity

of

the

wiring

used

in

the

switches

The

basic

idea

behind

this

organization

is

to

use

k

flit-wide

RAMs

-LRB-

FWRs

-RRB-

instead

of

a

single

chunk-wide

RAM

The

k

flits

of

the

chunk

are

written

into

the

FWRs

with

the

ith

flit

of

the

chunk

being

written

to

the

ith

FWR

-LRB-

i

k

-RRB-

Figure

gives

a

schematic

of

the

proposed

design

The

output

ports

read

flits

from

each

of

the

RAMs

in

sequence

To

set

up

the

pipeline

an

output

port

reads

a

flit

from

one

of

the

FWRs

and

stores

the

flit

in

the

corresponding

register

In

the

next

cycle

the

output

port

reads

the

flit

from

this

register

and

stores

it

in

the

output

buffer

while

reading

the

next

flit

from

the

next

FWR

into

the

corresponding

register

In

every

succeeding

cycle

the

output

port

transfers

the

flit

in

its

output

buffer

to

the

output

link

reads

in

the

next

flit

from

an

appropriate

register

into

the

output

buffer

and

reads

in

the

next

flit

from

a

FWR

into

a

corresponding

register

The

destination

output

ports

of

the

multicast

packet

that

are

ready

read

flits

from

the

k

FWRs

in

this

pipelined

fashion

Once

the

pipeline

is

established

each

of

the

ready

destination

output

ports

transmits

one

flit

every

cycle

on

their

associated

output

links

The

number

and

size

of

MUXs

used

under

this

approach

is

exactly

the

same

as

with

the

register

staircase

approach

However

there

is

no

need

for

the

register

staircase

logic

Also

k

flit-wide

RAMs

are

needed

instead

of

a

single

chunkwide

RAM

This

implies

that

for

the

flits

to

be

read

from

the

k

FWRs

in

parallel

k

separate

copies

of

the

read

control

logic

are

required

It

is

to

be

noted

that

this

method

performs

as

well

as

the

method

using

the

register

staircase

Again

the

complexity

of

the

wiring

problem

between

the

input

ports

and

output

ports

is

much

improved

MUX

MUX

Too/p

link

Fromi/p

link

MUX

MUX

MUX

Fromi/p

link

Fromi/p

link

Fromi/p

link

-LRB-

Flit-wide

RAMs

-RRB-

Input

Buffer

Output

Buffer

-LRB-

flit

-RRB-

Too/p

link

Output

Buffer

-LRB-

flit

-RRB-

Too/p

link

Output

Buffer

-LRB-

flit

-RRB-

Too/p

link

Output

Buffer

-LRB-

flit

-RRB-

-LRB-

Flit-wide

RAMs

-RRB-

Input

Buffer

-LRB-

Flit-wide

RAMs

-RRB-

Input

Buffer

-LRB-

Flit-wide

RAMs

-RRB-

Input

Buffer

MUX

MUX

MUX

Figure

The

switch

organization

under

the

Split-RAM

Approach

Summary

We

have

presented

an

internal

switch

architecture

capable

of

providing

dynamic

multicast

via

a

crossbar

with

chunkwide

data

paths

The

switch

crossbar

requires

k

of

these

flit-MUXs

for

each

of

the

k

output

ports

for

a

total

of

flit-MUXs

Each

input

port

sends

a

k-flit

output

to

all

output

ports

creating

a

difficult

wiring

problem

aggravated

by

multi-drop

signals

We

suggested

two

alternatives

to

address

this

wiring

problem

while

maintaining

the

same

performance

In

each

alternative

-LRB-

k

-RRB-

flit-MUXs

are

used

but

each

input

port

sends

a

separate

single

flit

to

each

output

port

We

anticipate

that

the

dramatic

improvement

in

the

input

port

to

output

port

wiring

congestion

will

easily

offset

the

extra

MUX

cost

and

the

additional

logic

required

to

set

up

these

outputs

In

addition

each

of

the

suggested

alternatives

requires

only

a

register

at

each

output

port

compared

to

a

k-flit

register

used

in

the

original

approach

The

Register-Staircase

approach

will

be

advantageous

for

technologies

in

which

latches

are

relatively

cheap

while

the

SplitRAM

approach

will

be

advantageous

when

the

penalty

for

dividing

RAMs

into

multiple

thinner

RAMs

is

not

excessive

These

considerations

vary

with

technology

and

we

therefore

refrain

from

supporting

one

approach

over

the

other

In

summary

the

primary

modifications

we

propose

to

a

switch

supporting

only

unicast

messages

so

that

it

supports

multidestination

messages

are

-LRB-

a

-RRB-

an

input

buffer

per

input

port

which

is

larger

than

the

largest

worm

allowed

in

the

system

-LRB-

b

-RRB-

an

arbitration

mechanism

to

select

among

competing

output

ports

and

-LRB-

c

-RRB-

counters

associated

with

packet

chunks

Note

that

switches

such

as

those

of

IBM

already

provide

for

some

form

of

packet

buffering

and

for

a

mechanism

for

packet

chunking

The

changes

proposed

for

supporting

the

multidestination

mechanism

are

therefore

few

and

can

be

easily

incorporated

into

existing

switch

architectures

Algorithms

for

Generating

Multiport

Encoded

Worms

In

this

section

we

present

three

algorithms

of

differing

complexities

that

generate

multiport

encoded

worms

that

cover

a

given

destination

set

using

a

unidirectional

multistage

cube

network

-LSB-

-RSB-

as

example

We

first

present

some

definitions

and

properties

of

the

multistage

cube

network

and

introduce

the

notion

of

a

multiport

tree

Next

we

formally

state

the

problem

of

doing

multicast

using

multiport

encoded

worms

and

present

heuristics

that

attempt

to

solve

it

Some

Basic

Definitions

and

Properties

Let

a

directed

graph

G

-LRB-

V

E

-RRB-

represent

a

multistage

cube

network

The

vertices

V

of

the

graph

are

the

processing

nodes

and

the

switches

of

the

interconnection

network

The

set

E

of

the

directed

edges

of

the

graph

consists

of

the

unidirectional

links

of

the

network

Let

us

assume

that

the

cube

network

consists

of

N

kn

processing

nodes

interconnected

by

a

network

consisting

of

n

stages

of

k

k

switches

Figure

-LRB-

b

-RRB-

shows

the

graph

G

-LRB-

V

E

-RRB-

corresponding

to

the

cube

MIN

of

Fig.

-LRB-

a

-RRB-

Here

we

have

k

n

and

N

Let

us

group

the

vertices

of

the

graph

into

-LRB-

n

-RRB-

stages

the

input

side

forming

the

th

stage

and

the

output

side

forming

stage

n

Let

Vi

represent

the

vertices

of

the

ith

stage

-LRB-

i

n

-RRB-

The

vertices

within

a

stage

j

-LRB-

j

n

-RRB-

may

be

numbered

with

increasing

numbers

from

to

N

k

-LRB-

using

base

k

numbers

-RRB-

from

top

to

bottom

The

vertices

of

the

first

and

last

stages

may

be

numbered

from

to

N

-LRB-

using

base

k

numbers

-RRB-

The

edges

emanating

from

a

vertex

at

stage

j

-LRB-

j

n

-RRB-

are

similarly

numbered

from

to

k

It

can

be

observed

that

every

vertex

in

Vi

-LRB-

i

n

-RRB-

has

out-degree

equal

to

k

The

vertices

of

have

in-degree

and

out-degree

The

vertices

of

Vn

have

in-degree

and

out-degree

Let

us

introduce

the

following

definitions

with

respect

to

a

multicast

on

a

multistage

cube

network

Let

S

represent

the

vertices

corresponding

to

the

source

in

G

-LRB-

V

E

-RRB-

-LRB-

one

in

stage

and

the

other

in

stage

n

-RRB-

and

let

D

represent

the

vertices

corresponding

to

the

destinations

of

the

multicast

Definition

A

Pertinent

Subgraph

Gpert

-LRB-

V

E

-RRB-

of

the

graph

G

-LRB-

V

E

-RRB-

is

the

graph

formed

by

removing

from

G

-LRB-

V

E

-RRB-

-LRB-

i

-RRB-

all

the

vertices

except

those

belonging

to

S

-LSB-

D

and

-LRB-

ii

-RRB-

their

associated

links

Figure

-LRB-

c

-RRB-

shows

the

pertinent

subgraph

of

the

graph

shown

in

Figure

-LRB-

b

-RRB-

for

a

multicast

from

source

to

Definition

A

Destination

Cover

Tree

-LRB-

DCT

-RRB-

is

a

tree

subgraph

of

Gpert

-LRB-

V

E

-RRB-

whose

root

is

the

vertex

corresponding

to

the

source

processing

node

in

stage

and

whose

leaves

in

stage

n

are

the

vertices

corresponding

to

the

destinations

of

the

multicast

Definition

Consider

a

tree

subgraph

of

Gpert

-LRB-

V

E

-RRB-

whose

root

ffl

and

whose

leaves

ffl

Vn

Consider

the

set

of

outgoing

edges

from

any

vertex

of

this

tree

The

set

formed

by

the

numbers

of

these

edges

is

called

the

Branch

Set

of

the

Tree

Vertex

-LRB-

BSTV

-RRB-

Note

that

the

cardinality

of

a

BSTV

is

never

greater

than

k

the

switch

size

in

the

cube

MIN

In

Fig.

the

BSTVs

of

three

of

the

vertices

are

shown

-LRB-

vertex

of

stage

and

vertices

and

of

stage

-RRB-

Definition

A

Multiport

Tree

-LRB-

MT

-RRB-

is

a

tree

subgraph

of

Gpert

-LRB-

V

E

-RRB-

such

that

-LRB-

i

-RRB-

its

root

ffl

-LRB-

ii

-RRB-

its

vertices

vtree

i

at

stage

i

-LRB-

i

p

-RRB-

have

identical

BSTVs

and

-LRB-

iii

-RRB-

its

leaf

vertices

ffl

fVn

Dg

By

the

above

definition

the

vertices

of

a

multiport

tree

that

belong

to

the

same

stage

of

Gpert

-LRB-

V

E

-RRB-

share

a

common

BSTV

Definition

The

Branch

Sets

of

a

multiport

tree

are

the

BSTVs

for

each

stage

of

the

multiport

tree

one

BSTV

for

vertices

belonging

to

each

stage

-LRB-

a

-RRB-

-LRB-

b

-RRB-

-LRB-

c

-RRB-

The

Pertinent

Subgraph

Graph

corresponding

to

MIN

Edges

of

the

DCT

123 1230 1230 123

Stage

Stage

Stage

Stage

Stage

Stage

Stage

Stage

Vertices

of

the

DCT

Figure

Graph

representation

of

a

MIN

-LRB-

a

-RRB-

a

processor

cube

network

-LRB-

b

-RRB-

its

representation

as

a

graph

G

-LRB-

V

E

-RRB-

and

-LRB-

c

-RRB-

the

pertinent

subgraph

Gpert

-LRB-

V

E

-RRB-

of

G

-LRB-

V

E

-RRB-

The

edges

and

vertices

that

form

the

Destination

Cover

Tree

are

marked

with

dashed

arrows

and

striped

circles

respectively

of

the

Multiport

Tree

123 1230 1230 123

Stage

Stage

Stage

Stage

Multiport

Encoded

Header

Flits

Branch

Set

at

Stage

-LCB-

-RCB-

Branch

Set

at

Stage

-LCB-

-RCB-

Leaves

of

the

Multiport

Tree

-LCB-

Source

Set

-LCB-

BSTV

-LCB-

-RCB-

BSTV

-LCB-

-RCB-

BSTV

-LCB-

-RCB-

Vertices

of

the

Multiport

Tree

-RCB-

-RCB-

-LRB-

for

Stage

-RRB-

-LRB-

for

Stage

-RRB-

Figure

An

example

multiport

tree

illustrating

the

concepts

of

BSTV

and

Branch

Sets

A

multiport

tree

represents

the

path

of

a

corresponding

multidestination

worm

using

multiport

encoding

For

every

multiport

encoded

multidestination

worm

there

exists

a

corresponding

multiport

tree

and

vice-versa

The

Branch

Sets

of

a

multiport

tree

define

the

encoding

of

a

multidestination

worm

header

Since

the

cardinalities

of

the

Branch

Sets

are

no

greater

than

k

and

every

element

of

a

Branch

Set

is

a

number

in

the

range

-LSB-

k

-RSB-

we

can

represent

the

elements

of

a

Branch

Set

using

a

k-bit

string

with

the

ith

bit

set

to

if

element

i

belong

to

the

Branch

Set

or

to

otherwise

These

kbit

strings

constitute

the

multiport

encoded

worm

header

Figure

shows

a

multiport

tree

to

cover

a

part

of

the

example

multicast

set

shown

in

Fig.

-LRB-

b

-RRB-

The

Branch

Sets

of

this

multiport

tree

are

and

The

multiport

header

of

the

corresponding

multidestination

worm

consists

of

two

flits

and

Note

that

a

multiport

tree

whose

Branch

Sets

are

all

singletons

corresponds

to

a

unicast

worm

The

multiport

encoding

mechanism

can

therefore

also

be

used

for

encoding

the

path

of

a

unicast

worm

Definition

The

Size

of

a

multiport

tree

is

the

number

of

leaves

in

the

multiport

tree

The

number

of

leaves

in

a

multiport

tree

is

the

product

of

the

cardinalities

of

its

Branch

Sets

The

size

of

a

multiport

tree

gives

the

number

of

destinations

covered

by

the

corresponding

multiport

worm

Definition

Given

two

sets

A

asg

and

B

btg

the

Concatenating

Cross

Product

A

B

is

the

set

asbtg

The

operator

is

associative

Lemma

The

set

of

vertices

that

are

the

leaves

of

a

multiport

tree

is

given

by

BSTVn

where

BSTVi

-LRB-

i

n

-RRB-

is

the

branch

set

corresponding

to

stage

i

of

the

tree

Proof

The

statement

follows

from

the

definition

of

a

multiport

tree

the

definition

of

BSTV

and

the

property

of

the

cube

networks

specified

above

Statement

of

the

problem

Based

on

the

definitions

above

we

now

state

the

problem

of

generating

multiport

encoded

worms

We

want

to

find

a

set

of

MT

s

-LRB-

SMT

-RRB-

such

that

-LRB-

i

-RRB-

The

intersection

of

the

set

of

leaves

of

any

two

MT

s

that

belong

to

SMT

is

OE

-LRB-

ii

-RRB-

The

union

of

the

set

of

leaves

of

all

MT

s

that

belong

to

SMT

is

the

set

of

leaves

of

DCT

-LRB-

i.e.

the

vertices

corresponding

to

the

destinations

of

the

multicast

-RRB-

and

-LRB-

iii

-RRB-

The

cardinality

of

SMT

is

minimal

Note

that

a

solution

to

this

problem

leads

to

a

set

of

multidestination

worms

that

cover

the

given

multicast

destination

set

since

every

MT

of

SMT

corresponds

to

a

multiport

encoded

multidestination

worm

Note

also

that

properties

-LRB-

i

-RRB-

and

-LRB-

ii

-RRB-

are

sufficient

for

performing

multicast

-LRB-

i.e.

to

generate

multiport

encoded

worms

that

cover

the

given

multicast

destination

set

-RRB-

However

property

-LRB-

iii

-RRB-

may

be

necessary

for

performing

an

optimal

multicast

using

multiport

encoded

worms

In

the

remaining

part

of

this

paper

we

will

concentrate

mainly

on

achieving

properties

-LRB-

i

-RRB-

and

-LRB-

ii

-RRB-

and

present

heuristics

that

produce

a

SMT

of

very

low

cardinality

To

solve

the

problem

stated

above

wemake

use

of

the

following

source

invariant

properties

of

the

cube

network

-LSB-

-RSB-

The

following

property

shows

that

the

path

to

a

destination

from

any

source

can

be

determined

using

only

the

digits

in

the

destination

ID

Property

In

a

cube

network

choosing

the

edge

dn

at

stage

of

Gpert

-LRB-

V

E

-RRB-

edge

dn

at

stage

of

Gpert

-LRB-

V

E

-RRB-

etc.

leads

us

to

the

destination

dn

-LRB-

di

k

i

n

-RRB-

By

inference

from

the

above

property

the

path

to

multiple

destinations

-LRB-

exemplified

by

the

multiport

tree

-RRB-

can

be

similarly

inferred

from

the

digits

in

the

destination

IDs

The

following

property

states

that

the

path

represented

by

-LRB-

and

the

vertices

covered

by

-RRB-

a

multiport

tree

are

source

invariant

Property

Given

a

multiport

tree

T

with

root

rT

and

with

the

set

of

vertices

vTi

ffl

Vi

at

stage

i

-LRB-

i

n

-RRB-

form

a

multiport

tree

by

using

a

root

rT

The

set

of

vertices

i

at

stage

i

-LRB-

i

n

-RRB-

is

obtained

by

following

the

edges

specified

in

the

BSTVs

of

vTi

The

tree

thus

obtained

has

the

same

leaf

vertices

as

T

In

terms

of

multidestination

worms

the

above

property

may

be

interpreted

as

follows

Given

a

multiport

tree

we

can

find

the

corresponding

multiport

encoded

worm

using

the

Branch

Sets

of

the

multiport

tree

The

above

property

of

cube

networks

means

that

the

multiport

encoded

worm

so

obtained

will

cover

the

same

set

of

destinations

no

matter

which

node

the

worm

originates

from

Approaches

for

solving

the

problem

In

this

section

we

present

algorithms

that

generate

sets

of

multiport

trees

obeying

properties

-LRB-

i

-RRB-

and

-LRB-

ii

-RRB-

of

the

problem

statement

above

We

first

examine

a

method

for

constructing

a

multiport

tree

Next

we

present

a

simple

algorithm

called

the

MTS

algorithm

that

uses

this

method

to

find

a

SMT

We

then

outline

a

greedy

approach

for

finding

a

SMT

and

present

a

method

for

forming

larger

multiport

trees

In

the

next

section

we

present

two

more

algorithms

-LRB-

called

the

MTM

and

MTE

algorithms

respectively

-RRB-

that

use

this

greedy

approach

to

find

a

SMT

A

Method

for

Constructing

a

Multiport

Tree

Let

us

examine

the

process

of

finding

a

multiport

tree

that

covers

a

subset

of

a

given

set

of

destinations

Let

the

destination

set

be

represented

by

Dn

where

-LRB-

Di

k

-RRB-

for

all

i

such

that

-LRB-

i

n

-RRB-

Given

a

set

of

destinations

that

differ

only

in

their

ith

digit

positions

we

can

construct

a

multiport

tree

that

has

singleton

BSTVs

for

vertices

of

every

stage

except

the

-LRB-

n

i

-RRB-

th

stage

The

BSTV

at

stage

-LRB-

n

i

-RRB-

is

given

by

the

set

of

Di

values

At

all

other

stages

the

BSTV

consists

of

the

common

value

of

Dj

-LRB-

j

n

j

i

-RRB-

shared

by

the

destinations

of

the

set

For

example

in

a

node

MIN

with

switches

destinations

and

may

be

covered

by

a

single

multiport

encoded

worm

since

they

only

differ

in

the

first

digit

position

-LRB-

the

least

significant

digit

is

the

digit

position

-RRB-

The

multiport

trees

with

the

vertices

corresponding

to

these

nodes

as

leaves

has

the

branch

set

for

vertices

ffl

vtree

for

vertices

ffl

vtree

and

for

vertices

ffl

vtree

Recall

that

the

expression

vtree

i

is

used

to

denote

the

set

of

tree

vertices

belonging

to

stage

i

of

Gpert

-LRB-

V

E

-RRB-

This

leads

to

the

following

Lemma

A

set

of

destinations

that

differ

in

only

the

ith

digit

position

-LRB-

i

n

-RRB-

can

be

covered

by

a

single

multiport

encoded

worm

Multiport

Tree?Simple

-LRB-

MTS

-RRB-

Algorithm

The

above

Lemma

leads

us

to

a

simple

algorithm

for

generating

multiport

trees

that

satisfy

properties

-LRB-

i

-RRB-

and

-LRB-

ii

-RRB-

of

the

problem

statement

Input

The

graph

Gpert

-LRB-

V

E

-RRB-

the

source

and

the

set

of

destination

vertices

Output

A

set

of

multiport

trees

that

satisfy

properties

-LRB-

i

-RRB-

and

-LRB-

ii

-RRB-

of

the

problem

statement

Divide

the

given

destination

set

into

subsets

such

that

-LRB-

a

-RRB-

the

subsets

are

mutually

exclusive

and

-LRB-

b

-RRB-

the

destinations

in

each

subset

differ

in

only

one

common

digit

position

By

Lemma

each

of

these

subsets

represent

vertices

that

form

leaves

of

multiport

trees

For

each

of

the

multiport

trees

the

Branch

Sets

for

vertices

at

stage

i

of

the

tree

can

be

obtained

by

grouping

together

the

different

dn?i

-LRB-

i

n

-RRB-

digits

of

the

destinations

in

the

corresponding

subset

These

sets

are

singletons

for

vertices

of

all

stages

except

one

The

Branch

Sets

can

be

used

to

obtain

the

corresponding

multidestination

worm

header

as

described

above

To

perform

the

first

step

of

the

MTS

algorithm

we

can

decide

on

an

arbitrary

digit

position

-LRB-

say

the

least

significant

digit

-RRB-

in

the

destination

vertex

numbers

The

subsets

consist

of

the

destinations

that

differ

only

in

this

digit

For

example

given

a

destination

set

we

may

choose

the

least

significant

digit

in

these

numbers

to

help

form

the

subsets

The

subsets

thus

formed

are

and

We

may

have

alternatively

chosen

the

most

significant

digit

giving

us

the

subsets

and

Each

of

these

subsets

represent

processing

nodes

that

can

be

covered

by

a

single

multiport

encoded

worm

It

is

to

be

noted

that

the

choice

of

the

digit

position

may

affect

the

number

of

worms

required

to

cover

a

given

multicast

destination

set

By

arbitrarily

choosing

a

digit

for

grouping

destinations

we

are

adopting

a

heuristic

A

more

complex

alternative

is

to

investigate

the

choice

of

digit

positions

that

result

in

a

minimal

number

of

subsets

It

must

be

noted

however

that

for

a

given

set

of

multiport

encoded

worms

covering

the

same

number

of

destinations

the

worms

that

replicate

at

a

later

stage

use

fewer

links

and

therefore

present

fewer

opportunities

for

contention

Choosing

the

least

significant

digit

as

basis

for

grouping

destinations

ensures

that

the

worms

generated

replicate

only

at

the

last

stage

switches

Therefore

choosing

the

last

digit

to

group

destinations

ensures

that

the

number

of

links

used

in

routing

the

corresponding

multiport

worms

to

their

destinations

is

minimal

and

this

reduces

the

opportunities

for

link

contention

Choosing

the

least

significant

digit

as

basis

for

grouping

destinatons

is

therefore

a

good

heuristic

although

choosing

another

digit

for

grouping

may

result

in

fewer

worms

the

opportunities

for

link

contention

are

also

greater

Since

only

those

destinations

that

differ

in

the

single

chosen

digit

position

are

grouped

into

a

subset

using

the

MTS

algorithm

the

subsets

have

a

cardinality

less

than

or

equal

to

k

This

leads

to

Lemma

No

more

than

k

destinations

can

be

covered

by

any

of

the

multiport

encoded

worms

generated

using

the

MTS

Algorithm

Theorem

A

multinomial

tree

based

multicast

to

d

destinations

using

the

multiport

encoded

worms

generated

by

the

MTS

Algorithm

requires

-LRB-

i

-RRB-

at

least

dlogk

-LRB-

d

-RRB-

e

phases

and

-LRB-

ii

-RRB-

no

more

than

-LRB-

d

-RRB-

e

phases

to

perform

multicast

Proof

The

first

part

can

be

directly

inferred

from

Lemma

In

the

best

case

the

set

of

d

destinations

can

be

broken

down

into

groups

of

k

destinations

each

of

which

can

be

covered

by

a

single

multidestination

worm

Using

a

multinomial

tree

based

approach

k

destinations

can

be

covered

by

a

single

worm

originating

from

the

source

in

the

first

phase

At

the

end

of

the

first

phase

therefore

k

nodes

-LRB-

including

the

source

-RRB-

have

the

message

The

k

destinations

and

the

source

can

each

cover

k

destinations

in

the

next

phase

Therefore

at

the

end

of

the

second

phase

k

-LRB-

k

-RRB-

-LRB-

k

-RRB-

-LRB-

k

-RRB-

nodes

have

the

message

These

nodes

may

each

cover

k

destinations

in

the

next

phase

and

so

on

This

means

that

at

least

dlogk

-LRB-

d

-RRB-

e

phases

are

required

to

cover

the

d

destinations

To

prove

the

second

part

let

us

consider

a

worst

case

multicast

destination

set

where

no

two

of

the

destinations

differ

in

exactly

one

digit

position

The

MTS

algorithm

applied

to

this

destination

set

yields

multiport

trees

each

having

a

single

leaf

-LRB-

destination

-RRB-

The

corresponding

multiport

encoded

worms

are

nothing

but

unicast

worms

The

multinomial

tree

based

multicast

therefore

reduces

to

a

binomial

tree

based

multicast

which

requires

at

most

-LRB-

d

-RRB-

e

phases

A

Greedy

Approach

for

finding

a

SMT

and

a

Method

for

Constructing

Larger

Multiport

Trees

Figure

presents

an

outline

of

a

greedy

approach

for

finding

a

SMT

It

is

to

be

noted

that

this

is

only

a

heuristic

and

does

not

guarantee

that

the

SMT

is

of

minimal

cardinality

-LRB-

property

-LRB-

iii

-RRB-

of

the

problem

statement

-RRB-

However

the

SMT

satisfies

the

other

requirements

specified

in

the

problem

statement

The

important

part

of

the

greedy

solution

to

the

problem

is

the

finding

of

the

largest

multiport

tree

In

the

following

discussion

we

attempt

to

solve

this

problem

Greedy

-LRB-

-RRB-

f

Find

the

largest

size

multiport

tree

-LRB-

the

multiport

tree

with

the

largest

number

of

leaves

-RRB-

that

is

a

subtree

of

DCT

Remove

the

leaves

of

the

multiport

tree

and

the

edges

associated

with

them

from

DCT

Repeat

the

above

steps

till

all

leaves

of

DCT

have

been

removed

g

Figure

Outline

of

a

greedy

approach

to

find

a

set

of

multiport

trees

We

can

devise

methods

for

grouping

the

destinations

into

larger

sets

that

are

the

leaves

of

a

single

multiport

tree

than

the

sets

obtained

by

Lemma

We

define

below

a

merge

operation

by

which

two

multiport

trees

can

be

combined

to

form

a

single

tree

We

then

state

the

property

that

the

two

multiport

trees

should

possess

so

that

the

tree

formed

by

applying

the

merge

operation

on

them

is

a

multiport

tree

Definition

Let

two

distinct

multiport

trees

and

have

the

following

properties

-LRB-

i

-RRB-

and

have

identical

BSTVs

for

all

stages

except

one

and

-LRB-

ii

-RRB-

their

BSTVs

are

mutually

exclusive

in

the

stage

where

their

BSTVs

differ

These

two

trees

are

defined

to

mergable

under

the

merge

operation

The

resultant

tree

is

defined

as

follows

-LRB-

i

-RRB-

its

BSTVs

at

any

stage

are

given

by

the

union

of

the

BSTVs

of

the

vertices

at

the

corresponding

stage

in

and

and

-LRB-

ii

-RRB-

its

leaves

are

the

union

of

their

leaves

Lemma

The

merged

tree

is

a

multiport

tree

In

the

next

section

we

present

an

algorithm

that

uses

Definition

to

form

multiport

trees

of

larger

size

than

those

generated

by

the

MTS

algorithm

Multiport

Tree?Merge

-LRB-

MTM

-RRB-

Algorithm

The

MTM

algorithm

uses

the

greedy

approach

outlined

in

Fig.

It

begins

by

finding

a

set

of

base

multiport

trees

using

the

MTS

algorithm

and

tries

to

merge

these

according

to

Definition

to

form

multiport

trees

of

larger

size

Then

the

remaining

steps

of

the

greedy

approach

are

followed

Let

us

examine

in

detail

the

steps

followed

by

the

algorithm

shown

in

Fig.

The

algorithm

follows

an

iterative

process

At

the

beginning

of

every

iteration

a

square

grid

of

width

equal

to

the

number

of

multiport

trees

is

built

An

entry

at

position

-LRB-

i

j

-RRB-

in

the

grid

is

set

to

if

multiport

tree

i

and

multiport

tree

j

are

not

mergable

Otherwise

the

entry

in

the

grid

is

set

to

the

size

of

the

multiport

tree

formed

by

merging

multiport

trees

i

and

j.

From

this

grid

a

new

set

of

multiport

trees

is

obtained

by

-LRB-

a

-RRB-

eliminating

and

storing

those

multiport

trees

that

are

not

mergable

with

any

of

the

other

multiport

trees

and

-LRB-

b

-RRB-

merging

pairs

of

multiport

trees

so

that

the

size

of

the

resultant

multiport

tree

is

maximal

under

the

constraint

that

a

multiport

tree

can

not

appear

in

more

than

one

pair

Note

that

the

set

of

multiport

trees

at

the

end

of

every

iteration

is

of

equal

or

lower

cardinality

than

the

set

of

multiport

trees

at

the

beginning

of

every

iteration

When

the

cardinality

is

lower

the

same

procedure

is

repeated

with

the

new

set

of

multiport

trees

when

the

cardinality

is

equal

it

indicates

that

no

more

merging

can

be

done

and

the

procedure

is

terminated

The

algorithm

is

presented

more

formally

in

Fig.

Procedure

MTM

Input

The

graph

Gpert

-LRB-

V

E

-RRB-

the

source

and

the

set

of

destinations

vertices

Output

A

reduced

set

of

multiport

trees

that

satisfy

properties

-LRB-

i

-RRB-

and

-LRB-

ii

-RRB-

of

the

problem

statement

MTM

-LRB-

-RRB-

f

ResultSet

OE

T

reeSet

MTS

-LRB-

-RRB-

while

-LRB-

T

reeSet

OE

-RRB-

f

Form

a

square

grid

of

width

j

T

reeSet

j.

grid

-LRB-

i

j

-RRB-

if

Mergable

-LRB-

treei

treej

-RRB-

grid

-LRB-

i

j

-RRB-

Sizeof

-LRB-

treei

treej

-RRB-

otherwise

For

all

i

such

that

row

i

and

column

i

contain

all

entries

f

T

reeSet

T

reeSet

ftreeig

ResultSet

ResultSet

-LSB-

ftreeig

g

Repeat

till

grid

contains

all

zero

entries

f

Find

a

maximal

entry

in

the

grid

-LRB-

let

's

assume

this

entry

is

at

position

-LRB-

i

j

-RRB-

-RRB-

T

reeSet

T

reeSet

-LSB-

ftreei

treejg

T

reeSet

T

reeSet

ftreeig

T

reeSet

T

reeSet

ftreejg

grid

-LRB-

i

-RRB-

grid

-LRB-

i

-RRB-

grid

-LRB-

j

-RRB-

grid

-LRB-

j

-RRB-

g

For

all

treei

not

in

ResultSet

and

that

has

not

been

merged

f

These

trees

were

mergable

with

other

trees

however

those

trees

were

eliminated

because

of

their

being

merged

with

other

trees

T

reeSet

T

reeSet

ftreeig

ResultSet

ResultSet

-LSB-

ftreeig

g

g

return

-LRB-

ResultSet

-RRB-

g

Figure

The

MTM

algorithm

for

generating

multiport

encoded

worms

that

cover

a

given

destination

set

For

example

consider

a

multicast

to

the

set

Following

the

MTS

algorithm

and

using

the

least

significant

digit

to

group

destinations

this

set

can

be

divided

into

the

subsets

and

The

Branch

Sets

corresponding

to

each

of

the

subsets

are

for

the

subset

for

the

subset

for

the

subset

and

for

the

subset

Of

these

only

the

multiport

trees

corresponding

to

the

subsets

and

can

be

merged

as

per

Lemma

giving

us

a

multiport

tree

with

leaves

and

Branch

Sets

We

can

thus

reduce

the

number

of

multiport

trees

generated

to

using

the

MTM

algorithm

-LRB-

from

using

the

MTS

algorithm

alone

-RRB-

It

can

be

observed

that

the

number

of

trees

generated

using

the

MTM

algorithm

is

always

less

than

or

equal

to

the

number

of

trees

generated

using

the

MTS

algorithm

Note

that

for

a

multicast

to

the

destination

set

-LRB-

the

example

used

in

the

rest

of

the

paper

-RRB-

the

MTM

algorithm

generates

the

same

multiport

trees

as

the

MTS

algorithm

as

of

the

multiport

trees

generated

by

the

MTS

algorithm

can

be

merged

according

to

Definition

to

generate

a

new

multiport

tree

The

largest

tree

generated

using

the

MTM

algorithm

may

be

smaller

than

the

largest

possible

multiport

tree

in

a

given

destination

set

The

size

of

the

largest

multiport

tree

generated

by

the

MTM

algorithm

depends

on

the

trees

generated

by

the

MTS

algorithm

and

trees

are

generated

by

the

MTS

algorithm

without

keeping

in

mind

their

future

mergability

For

example

given

the

destination

set

we

could

have

generated

base

multiport

trees

with

the

following

destination

sets

and

Applying

the

MTM

algorithm

on

this

set

of

multiport

trees

results

in

a

reduced

set

of

multiport

trees

of

which

the

largest

has

the

destination

set

The

size

of

the

largest

multiport

tree

has

thus

improved

from

to

-LRB-

Previously

the

set

of

multiport

trees

generated

by

the

MTM

algorithm

for

this

destination

set

was

the

same

as

the

set

of

multiport

trees

generated

by

the

MTS

algorithm

the

largest

of

these

sets

was

-RRB-

Such

an

increase

in

the

size

of

the

largest

multiport

tree

may

reduce

the

number

of

phases

required

for

multicast

We

present

next

an

algorithm

that

generates

multiport

trees

by

searching

for

a

largest

multiport

tree

more

exhaustively

Multiport

Tree?Exhaustive

-LRB-

MTE

-RRB-

Algorithm

The

MTE

algorithm

searches

the

destination

set

exhaustively

for

the

largest

subset

of

destinations

that

can

be

covered

by

a

single

multiport

tree

As

opposed

to

a

naive

solution

the

MTE

algorithm

uses

heuristics

to

reduce

the

time

needed

to

find

the

largest

multiport

tree

Before

examining

the

working

of

the

MTE

algorithm

let

us

first

examine

a

naive

exhaustive

search

solution

to

the

problem

of

finding

the

largest

multiport

tree

Naive

Algorithm

Let

us

consider

Lemma

In

addition

to

giving

us

the

total

number

of

multiport

trees

that

may

be

formed

in

a

given

network

Lemma

also

provides

a

hint

for

a

method

for

determining

them

This

method

is

as

follows

Form

a

set

of

all

possible

combinations

of

output

ports

that

may

be

chosen

at

any

stage

of

the

MIN

Given

a

set

kg

of

output

ports

there

exist

-LRB-

-RRB-

possible

combinations

of

ports

that

can

be

chosen

at

a

stage

-LRB-

note

that

at

least

one

port

must

always

be

chosen

-RRB-

These

combinations

are

given

by

the

elements

of

the

power

set

of

kg

minus

the

element

OE

which

is

the

set

Let

us

refer

to

these

combinations

as

port

alternatives

A

multiport

tree

may

be

formed

by

choosing

one

of

the

-LRB-

-RRB-

port

alternatives

for

each

stage

The

set

of

all

possible

multiport

trees

is

a

set

of

combinations

where

every

combination

is

formed

by

choosing

one

of

the

-LRB-

-RRB-

port

alternatives

at

every

stage

As

described

in

Lemma

there

are

-LRB-

-RRB-

n

such

combinations

in

an

n

stage

network

giving

us

-LRB-

-RRB-

n

possible

multiport

trees

A

multiport

tree

is

said

to

be

feasible

for

a

given

destination

set

if

the

multiport

tree

covers

a

subset

of

the

given

set

of

destinations

A

naive

algorithm

can

be

designed

to

determine

the

feasible

multiport

trees

that

cover

a

given

destination

set

First

enumerate

all

possible

-LRB-

-RRB-

n

multiport

trees

using

the

method

described

above

Next

arrange

the

trees

in

descending

order

of

size

Finally

proceed

by

going

down

this

list

of

multiport

trees

and

examining

whether

the

multiport

tree

is

feasible

for

the

given

destination

set

The

multiport

tree

that

covers

a

largest

subset

of

the

destination

set

can

be

determined

in

this

manner

and

the

destinations

that

are

covered

by

this

tree

can

be

removed

from

further

consideration

With

the

remaining

destinations

we

can

again

go

through

the

list

of

multiport

trees

and

find

the

largest

one

that

is

feasible

This

procedure

can

be

continued

until

all

destinations

have

been

covered

This

naive

algorithm

follows

the

greedy

approach

outlined

in

Fig.

MTE

Algorithm

The

MTE

algorithm

tries

to

reduce

the

number

of

multiport

trees

enumerated

and

examined

It

uses

a

heuristic

to

reduce

the

number

of

port

alternatives

considered

for

every

stage

The

MTE

algorithm

is

presented

in

Fig.

We

will

briefly

explain

the

implication

of

some

of

the

algorithm

steps

below

The

sub-procedures

corresponding

to

each

step

of

the

algorithm

are

presented

in

in

Figs

to

in

Appendix

A

We

explain

the

steps

of

the

algorithm

with

the

help

of

an

example

of

a

multicast

to

the

destination

set

in

a

node

cube

system

with

switches

-LRB-

n

N

and

k

-RRB-

Building

a

Modified

Karnaugh

Map

The

MTE

algorithm

uses

a

representation

of

the

destinations

in

a

n-dimensional

array

-LRB-

for

a

system

with

kn

nodes

-RRB-

with

k

entries

along

each

dimension

This

array

contains

kn

positions

one

position

corresponding

to

each

node

in

the

system

A

position

in

the

array

can

be

identified

by

the

n

digit

number

used

to

identify

the

corresponding

node

In

terms

of

Cartesian

coordinates

the

position

corresponding

to

a

node

an

is

-LRB-

an

an

-RRB-

The

position

corresponding

to

a

node

is

set

to

if

the

node

is

a

destination

of

the

multicast

otherwise

it

is

set

to

The

algorithm

uses

properties

of

the

array

by

virtue

of

which

some

destinations

may

be

covered

by

a

single

multiport

tree

These

properties

are

similar

to

the

grouping

properties

of

entries

in

a

Karnaugh

Map

-LSB-

-RSB-

We

therefore

refer

to

the

array

as

a

Modified

Karnaugh

Map

-LRB-

MKM

-RRB-

Fig.

in

Appendix

A

describes

the

procedure

for

forming

the

MKM

more

formally

This

representation

is

shown

in

Fig.

-LRB-

a

-RRB-

for

the

example

multicast

destination

set

presented

above

Procedure

MTE

Input

A

set

of

destinations

D

Output

A

set

of

multiport

trees

that

satisfy

properties

-LRB-

i

-RRB-

and

-LRB-

ii

-RRB-

of

the

problem

statement

MTE

-LRB-

-RRB-

f

BuildModifiedKarnaughMap

-LRB-

-RRB-

for

D

Fig.

in

Appendix

A

while

-LRB-

D

OE

-RRB-

f

Grid

BuildGrid

-LRB-

-RRB-

choosing

any

value

for

dimension

Fig.

in

Appendix

A

maxima

FindMaxima

-LRB-

Grid

-RRB-

Fig.

in

Appendix

A

FindEnhancedGroup

-LRB-

maxima

-RRB-

Fig.

in

Appendix

A

Store

enhanced

group

in

TreeGroup

Remove

destinations

covered

by

enhanced

group

from

D

-LRB-

set

the

corresponding

entries

in

MKM

to

-RRB-

g

return

-LRB-

TreeGroup

-RRB-

g

Figure

The

MTE

algorithm

for

generating

multiport

encoded

worms

that

cover

a

given

destination

set

Identifying

a

row

of

the

MKM

Consider

a

row

of

the

MKM

entries

along

any

dimension

d

-LRB-

we

use

row

with

the

quotes?to

signify

that

this

row

belongs

to

a

multidimensional

array

as

opposed

to

a

two

dimensional

array

with

which

the

term

is

usually

associated

We

use

row?without

the

quotes?when

we

refer

to

the

row

of

a

grid

that

is

definitely

two

dimensional

-RRB-

A

row

along

a

dimension

d

of

the

MKM

may

be

represented

by

a

-LRB-

n

-RRB-

digit

number

This

-LRB-

n

-RRB-

digit

number

is

the

number

formed

by

removing

the

dth

digit

-LRB-

this

is

the

only

digit

in

which

these

destinations

differ

-RRB-

from

the

destinations

represented

in

the

row

For

example

the

row

with

destinations

and

described

above

is

identified

as

by

the

number

Removing

the

dth

digit

from

any

one

destination

in

the

row

is

sufficient

for

finding

the

row

number

Alternatively

a

row

may

be

indentified

uniquely

by

an

n-tuple

where

the

entry

in

the

tuple

corresponding

to

the

digit

position

where

the

destinations

in

the

row

differ

is

set

to

In

our

example

therefore

the

row

with

destinations

and

may

alternatively

be

identified

by

the

-LRB-

-RRB-

We

will

use

both

the

representations

described

above

for

identifying

rows

in

the

following

discussion

The

representation

used

should

be

clear

from

the

context

The

entries

that

are

in

a

row

along

dimension

d

correspond

to

destinations

that

differ

in

their

dth

digit

position

alone

Therefore

by

Lemma

these

destinations

can

be

covered

by

a

single

multiport

tree

For

example

consider

the

row

-LRB-

-RRB-

There

are

three

entries

in

this

row

corresponding

to

destinations

and

These

destinations

differ

in

their

digit

position

alone

and

can

be

covered

by

a

multiport

tree

with

Branch

Sets

and

It

can

be

observed

that

a

correspondence

exists

between

a

dimension

of

the

MKM

a

digit

in

the

destination

IDs

and

a

stage

in

the

MIN?the

dimension

d

corresponds

to

digit

d

in

the

destination

IDs

and

this

digit

is

used

for

routing

at

the

stage

-LRB-

n

d

-RRB-

of

the

that

this

stage

refers

to

the

stage

of

switches

in

the

MIN

It

is

different

from

the

stage

used

in

the

context

of

vertices

in

the

graphical

representation

described

earlier

in

this

section

In

Fig

-LRB-

a

-RRB-

the

leftmost

stage

of

switches

in

the

MIN

forms

stage

and

the

next

stage

of

switches

forms

stage

-LRB-

a

-RRB-

-LRB-

b

-RRB-

XX

XX

XXXXXXXXXXXXX

XXXXXXX

X

XXXXX

XXXXXXXX

X

X

X

X

XXXXXX

X

X

X

XXXXXX

X

X

X

X

XXXXXXXXX

XXXX

XX

X

XX

X

X

X

X

XX

X

X

XX

X

X

XXXXXXX

XXX

XX

X

XXXX

XX

XXXX

X

X

XX

XX

XXXX

XX

X

X

XX

X

X

XXXXXX

Dimension

Dimension

Dimension

Figure

An

illustration

of

the

steps

of

the

MTE

algorithm

for

an

example

destination

set

The

MKM

and

grid

corresponding

to

the

first

iteration

are

shown

Forming

a

Grid

Let

us

choose

any

dimension

and

form

a

square

grid

using

the

rows

along

that

dimension

according

to

the

procedure

described

in

Fig.

-LRB-

Appendix

A

-RRB-

This

square

grid

is

of

size

kn

kn

An

entry

in

the

grid

at

position

-LRB-

i

j

-RRB-

represents

the

number

of

's

that

are

in

the

same

position

in

rows

i

and

j

along

the

chosen

dimension

Note

that

this

grid

is

always

a

two-dimensional

array

irrespective

of

the

dimensionality

of

the

MKM

The

rows

and

columns

of

this

grid

are

represented

by

-LRB-

n

-RRB-

digit

numbers

Figure

-LRB-

b

-RRB-

shows

an

example

of

such

a

grid

formed

by

selecting

rows

along

dimension

of

the

MKM

The

entries

in

the

grid

provide

a

hint

for

deciding

which

multiport

trees

-LRB-

represented

by

rows

along

the

chosen

dimension

-RRB-

should

be

combined

in

the

bid

to

form

the

largest

possible

multiport

tree

For

example

in

Fig.

-LRB-

b

-RRB-

the

entry

at

any

position

-LRB-

i

j

-RRB-

is

formed

by

doing

a

position-wise

AND

operation

on

the

ith

and

jth

row

of

entries

along

dimension

of

the

MKM

If

i

j

the

entry

is

set

to

twice

the

number

of

's

in

the

result

of

this

AND

operation

as

an

estimate

of

the

number

of

destinations

that

can

be

covered

by

combining

the

two

different

multiport

trees

represented

by

i

and

j

If

i

j

the

entry

is

set

to

the

number

of

's

in

the

result

of

this

AND

operation

reflecting

the

number

of

destinations

covered

by

the

single

multiport

tree

represented

by

i.

Note

that

in

the

former

case

-LRB-

i

j

-RRB-

the

trees

i

and

j

may

not

be

mergable

in

terms

of

Definition

However

there

may

exist

subtrees

of

the

trees

that

can

be

merged

according

to

Definition

The

entries

in

the

grid

are

only

used

to

derive

a

subset

of

output

ports

that

should

be

considered

to

form

port

alternatives

as

described

later

in

the

section

These

entries

provide

a

hint

about

the

size

of

the

tree

that

can

be

formed

by

merging

the

subtrees

of

the

multiport

trees

corresponding

to

the

rows

For

example

the

entry

at

position

-LRB-

-RRB-

is

obtained

by

performing

a

position-wise

AND

of

the

entries

in

rows

-LRB-

-RRB-

and

-LRB-

-RRB-

There

are

three

's

in

the

result

of

this

AND

operation

Since

i

j

the

number

of

ones

is

doubled

and

the

value

is

stored

in

the

position

-LRB-

-RRB-

Note

that

the

same

value

would

have

been

stored

in

the

entry

-LRB-

-RRB-

However

the

grid

is

symmetric

and

we

therefore

ignore

the

lower

half

of

the

grid

-LRB-

represented

by

X

's

-RRB-

Find

Maxima

The

entries

in

the

grid

represent

pairs

of

rows

along

the

chosen

dimension

-LRB-

these

rows

are

identified

by

the

row

and

column

in

the

grid

on

which

the

entry

lies

-RRB-

In

our

heuristic

we

only

consider

the

entries

with

maximal

values

in

the

grid

as

outlined

in

Fig.

-LRB-

Appendix

A

-RRB-

These

maxima

are

presented

in

boldface

and

underlined

in

Fig.

-LRB-

b

-RRB-

The

digits

in

the

rows

corresponding

to

these

maxima

help

us

find

a

set

of

output

ports

corresponding

to

the

stages

of

the

MIN

Since

the

row

numbers

are

-LRB-

n

-RRB-

digit

numbers

they

provide

us

with

a

clue

about

the

possible

output

ports

that

may

be

used

at

-LRB-

n

-RRB-

of

the

n

stages

we

still

have

to

find

the

possible

output

ports

at

the

stage

corresponding

to

the

dimension

along

which

we

examine

the

rows

Our

method

is

better

illustrated

as

follows

Consider

the

maximal

entries

in

the

grid

of

Fig.

-LRB-

b

-RRB-

The

rows

that

these

maximal

entries

correspond

to

are

obtained

by

the

row

and

column

number

of

the

grid

on

which

they

lie

The

maxima

in

our

example

correspond

to

the

rows

and

The

digits

in

these

row

IDs

help

us

find

the

set

of

output

ports

that

we

should

consider

at

stage

and

stage

of

the

MIN

Before

finding

the

set

of

output

ports

we

first

form

a

set

of

all

rows

associated

with

the

maxima

in

the

grid

This

set

is

the

union

of

the

pair

sets

presented

above

and

has

the

value

in

our

example

Find

Enhanced

Group

Given

the

set

of

rows

associated

with

the

maximal

entries

in

the

grid

we

proceed

as

follows

to

find

the

set

of

output

ports

that

we

should

consider

at

stages

and

of

the

MIN

We

separate

the

digits

of

every

entry

in

this

set

and

store

them

in

-LRB-

n

-RRB-

different

sets

In

our

example

the

sets

obtained

by

this

method

are

and

The

set

represents

the

set

of

output

ports

to

be

considered

at

stage

-LRB-

The

elements

of

this

set

correspond

to

the

digit

of

the

set

of

destinations

being

combined

and

therefore

are

used

for

stage

n

d

-RRB-

Similarly

the

set

represents

the

set

of

output

ports

to

be

considered

at

stage

of

the

MIN

We

still

have

to

find

the

set

of

output

ports

to

be

considered

at

the

stage

corresponding

the

dimension

along

which

the

rows

are

being

considered

-LRB-

stage

corresponding

to

dimension

in

our

example

-RRB-

It

can

be

observed

that

the

number

of

output

ports

considered

at

any

stage

is

less

than

or

equal

to

k.

Under

the

naive

approach

described

earlier

all

k

output

ports

are

considered

at

every

stage

while

enumerating

the

multiport

trees

Thus

the

heuristic

described

above

has

the

potential

to

reduce

the

number

of

multiport

trees

that

are

enumerated

-LRB-

and

examined

-RRB-

as

compared

to

the

naive

algorithm

The

naive

approach

uses

the

port

alternatives

at

every

stage

to

find

all

possible

multiport

trees

and

then

searches

for

the

largest

of

the

trees

that

is

feasible

In

contrast

the

MTE

algorithm

combines

the

forming

of

the

multiport

tree

and

the

determination

of

whether

the

multiport

tree

is

feasible

Assuming

that

we

are

considering

rows

along

dimension

d

of

the

MKM

we

now

describe

the

step

which

helps

us

find

the

largest

feasible

multiport

tree

and

the

port

alternative

corresponding

to

stage

-LRB-

n

d

-RRB-

Consider

the

port

alternatives

for

all

stages

except

the

stage

-LRB-

n

d

-RRB-

A

set

of

-LRB-

n

-RRB-

digit

numbers

can

be

formed

by

choosing

port

alternatives

one

from

the

alternatives

at

each

of

the

-LRB-

n

-RRB-

stages

apart

from

the

-LRB-

n?d

-RRB-

th

stage

The

-LRB-

n

-RRB-

digit

numbers

are

formed

by

choosing

one

digit

from

the

port

alternative

chosen

for

every

stage

For

example

consider

the

sets

of

output

ports

corresponding

to

stages

and

described

in

the

previous

paragraph

The

port

alternatives

corresponding

to

the

set

are

and

The

port

alternatives

corresponding

to

the

set

are

and

Choosing

the

port

alternative

corresponding

to

stage

and

the

port

alternative

corresponding

to

stage

we

can

form

the

following

-LRB-

n

-RRB-

digit

numbers

and

These

-LRB-

n

-RRB-

digit

numbers

correspond

to

rows

in

the

grid

We

now

perform

a

position-wise

AND

operation

on

the

entries

along

these

rows

The

set

of

positions

where

the

result

of

this

AND

operation

is

is

the

port

alternative

corresponding

to

stage

-LRB-

n

d

-RRB-

If

this

set

is

not

the

multiport

tree

formed

is

feasible

If

the

set

is

there

exists

no

port

alternative

in

stage

-LRB-

n

d

-RRB-

such

that

the

multiport

tree

formed

with

the

chosen

alternatives

of

the

other

stages

is

feasible

The

size

of

the

feasible

multiport

tree

can

be

obtained

by

the

expression

NumOnes

NumRows

where

NumOnes

is

the

number

of

positions

with

in

the

result

of

the

AND

operation

and

NumRows

is

the

number

of

rows

on

which

the

AND

operation

is

performed

Therefore

using

the

procedure

described

above

a

near-largest

feasible

multiport

tree

can

be

found

as

the

tree

with

a

maximal

value

for

NumOnes

NumRows

In

our

example

a

position-wise

AND

of

rows

and

results

in

all

's

as

row

along

dimension

has

all

entries

Therefore

there

exists

no

port

alternative

at

stage

such

that

the

multiport

tree

formed

with

port

alternative

corresponding

to

stage

and

port

alternative

corresponding

to

stage

is

feasible

The

port

alternative

corresponding

to

stage

and

corresponding

to

stage

gives

us

rows

and

A

position-wise

AND

of

these

rows

results

in

a

maximal

value

of

NumOnes?NumRows

with

NumOnes

and

NumRows

The

positions

in

which

the

result

of

this

AND

operation

have

bits

are

The

port

alternatives

chosen

are

therefore

at

stage

at

stage

and

at

stage

and

the

destinations

covered

are

Figure

in

Appendix

A

presents

this

procedure

more

formally

Next

the

above

destinations

are

removed

from

the

destination

set

D

and

the

corresponding

positions

in

MKM

are

set

to

A

grid

is

again

formed

using

the

rows

along

dimension

and

the

procedure

is

repeated

Destinations

are

covered

in

the

second

iteration

destinations

are

covered

in

the

third

iteration

and

destination

is

covered

in

the

fourth

-LRB-

and

last

-RRB-

iteration

It

is

to

be

noted

that

the

method

presented

above

is

only

a

heuristic

There

may

be

cases

where

the

largest

possible

multiport

tree

is

not

obtained

using

this

heuristic

We

performed

an

even

more

exhaustive

search

considering

rows

along

every

dimension

and

choosing

the

largest

tree

obtained

However

we

found

that

this

approach

does

only

slightly

better

and

does

not

justify

the

considerable

increase

in

computational

complexity

Comparison

of

the

Algorithms

Let

us

analyze

the

cost-performance

tradeoff

between

the

proposed

three

algorithms

The

MTS

algorithm

is

the

simplest

of

the

three

The

MTM

algorithm

is

slightly

more

complex

and

the

MTE

algorithm

is

the

most

complex

of

the

three

due

to

the

exhaustive

search

it

performs

Table

shows

the

average

number

of

worms

required

to

perform

multicast

to

destination

sets

of

different

sizes

along

with

the

number

of

phases

required

to

perform

multicast

using

the

multinomial

tree

based

approach

discussed

in

Sec.

It

can

be

seen

that

the

difference

in

the

number

of

worms

required

for

multicasts

using

the

MTM

and

MTE

algorithms

is

quite

small

despite

the

fact

that

the

MTM

algorithm

is

much

simpler

Although

the

MTS

algorithm

requires

a

greater

number

of

worms

to

perform

multicast

the

number

of

phases

required

for

multicast

is

comparable

to

the

MTM

and

MTE

schemes

Table

THE

AVERAGE

NUMBER

OF

WORMS

NEEDED

TO

COVER

A

GIVEN

DESTINATION

SET

USING

THE

MTS

MTM

AND

MTE

ALGORITHMS

GIVEN

IN

PARENTHESIS

IS

THE

AVERAGE

NUMBER

OF

PHASES

REQUIRED

TO

COVER

THE

GIVEN

DES

TINATION

SET

USING

THE

MULTINOMIAL

TREE

BASED

APPROACH

DISCUSSED

IN

SEC.

No

System

Size

of

Switch

Size

Switch

Size

Dests

MTS

MTM

MTE

MTS

MTM

MTE

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

No

System

Size

of

Switch

Size

Dests

MTS

MTM

MTE

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

It

is

to

be

noted

that

for

a

node

system

with

switches

the

number

of

worms

required

to

cover

certain

destination

sets

using

the

MTE

algorithm

actually

exceeds

the

number

of

worms

required

using

the

MTS

and

MTM

algorithms

This

shows

that

a

greedy

approach

may

not

be

optimal

in

terms

of

the

number

of

worms

required

to

cover

a

given

destination

set

However

the

largest

worm

generated

using

the

MTE

algorithm

covers

a

larger

set

of

destinations

than

the

corresponding

worm

generated

using

the

MTM

and

MTS

algorithms

Therefore

the

number

of

phases

in

the

multicast

using

the

MTE

algorithm

remains

low

Section

presents

actual

multicast

latencies

obtained

by

simulation

using

worms

generated

by

each

of

the

three

algorithms

Application

of

Multiport

Encoding

to

Bidirectional

MINs

We

have

so

far

discussed

multiport

encoding

in

the

context

of

unidirectional

MINs

However

multiport

encoding

is

quite

general

and

can

also

be

applied

to

bidirectional

MINs

In

this

section

we

describe

why

the

algorithms

described

in

the

previous

sections

can

be

used

without

any

change

for

the

purpose

of

generating

multiport

encoded

headers

in

bidirectional

-LRB-

bidi

-RRB-

MINs

In

Sec.

we

present

simulation

results

for

multicast

on

bidi

MINs

and

show

that

performance

gains

similar

to

those

achieved

for

unidirectional

MINs

can

be

obtained

Figure

-LRB-

a

-RRB-

shows

a

node

bidi

butterfly

network

Figure

-LRB-

b

-RRB-

shows

its

equivalent

representation

using

unidirectional

links

and

switches

It

can

be

observed

that

the

interconnection

of

the

links

of

the

bidi-MIN

on

its

reverse

network

corresponds

to

the

interconnection

pattern

of

a

unidirectional

cube

MIN

Therefore

the

properties

of

the

cube

network

described

in

Sec.

hold

in

the

reverse

network

of

the

bidi-MIN

In

particular

a

given

multiport

encoded

worm

will

cover

the

same

set

of

destinations

starting

from

any

switch

in

the

leftmost

stage

of

reverse

network

of

Fig.

-LRB-

b

-RRB-

and

headed

towards

the

rightmost

stage

of

the

network

Furthermore

the

same

set

of

destinations

will

be

covered

as

those

covered

by

the

corresponding

worm

in

a

unidirectional

cube

network

The

multiport

header

used

to

route

a

multiport

encoded

worm

to

a

set

of

destinations

in

a

unidirectional

cube

MIN

can

therefore

be

used

to

route

the

worm

through

the

reverse

network

of

the

bidi-butterfly

MIN

to

the

same

set

of

destinations

Thus

the

multiport

encoded

worms

to

cover

a

given

multicast

set

in

the

bidi-MIN

can

be

found

using

the

algorithms

presented

in

Sec.

Stage

Stage

Forward

Network

Reverse

Network

-LRB-

a

-RRB-

Forward

Network

Reverse

Network

Stage

Stage

Stage

Stage

-LRB-

b

-RRB-

Figure

-LRB-

a

-RRB-

A

bidirectional

butterfly

MIN

and

-LRB-

b

-RRB-

an

equivalent

representation

using

unidirectional

links

and

switches

However

unlike

with

unidirectional

networks

there

are

no

processors

attached

to

the

leftmost

stage

of

switches

on

the

reverse

network

of

Fig.

-LRB-

b

-RRB-

A

worm

must

therefore

first

make

its

way

from

a

processor

to

the

leftmost

switch

stage

in

the

reverse

network

It

is

sufficient

for

the

worm

to

reach

any

of

the

switches

in

this

leftmost

switch

stage

and

therefore

considerable

adaptivity

exists

on

the

forward

network

Upon

reaching

the

leftmost

stage

of

the

reverse

network

the

worm

can

then

propagate

as

it

would

in

a

unidirectional

cube

network

to

reach

its

destinations

The

propagation

of

the

worm

in

the

forward

network

can

be

described

as

follows

The

worm

can

be

adaptively

routed

to

the

last

stage

of

the

bidi-MIN

As

we

discuss

later

in

this

section

traveling

all

the

way

to

this

stage

may

not

always

be

necessary

The

worm

may

choose

any

link

along

its

forward

path

that

leads

it

to

the

last

stage

of

switches

The

choice

of

link

may

be

made

depending

on

local

traffic

conditions

This

adaptivity

can

be

captured

using

multiport

encoding

as

described

in

-LSB-

-RSB-

and

as

mentioned

in

Sec.

Each

header

flit

has

a

bit

to

distinguish

if

it

is

to

be

interpreted

for

adaptive

routing

or

multiport

routing

In

the

case

of

the

former

bits

with

value

in

the

k-bit

string

header

represent

alternative

output

ports

to

which

the

worms

may

be

forwarded

In

the

latter

case

the

worm

is

forwarded

to

every

output

port

whose

corresponding

bit

in

the

k-bit

string

is

Alternatively

the

worm

can

carry

a

count

of

the

number

of

stages

it

has

to

move

in

the

forward

direction

before

turning

around

This

count

is

decremented

by

every

switch

along

the

worm

's

forward

path

The

worm

turns

back

at

the

switch

where

this

count

reaches

zero

This

count

field

can

then

be

stripped

After

the

worm

turns

back

it

traverses

the

reverse

network

replicating

in

switches

along

its

path

using

the

multiport

encoded

part

of

the

header

It

may

be

noted

that

a

worm

need

not

always

travel

up

to

the

last

stage

of

the

bidi-MIN

before

turning

around

Traveling

up

to

a

least

common

ancestor

stage

-LSB-

-RSB-

corresponding

to

the

given

set

of

destinations

and

the

source

is

sufficient

As

described

in

-LSB-

-RSB-

in

the

bidi-butterfly

network

the

least

common

ancestor

stage

between

a

source

and

destination

node

is

given

by

the

position

of

the

leftmost

digit

in

which

their

IDs

differ

The

least

common

ancestor

stage

between

a

source

and

a

set

of

destinations

may

similarly

be

found

This

least

common

ancestor

stage

is

the

maximum

of

the

least

common

ancestor

stages

between

the

source

and

each

of

the

destinations

Simulation

Results

for

Multicast

on

Unidirectional

MINs

In

this

section

we

study

the

performance

of

the

above

algorithms

using

a

C++

CSIM

based

simulator

testbed

The

simulator

models

the

switch

architecture

described

in

Section

and

implements

flit-level

simulation

It

is

flexible

enough

to

be

modeled

for

a

range

of

different

topological

and

technological

parameters

Experiments

We

assumed

system

parameters

representing

the

current

trend

in

technology

Link

propagation

time

-LRB-

tp

-RRB-

was

assumed

to

be

microseconds

for

Mbytes/sec

links

The

switch

routing

time

for

the

header

flit

and

the

switch

propagation

time

for

all

the

flits

were

assumed

to

be

microseconds

each

This

value

was

assumed

to

be

the

same

for

unicast

and

multidestination

worms

a

fair

assumption

given

the

simplicity

of

routing

a

multiport

encoded

worm

Every

input

buffer

was

assumed

to

be

of

size

flits

The

chunk

size

described

in

Section

was

assumed

to

be

flits

for

experiments

performed

with

switches

and

flits

for

experiments

performed

with

switches

These

values

are

similar

to

those

used

in

the

IBM

-LSB-

-RSB-

We

assume

a

single

read

port

model

as

discussed

in

Sec.

We

performed

experiments

using

the

MTS

MTM

and

MTE

algorithms

and

compared

the

results

with

the

unicast

based

C-Min

algorithm

-LSB-

-RSB-

For

the

MTS

algorithm

we

chose

the

least

significant

digit

of

the

destination

ID

to

group

destinations

We

studied

the

effect

of

startup

time

-LRB-

ts

-RRB-

on

multicast

latency

using

different

values

for

ts

and

microseconds

These

values

represent

the

current

as

well

as

future

trends

of

communication

startup

time

We

performed

experiments

for

and

switch

sizes

We

evaluated

broadcast

and

multicast

on

and

unidirectional

cube

systems

Finally

we

studied

the

effect

of

message

length

on

multicast

latency

Every

data

point

was

simulated

for

runs

to

derive

average

values

Impact

of

Startup

Time

Figure

shows

the

overall

comparison

of

multicast

and

broadcast

latency

for

three

different

start-up

values

It

can

be

observed

that

the

multidestination

scheme

performs

to

times

better

than

the

unicast

based

multicast

Even

for

a

very

low

startup

of

microsec

multicast

using

multiport

encoded

worms

outperforms

unicast

based

multicast

by

a

factor

of

up

to

for

destination

sets

of

size

or

more

The

gain

is

greater

for

startups

of

and

microseconds

Also

larger

switch

sizes

help

in

reducing

multicast

latency

with

multiport

encoded

worms

because

each

worm

has

greater

potential

to

cover

larger

destination

subsets

Such

benefits

do

not

arise

for

unicast-based

schemes

As

the

number

of

destinations

participating

in

a

multicast

increases

the

multidestination

based

scheme

reduces

multicast

latency

This

demonstrates

the

true

potential

of

the

multidestination

based

approach

It

can

be

seen

that

the

difference

in

performance

-LRB-

in

terms

of

multicast

latency

-RRB-

between

the

multicasts

based

on

the

worms

generated

by

the

MTM

and

MTE

algorithms

is

quite

small

For

large

destination

sets

and

for

low

values

of

startup

it

can

be

seen

that

the

MTM

algorithm

actually

outperforms

the

MTE

algorithm

based

multicast

The

reason

for

this

is

that

the

MTE

algorithm

generates

multiport

worms

that

cover

the

largest

destination

sets

These

destinations

tend

to

be

spread

out

over

the

system

To

cover

these

destinations

the

worms

replicate

at

the

earlier

stages

of

the

MIN

and

hence

tend

to

utilize

a

greater

number

of

links

thereby

increasing

the

opportunities

for

link

contention

to

occur

The

multiport

worms

generated

by

the

MTM

algorithm

tend

to

replicate

at

later

stages

of

the

MIN

these

worms

therefore

utilize

fewer

links

and

present

fewer

opportunities

for

link

contention

The

worms

generated

by

the

MTS

algorithm

only

replicate

at

the

last

stage

switches

and

therefore

contend

the

least.The

multinomial

tree

based

multicast

based

on

the

worms

generated

by

the

MTS

algorithm

performs

quite

well

keeping

in

mind

the

extreme

simplicity

of

the

algorithm

Impact

of

System

Size

Figure

shows

how

our

scheme

compares

with

unicast-based

scheme

for

a

MIN

Here

only

switches

are

considered

because

an

exact

system

can

not

be

built

with

switches

Here

also

it

can

be

observed

that

the

multidestination

scheme

outperforms

the

unicast-based

scheme

The

reduction

in

multicast

latency

is

up

to

a

factor

of

Similar

to

the

system

the

latency

reduces

as

the

degree

of

multicast

increases

It

can

be

observed

that

multicasts

to

more

than

destinations

based

on

the

worms

generated

using

the

MTS

algorithm

incurs

almost

constant

latency

for

ts

and

microsecs

As

was

observed

with

the

node

system

for

multicasts

to

large

destination

sets

with

low

values

of

startup

multicasts

using

worms

generated

by

the

MTM

algorithm

perform

better

than

multicasts

using

worms

generated

by

the

MTE

algorithm

This

effect

can

be

clearly

observed

for

multicasts

to

more

than

destinations

for

a

startup

time

of

microsecs

The

MTM

algorithm

thus

performs

almost

as

well

if

not

better

than

Multicast

Latency

-LRB-

microsec

-RRB-

Number

of

destinations

ts

microsec

L

flits

n

C-Min

k

C-Min

k

mdest

MTS

k

mdest

MTM

k

mdest

MTE

k

mdest

MTS

k

mdest

MTM

k

mdest

MTE

k

Multicast

Latency

-LRB-

microsec

-RRB-

Number

of

destinations

ts

microsec

L

flits

n

C-Min

k

C-Min

k

mdest

MTS

k

mdest

MTM

k

mdest

MTE

k

mdest

MTS

k

mdest

MTM

k

mdest

MTE

k

Multicast

Latency

-LRB-

microsec

-RRB-

Number

of

destinations

ts

microsec

L

flits

n

C-Min

k

C-Min

k

mdest

MTS

k

mdest

MTM

k

mdest

MTE

k

mdest

MTS

k

mdest

MTM

k

mdest

MTE

k

Figure

Impact

of

startup

time

on

multicast

latency

with

unicast-based

and

multidestination-based

schemes

Three

different

values

of

startup

ts

and

microsec

and

two

different

switch

sizes

of

and

are

considered

for

a

node

system

the

MTE

algorithm

for

all

multicast

destination

sets

Since

the

MTM

algorithm

is

also

much

simpler

than

the

MTE

algorithm

we

conclude

that

the

MTM

algorithm

is

the

best

of

three

algorithms

50 100 150 200

Multicast

Latency

-LRB-

microsec

-RRB-

Number

of

destinations

ts

microsec

L

flits

n

k

C-Min

mdest

MTS

mdest

MTM

mdest

MTE

50 100 150 200

Multicast

Latency

-LRB-

microsec

-RRB-

Number

of

destinations

ts

microsec

L

flits

n

k

C-Min

mdest

MTS

mdest

MTM

mdest

MTE

50 100 150 200

Multicast

Latency

-LRB-

microsec

-RRB-

Number

of

destinations

ts

microsec

L

flits

n

k

C-Min

mdest

MTS

mdest

MTM

mdest

MTE

Figure

Impact

of

increase

in

system

size

on

multicast

latency

with

unicast-based

and

multidestination-based

schemes

Three

different

values

of

startup

ts

and

microsec

are

considered

for

a

node

system

with

switches

Impact

of

Message

Length

Since

we

concluded

that

the

MTM

algorithm

was

the

best

of

the

three

algorithms

we

considered

the

MTM

algorithm

alone

for

the

experiments

in

this

section

Figure

shows

the

effect

of

increase

in

message

length

on

the

overall

multicast/broadcast

latency

Three

different

message

sizes

were

considered

Here

also

an

overall

reduction

in

multicast

latency

up

to

a

factor

of

can

be

observed

With

increase

in

message

size

multicast

latency

increases

under

the

unicastbased

scheme

However

the

effect

is

minimal

under

the

multidestination

based

scheme

This

is

because

multidestination

scheme

is

capable

of

implementing

a

given

multicast

in

fewer

phases

than

the

unicast

based

scheme

Simulation

Results

for

Multicast

on

Bidirectional

MINs

We

conducted

experiments

similar

to

the

above

on

bidirectional

MINs

We

assumed

the

same

values

for

the

link

propagation

time

switch

routing

time

for

the

header

flit

and

switch

propagation

time

for

all

flits

as

with

unidirectional

MINs

Experiments

were

conducted

for

node

bidi-butterfly

systems

made

of

bidi

and

switches

-LRB-

unidirectional

and

switches

-RRB-

and

for

node

systems

made

of

bidi

switches

Since

we

have

already

seen

that

the

MTM

algorithm

is

the

best

of

the

three

algorithms

we

only

used

multiport

worms

generated

by

the

MTM

algorithm

for

our

experiments

For

unicast

based

multicast

we

used

the

U-Min

algorithm

proposed

for

bidirectional

MIN

networks

in

-LSB-

-RSB-

Note

that

the

C-Min

algorithm

was

proposed

for

unidirectional

cube

networks

and

is

therefore

not

considered

Multicast

Latency

-LRB-

microsec

-RRB-

Number

of

destinations

ts

microsec

n

k

C-Min

L

flits

C-Min

L

flits

C-Min

L

flits

mdest

MTM

L

flits

mdest

MTM

L

flits

mdest

MTM

L

flits

Multicast

Latency

-LRB-

microsec

-RRB-

Number

of

destinations

ts

microsec

n

k

C-Min

L

flits

C-Min

L

flits

C-Min

L

flits

mdest

MTM

L

flits

mdest

MTM

L

flits

mdest

MTM

L

flits

Figure

Impact

of

message

length

on

multicast

latency

with

unicast-based

and

multidestination-based

schemes

Three

different

values

of

message

length

L

and

flits

are

considered

for

node

systems

with

and

switches

in

this

section

In

our

experiments

unicast

worms

used

the

least

common

ancestor

stage

of

the

source

and

destination

to

turn

back

For

simplicity

we

assumed

that

the

multiport

encoded

multidestination

worms

travel

all

the

way

up

to

the

last

stage

of

the

MIN

before

turning

back

The

multidestination

worm

replicates

only

along

the

reverse

network

of

the

bidirectional

MIN

as

described

in

Sec.

Both

unicast

and

multidestination

worms

were

adaptively

routed

in

the

forward

network

till

the

stage

where

they

turned

around

Since

a

bidirectional

k

k

switch

is

actually

a

unidirectional

switch

we

assumed

chunk

sizes

of

bytes

and

bytes

for

and

bidi-switches

As

shown

in

Fig.

the

multicast

latency

decreases

as

the

destination

set

size

increases

for

multicasts

in

node

bidi-MIN

systems

using

the

multiport

encoded

worms

generated

by

the

MTM

algorithm

The

gain

in

performance

due

to

the

use

of

multiport

worm

based

multicast

is

upto

a

factor

of

Furthermore

it

can

again

be

observed

that

larger

switch

sizes

result

in

better

performances

because

destinations

can

be

grouped

better

in

systems

with

larger

switches

Such

benefits

do

not

arise

for

the

unicast

based

scheme

Figure

shows

that

performance

gains

similar

to

those

for

unidirectional

MINs

can

be

obtained

with

increase

in

system

size

A

system-wide

broadcast

using

multiport

encoded

multidestination

worms

-LRB-

to

destinations

-RRB-

in

a

node

system

performs

better

than

the

corresponding

unicast

based

multicast

using

the

U-Min

algorithm

This

shows

that

the

gain

in

multicast

performance

scales

with

increase

in

system

size

as

was

observed

with

unidirectional

MINs

Finally

it

can

again

be

observed

-LRB-

as

shown

in

Fig.

-RRB-

that

with

increase

in

message

length

the

gain

in

performance

of

multidestination

worm

based

multicast

increases

This

is

due

to

the

fact

that

multicast

using

multiport

encoded

multidestination

worms

requires

fewer

communication

phases

than

the

corresponding

unicast-based

multicast

Conclusion

In

this

paper

we

have

proposed

a

new

approach

to

implement

fast

broadcast

and

multicast

in

multistage

interconnection

networks

The

main

contribution

of

this

paper

has

been

to

use

multiport

encoded

multidestination

worms

instead

of

unicast

messages

We

have

shown

how

such

a

mechanism

can

be

implemented

with

few

modifications

to

existing

switch

architectures

Theoretical

properties

associated

with

multiport

trees

corresponding

to

multiport

encoded

multidestination

worms

have

been

analyzed

Three

algorithms

have

been

proposed

to

generate

multiport

worms

for

a

given

multicast

destination

set

A

simple

multinomial

tree

based

approach

has

been

proposed

for

implementing

multiport

worm

based

multicast

using

very

few

communication

phases

The

proposed

schemes

were

evaluated

through

simulation

for

a

range

of

system

sizes

switch

sizes

message

lengths

and

communication

startup

times

in

unidirectional

and

bidirectional

MIN

systems

The

simulation

results

show

that

multicasting

using

multiport

worms

can

reduce

multicast

and

broadcast

latencies

by

factors

of

up

to

in

parallel

systems

using

both

unidirectional

and

bidirectional

multistage

interconnects

These

results

show

that

using

multiport

worms

to

perform

efficient

collective

communications

in

parallel

Multicast

Latency

-LRB-

microsec

-RRB-

Number

of

destinations

ts

microsec

L

flits

n

U-Min

k

U-Min

k

MTM

k

MTM

k

Multicast

Latency

-LRB-

microsec

-RRB-

Number

of

destinations

ts

microsec

L

flits

n

U-Min

k

U-Min

k

MTM

k

MTM

k

Multicast

Latency

-LRB-

microsec

-RRB-

Number

of

destinations

ts

microsec

L

flits

n

U-Min

k

U-Min

k

MTM

k

MTM

k

Figure

Impact

of

startup

time

on

multicast

latency

with

unicast-based

and

multidestination-based

schemes

on

bidirectional

MINs

Three

different

values

of

startup

ts

and

microsec

are

considered

for

node

bidi-MIN

systems

with

bidi

and

switches

50 100 150 200

Multicast

Latency

-LRB-

microsec

-RRB-

Number

of

destinations

ts

microsec

L

flits

k

n

U-Min

MTM

50 100 150 200

Multicast

Latency

-LRB-

microsec

-RRB-

Number

of

destinations

ts

microsec

L

flits

k

n

U-Min

MTM

Figure

Impact

of

increase

in

system

size

on

multicast

latency

with

unicast-based

and

multidestination-based

schemes

on

bidirectional

MINs

Two

different

values

of

startup

ts

and

microsec

are

considered

for

a

node

bidi-MIN

system

with

bidi

switches

Multicast

Latency

-LRB-

microsec

-RRB-

Number

of

destinations

ts

microsec

k

n

U-Min

L

flits

U-Min

L

flits

U-Min

L

flits

MTM

L

flits

MTM

L

flits

MTM

L

flits

Multicast

Latency

-LRB-

microsec

-RRB-

Number

of

destinations

ts

microsec

k

n

U-Min

L

flits

U-Min

L

flits

U-Min

L

flits

MTM

L

flits

MTM

L

flits

MTM

L

flits

Figure

Impact

of

message

length

on

multicast

latency

with

unicast-based

and

multidestination-based

schemes

on

bidirectional

MINs

Three

different

values

of

message

length

L

and

flits

are

considered

for

node

bidi-MIN

systems

with

bidi

and

switches

systems

is

extremely

viable

We

are

currently

investigating

various

methods

for

assigning

the

generated

multiport

worms

to

multiple

phases

so

as

to

reduce

contention

and

lower

multicast/broadcast

latencies

even

further

We

are

also

studying

mechanisms

and

algorithms

for

implementing

other

collective

communication

operations

efficiently

on

multistage

interconnection

networks

using

multiport

worms

Finally

we

are

also

currently

investigating

alternative

switch

architectures

for

implementing

multidestination

worms

Acknowledgments

Our

sincere

thanks

to

Professor

Jose

Duato

for

informing

us

about

the

result

discussed

in

Lemma

Additional

Information

A

number

of

related

papers

and

technical

reports

are

available

electronically

through

the

home

page

of

Parallel

Architecture

and

Communication

-LRB-

PAC

-RRB-

research

group

The

URL

is

Alternatively

the

papers

can

be

obtained

by

anonymous

ftp

from

ftp.cis.ohio-state

edu

cd

pub/communication

References

-LSB-

-RSB-

Y.

Aydogan

C.

B.

Stunkel

C.

Aykanat

and

B.

Abali

Adaptive

Source

Routing

in

Multistage

Interconnection

Networks

In

Proceedings

of

the

International

Parallel

Processing

Symposium

pages

-LSB-

-RSB-

V.

Bala

J.

Bruck

R.

Cypher

P.

Elustondo

A.

Ho

C.-T

Ho

S.

Kipnis

and

M.

Snir

CCL

A

Portable

and

Tunable

Collective

Communication

Library

for

Scalable

Parallel

Computers

In

Proceedings

of

the

International

Parallel

Processing

Symposium

-LSB-

-RSB-

M.

Barnett

D.

G.

Payne

and

R.

Van

de

Geijn

Optimal

Broadcasting

in

Mesh-Connected

Architectures

Technical

Report

Dept.

of

Computer

Science

Universityof

Texas

at

Austin

Dec

-LSB-

-RSB-

J.

Beecroft

M.

Homewood

and

M.

McLaren

Meiko

Interconnect

Elan-Elite

Design

Parallel

Computing

Nov

-LSB-

-RSB-

E.

D.

Brooks

and

J.

E.

Hoag

A

Scalable

Coherent

Cache

System

with

Incomplete

Directory

State

In

International

Conference

on

Parallel

Processing

-LRB-

ICPP

-RRB-

volume

pages

-LSB-

-RSB-

J.

Bruck

R.

Cypher

and

C.-T

Ho

Multiple

Message

Broadcasting

with

Generalized

Fibonacci

Trees

In

Symposium

on

Parallel

and

Distributed

Processing

pages

-LSB-

-RSB-

C.-M

Chiang

and

L.

M.

Ni

Deadlock-Free

Multi-Head

Wormhole

Routing

In

Proceedings

of

the

First

High

Performance

Computing-Asia

-LSB-

-RSB-

C.-M

Chiang

and

L.

M.

Ni

Efficient

Software

Multicast

in

Wormhole-routed

Unidirectional

Multistage

Networks

In

Symposium

on

Parallel

and

Distributed

Processing

-LSB-

-RSB-

Cray

Research

Inc.

Cray

System

Architecture

Overview

-LSB-

-RSB-

W.

J.

Dally

and

C.

L.

Seitz

Deadlock-Free

Message

Routing

in

Multiprocessor

Interconnection

Networks

IEEE

Transactions

on

Computers

pages

May

-LSB-

-RSB-

J.

Duato

A

Theory

of

Deadlock-Free

Adaptive

Multicast

Routing

in

Wormhole

Networks

IEEE

Transactions

on

Parallel

and

Distributed

Systems

pages

September

-LSB-

-RSB-

B.

Duzett

and

R.

Buck

An

Overview

of

the

Supercomputer

In

Proceedings

of

the

Frontiers

of

Massively

Parallel

Computation

pages

-LSB-

-RSB-

J.

L.

Hennessy

and

D.

Patterson

Computer

Organization

and

Design

The

Hardware/Software

Interface

Morgan

Kaufmann

-LSB-

-RSB-

Intel

Corporation

Paragon

XP/S

Product

Overview

-LSB-

-RSB-

S.

L.

Johnsson

and

C.-T

Ho

Optimum

Broadcasting

and

Personalized

Communication

in

Hypercubes

IEEE

Transactions

on

Computers

pages

September

-LSB-

-RSB-

P.

Kermani

and

L.

Kleinrock

Virtual

Cut-Through

A

new

Computer

Communications

Switching

Technique

Computer

Networks

-LRB-

-RRB-

Sept.

-LSB-

-RSB-

S.

Konstantinidou

and

L.

Snyder

Chaos

Router

Architecture

and

Performance

In

Proceedings

of

the

International

Symposium

on

Architecture

pages

-LSB-

-RSB-

C.

E.

Leiserson

et

al.

The

Network

Architecture

of

the

Connection

Machine

In

Proceedings

of

the

ACM

Symposium

on

Parallel

Algorithms

and

Architectures

pages

-LSB-

-RSB-

X.

Lin

P.

K.

McKinley

and

L.

M.

Ni

Performance

Evaluation

of

Multicast

Wormhole

Routing

in

Multicomputers

In

Proceedings

of

the

International

Conference

on

Parallel

Processing

pages

I

-LSB-

-RSB-

X.

Lin

and

L.

M.

Ni

Deadlock-free

Multicast

Wormhole

Routing

in

Multicomputer

Networks

In

Proceedings

of

the

International

Symposium

on

Computer

Architecture

pages

-LSB-

-RSB-

P.

K.

McKinley

and

D.

F.

Robinson

Collective

Communication

in

Wormhole-Routed

Massively

Parallel

Computers

IEEE

Computer

pages

Dec

-LSB-

-RSB-

P.

K.

McKinley

H.

Xu

A.-H

Esfahanian

and

L.

M.

Ni

Unicast-based

Multicast

Communication

in

Wormholerouted

Networks

IEEE

Transactions

on

Parallel

and

Distributed

Systems

-LRB-

-RRB-

Dec

-LSB-

-RSB-

Message

Passing

Interface

Forum

MPI

A

Message-Passing

Interface

Standard

Mar

-LSB-

-RSB-

P.

Mohapatra

and

C.

R.

Das

A

Queuing

Model

for

Finite-Buffered

Multistage

Interconnection

Networks

In

International

Conference

on

Parallel

Processing

pages

I

Aug

-LSB-

-RSB-

L.

Ni

and

P.

K.

McKinley

A

Survey

of

Wormhole

Routing

Techniques

in

Direct

Networks

IEEE

Computer

pages

Feb.

-LSB-

-RSB-

D.

K.

Panda

Issues

in

Designing

Efficient

and

Practical

Algorithms

for

Collective

Communication

in

WormholeRouted

Systems

In

Workshop

on

Challenges

for

Parallel

Processing

pages

-LSB-

-RSB-

D.

K.

Panda

S.

Singal

and

P.

Prabhakaran

Multidestination

Message

Passing

Mechanism

Conforming

to

Base

Wormhole

Routing

Scheme

In

Proceedings

of

the

Parallel

Computer

Routing

and

Communication

Workshop

pages

-LSB-

-RSB-

D.

K.

Panda

and

R.

Sivaram

Fast

Broadcast

and

Multicast

in

Wormhole

Multistage

Networks

with

Multidestination

Worms

Technical

Report

Dept.

of

Computer

and

Information

Science

The

Ohio

State

University

April

-LSB-

-RSB-

I.

D.

Scherson

and

C.-H

Chien

Least

common

ancestor

networks

In

Proc

Int

Parallel

Processing

Symp.

pages

-LSB-

-RSB-

H.

J.

Siegel

and

R.

McMillen

The

Multistage

Cube

a

Versatile

Interconnection

Network

IEEE

Computer

-LRB-

-RRB-

December

-LSB-

-RSB-

H.

J.

Siegel

W.

G.

Nation

C.

P.

Kruskal

and

L.

M.

Napolitano

Using

the

Multistage

Cube

Network

Topology

in

Parallel

Supercomputers

Proceedings

of

the

IEEE

-LRB-

-RRB-

December

-LSB-

-RSB-

C.

B.

Stunkel

D.

Shea

D.

G.

Grice

P.

H.

Hochschild

and

M.

Tsao

The

High

Performance

Switch

In

Scalable

High

Performance

Computing

Conference

pages

-LSB-

-RSB-

C.

B.

Stunkel

D.

G.

Shea

B.

Abali

M.

M.

Denneau

P.

H.

Hochschild

D.

J.

Joseph

B.

J.

Nathanson

M.

Tsao

and

P.

R.

Varker

Architecture

and

Implementation

of

Vulcan

In

Proceedings

of

the

International

Parallel

Processing

Symposium

pages

-LSB-

-RSB-

C.

B.

Stunkel

D.

G.

Shea

B.

Abali

et

al.

The

High-Performance

Switch

IBM

System

Journal

-LRB-

-RRB-

-LSB-

-RSB-

H.

Xu

Y.-D

Gui

and

L.

M.

Ni

Optimal

Software

Multicast

in

Wormhole-Routed

Multistage

Networks

In

Proceedings

of

the

Supercomputing

Conference

pages

Appendix

A

The

Sub-procedures

of

the

MTE

Algorithm

Procedure

BuildModifiedKarnaughMap

Input

A

set

of

destinations

D

Output

A

modified

Karnaugh

map

with

's

at

positions

corresponding

to

destinations

and

's

at

all

other

places

BuildModifiedKarnaughMap

-LRB-

-RRB-

f

for

-LRB-

i

i

N

i

-RRB-

f

KarnaughMap

-LSB-

i

-RSB-

g

for

-LRB-

i

i

j

D

j

i

-RRB-

f

KarnaughMap

-LSB-

CalculateIndex

-LRB-

D

-LSB-

i

-RSB-

-RRB-

-RSB-

g

return

-LRB-

KarnaughMap

-RRB-

g

Figure

The

algorithm

for

building

the

modified

Karnaugh

representation

Procedure

BuildGrid

Input

The

modified

Karnaugh

map

representation

of

the

destinations

and

a

chosen

dimension

Output

A

grid

corresponding

to

the

chosen

dimension

BuildGrid

-LRB-

-RRB-

f

GridSide

N

k

for

-LRB-

i

i

GridSide

i

-RRB-

f

for

-LRB-

j

i

j

GridSide

j

-RRB-

f

Find

k

entries

corresponding

to

i

and

j

in

Karnaugh

Map

-LRB-

call

these

KBitsi

and

KBitsj

-RRB-

Res

KBitsi

KBitsj

NumberOfOnes

FindNumberOfOnes

-LRB-

Res

-RRB-

if

-LRB-

i

j

-RRB-

f

Grid

-LSB-

i

-RSB-

-LSB-

j

-RSB-

NumberOfOnes

g

else

f

Grid

-LSB-

i

-RSB-

-LSB-

j

-RSB-

NumberOfOnes

g

g

g

return

-LRB-

Grid

-RRB-

g

Figure

The

algorithm

for

building

the

grid

Procedure

FindMaxima

Input

A

grid

corresponding

to

the

chosen

dimension

Output

The

maximum

value

the

positions

of

the

maxima

and

the

number

of

maxima

in

the

grid

FindMaxima

-LRB-

-RRB-

f

maximum

P

ositionList

OE

NumberOfMaxima

GridSide

N

k

for

-LRB-

i

i

GridSide

i

-RRB-

f

for

-LRB-

j

i

j

GridSide

j

-RRB-

f

if

-LRB-

Grid

-LSB-

i

-RSB-

-LSB-

j

-RSB-

maximum

-RRB-

f

maximum

Grid

-LSB-

i

-RSB-

-LSB-

j

-RSB-

P

ositionList

OE

NumberOfMaxima

g

else

if

-LRB-

Grid

-LSB-

i

-RSB-

-LSB-

j

-RSB-

maximum

-RRB-

f

NumberOfMaxima

If

not

already

present

add

i

and

j

to

P

ositionList

g

g

g

return

-LRB-

maximum

P

ositionList

NumberOfMaxima

-RRB-

g

Figure

The

algorithm

for

finding

the

maxima

in

a

grid

Procedure

FindEnhancedGroup

Input

The

maximum

value

the

positions

of

the

maxima

and

the

number

of

maxima

in

the

grid

Output

The

best

enhanced

group

corresponding

to

the

grid

FindEnhancedGroup

-LRB-

-RRB-

f

MaxCover

For

all

elements

in

P

ositionList

f

Seperate

the

digits

of

the

element

Add

the

ith

digit

-LRB-

i

n

-RRB-

to

Seti

g

For

every

Seti

form

the

corresponding

power

set

PSi

do

f

Form

CandidateEnhancedGroup

by

choosing

a

set

CandidateSeti

from

every

PSi

Form

all

numbers

such

that

the

ith

-LRB-

i

n

-RRB-

digit

belongs

to

CandidateSeti

Call

the

set

of

these

numbers

P

digits

For

every

element

of

P

digits

find

the

corresponding

set

of

k

entries

in

the

Karnaugh

Map

Find

the

logical

AND

of

all

these

k-entry

sets

Call

the

result

AndRes

Find

the

positions

in

AndRes

with

's

Call

the

set

of

these

positions

AndP

os

Cover

FindNumberOfOnes

-LRB-

AndRes

-RRB-

j

P

digits

j

if

-LRB-

Cover

MaxCover

-RRB-

f

Store

CandidateEnhancedGroup

and

AndP

os

as

EnhancedGroup

MaxCover

Cover

g

g

while

-LRB-

CandidateEnhancedGroupsStillExist

-RRB-

return

-LRB-

EnhancedGroup

-RRB-

g

Figure

The

algorithm

for

finding

the

enhanced

group

in

a

grid

broadcast
collective communication
multicast
wormhole routing

interprocessor communication
multistage interconnection network
parallel computer architecture
virtual cut-through