Scalable

Parallel

Data

Mining

for

Association

Rules

Eui-Hong

-LRB-

Sam

-RRB-

Han

George

Karypis

and

Vipin

Kumar

Department

of

Computer

Science

EECS

Bldg.

Union

St.

SE

University

of

Minnesota

Minneapolis

MN

USA

December

Abstract

One

of

the

important

problems

in

data

mining

is

discovering

association

rules

The

association

rule

discovery

techniques

are

generally

applied

to

databases

of

transactions

where

each

transaction

consists

of

a

set

of

items

Since

usually

such

transaction-based

databases

contain

extremely

large

amounts

of

data

and

large

number

of

distinct

items

current

association

rule

discovery

techniques

try

to

prune

the

search

space

according

to

the

user

specified

minimum

support

for

items

under

consideration

Even

with

the

pruning

the

task

of

finding

all

association

rules

requires

a

lot

of

computation

power

and

time

Widely

available

parallel

computers

offer

solutions

to

the

computation

requirement

of

the

task

on

hand

Efficient

and

scalable

parallel

algorithms

are

necessary

to

take

full

advantage

of

the

parallel

computers

In

this

paper

we

present

two

new

parallel

algorithms

for

mining

association

rules

The

Intelligent

Data

Distribution

algorithm

exploits

aggregate

memory

of

the

parallel

computers

and

employs

intelligent

candidate

partitioning

scheme

and

uses

efficient

communication

mechanism

to

move

data

among

the

processors

The

Hybrid

Distribution

algorithm

dynamically

partitions

the

candidate

set

and

utilizes

the

main

memory

efficiently

while

maintaining

good

load

balance

The

experimental

results

on

a

Cray

parallel

computer

show

that

the

Hybrid

Distribution

algorithm

scales

linearly

and

exploits

the

aggregate

memory

better

and

can

generate

more

association

rules

with

a

single

scan

of

database

per

pass

This

work

was

supported

by

NSF

grant

Army

Research

Office

contract

and

Cray

Research

Inc.

Fellowship

the

content

of

which

does

not

necessarily

reflect

the

policy

of

the

government

and

no

official

endorsement

should

be

inferred

Access

to

computing

facilities

was

provided

by

AHPCRC

Minnesota

Supercomputer

Institute

Cray

Research

Inc.

and

NSF

grant

Introduction

One

of

the

important

problems

in

data

mining

-LSB-

SAD

-RSB-

is

discovering

association

rules

These

association

can

be

easily

determined

by

computing

the

frequencies

of

the

occurrence

of

different

candidate

patterns

-LRB-

i.e.

subsets

of

items

-RRB-

in

the

database

of

transactions

Since

usually

such

transaction-based

databases

contain

extremely

large

amounts

of

data

and

large

number

of

distinct

items

the

total

number

of

candidate

sets

is

prohibitively

large

Hence

current

association

rule

discovery

techniques

-LSB-

-RSB-

try

to

prune

the

search

space

by

requiring

a

minimum

level

of

support

for

candidates

under

consideration

Support

is

a

measure

based

on

the

number

of

occurrences

of

the

candidates

in

database

transactions

Even

with

the

pruning

the

task

of

finding

all

association

rules

requires

a

lot

of

computation

power

that

is

available

only

in

parallel

computers

Efficient

and

scalable

parallel

algorithms

for

association

rule

mining

are

necessary

to

take

full

advantage

of

the

parallel

computers

Two

parallel

algorithms

based

on

Count

Distribution

and

Data

Distribution

were

proposed

in

-LSB-

-RSB-

The

Count

Distribution

algorithm

has

shown

to

scale

linearly

and

have

excellent

speedup

and

sizeup

behavior

-LSB-

-RSB-

However

this

algorithm

works

well

only

when

all

the

candidates

in

each

pass

of

the

algorithm

fit

into

the

main

memory

of

single

processor

of

the

parallel

computers

If

the

number

of

candidates

are

too

large

to

fit

into

the

main

memory

then

at

each

processor

this

algorithm

has

to

process

each

database

transaction

multiple

times

at

each

processor

The

Data

Distribution

algorithm

addresses

the

memory

problem

of

the

Count

Distribution

algorithm

by

partitioning

the

candidate

set

and

assigning

a

partition

to

each

processor

in

the

system

However

this

algorithm

results

in

high

communication

overhead

due

to

data

movement

and

extra

computation

per

node

due

to

the

fact

that

each

node

has

to

process

every

single

data

base

transaction

-LSB-

-RSB-

We

present

in

this

paper

two

parallel

algorithms

for

mining

association

rules

We

first

present

Intelligent

Data

Distribution

algorithm

that

improves

upon

the

Data

Distribution

algorithm

such

that

the

communication

overhead

and

extra

computation

is

minimized

The

Hybrid

Distribution

algorithm

is

a

hybrid

of

the

Count

Distribution

and

the

Intelligent

Data

Distribution

such

that

it

combines

good

features

of

the

both

algorithms

The

experimental

results

of

these

algorithms

on

a

Cray

parallel

computer

show

that

the

Hybrid

Distribution

algorithm

algorithm

scales

linearly

and

has

excellent

sizeup

behavior

and

yet

exploits

the

aggregate

main

memory

of

the

multiple

processors

The

rest

of

this

paper

is

organized

as

follows

Section

provides

an

overview

of

mining

association

rules

and

serial

algorithms

will

be

presented

in

Section

The

proposed

algorithms

are

described

in

Section

The

scalability

analysis

of

the

algorithms

will

be

presented

in

Section

Experimental

results

are

shown

in

Section

Section

contains

conclusions

Basic

Concepts

The

association

rule

discovery

is

composed

of

two

steps

The

first

step

is

to

discover

all

the

frequent

item

sets

that

have

support

greater

than

the

user

specified

minimum

support

The

second

step

is

to

generate

association

rules

from

the

frequent

sets

discovered

in

the

first

step

The

amount

of

time

required

to

generate

this

association

rules

is

insignificant

compared

to

f

frequent

for

-LRB-

k

Fk

OE

k

-RRB-

do

begin

Ck

apriori

gen

-LRB-

Fk

-RRB-

New

Candidates

are

generated

subset

-LRB-

Ck

-RRB-

Fk

fc

Ck

j

c.count

minsupg

end

Answer

S

Fk

Figure

Apriori

Algorithm

the

amount

of

time

required

to

find

the

frequent

sets

For

this

reason

parallel

association

algorithms

focus

on

how

to

parallelize

the

first

step

Apriori

algorithm

presented

in

-LSB-

-RSB-

has

been

shown

to

perform

well

compared

to

other

sequential

algorithms

-LSB-

-RSB-

and

is

the

base

for

the

parallel

algorithms

proposed

in

-LSB-

-RSB-

We

have

also

chosen

to

base

our

parallel

algorithms

on

the

Apriori

algorithm

In

the

rest

of

this

section

we

briefly

describe

the

Apriori

algorithm

The

reader

should

refer

to

-LSB-

-RSB-

for

further

details

The

high

-LCB-

level

structures

of

the

Apriori

algorithm

is

shown

in

Figure

The

first

step

of

the

algorithm

is

to

construct

frequent

sets

of

items

of

length

one

-LRB-

i.e.

consisting

of

a

single

item

-RRB-

by

scanning

the

database

During

k-th

pass

of

the

loop

the

apriori

gen

function

generates

Ck

the

set

of

candidates

of

size

k

from

Fk

set

of

frequent

-LRB-

k

-RRB-

itemsets

The

subset

function

scans

the

database

to

gather

support

count

of

the

candidates

in

Ck

Then

the

candidates

in

Ck

with

minimum

support

become

Fk

Most

of

the

computation

work

in

the

main

loop

of

the

Apriori

algorithm

is

in

the

subset

function

Candidate

itemsets

are

stored

in

a

hash

tree

-LSB-

-RSB-

The

interior

node

of

the

hash

tree

contains

a

hash

table

and

the

leaf

node

contains

list

of

candidates

The

candidates

are

inserted

in

to

the

hash

tree

by

hashing

their

items

in

sequence

and

following

pointers

of

the

hash

tables

in

the

internal

nodes

The

subset

function

reads

one

transaction

from

the

buffer

and

traverses

the

hash

tree

from

the

root

with

every

item

in

the

transaction

as

a

possible

starting

item

of

the

candidates

In

the

next

level

of

the

tree

all

the

items

of

the

transaction

following

the

starting

item

are

hashed

Once

a

leaf

is

reached

all

the

candidates

at

the

leaf

are

checked

against

the

transaction

and

their

counts

are

updated

accordingly

For

more

details

on

the

algorithm

and

the

hash

tree

data

structure

and

subset

function

refer

to

-LSB-

-RSB-

Parallel

Algorithms

In

this

section

we

will

focus

on

the

parallelization

of

the

first

task

that

finds

all

frequent

itemsets

The

parallel

implementation

of

the

second

step

is

straightforward

and

is

discussed

in

-LSB-

-RSB-

We

first

discuss

two

parallel

algorithms

proposed

in

-LSB-

-RSB-

to

help

motivate

our

Proc

Proc

Proc

Proc

Candidate

Hash

Tree

N/P

Data

-LCB-

B

E

-RCB-

-LCB-

A

D

-RCB-

M

Count

N/P

Data

-LCB-

A

C

-RCB-

-LCB-

B

C

-RCB-

-LCB-

C

D

-RCB-

M

Count

N/P

Data

-LCB-

A

B

-RCB-

-LCB-

B

E

-RCB-

-LCB-

A

D

-RCB-

-LCB-

C

D

-RCB-

M

Count

-LCB-

A

B

-RCB-

-LCB-

A

C

-RCB-

-LCB-

B

C

-RCB-

-LCB-

C

D

-RCB-

-LCB-

A

B

-RCB-

-LCB-

A

D

-RCB-

-LCB-

B

E

-RCB-

-LCB-

D

E

-RCB-

-LCB-

D

E

-RCB-

-LCB-

A

C

-RCB-

-LCB-

B

C

-RCB-

-LCB-

D

E

-RCB-

N/P

Data

-LCB-

A

B

-RCB-

-LCB-

A

C

-RCB-

-LCB-

B

C

-RCB-

-LCB-

B

E

-RCB-

-LCB-

A

D

-RCB-

-LCB-

D

E

-RCB-

-LCB-

C

D

-RCB-

M

Count

Candidate

Hash

Tree

Candidate

Hash

Tree

Candidate

Hash

Tree

N

number

of

data

items

M

size

of

candidate

set

P

number

of

processors

Global

Reduction

Figure

Count

Distribution

-LRB-

CD

-RRB-

Algorithm

algorithm

formulations

In

all

our

discussions

we

assume

that

data

is

evenly

distributed

to

each

processor

Count

Distribution

Algorithm

In

the

Count

Distribution

-LRB-

CD

-RRB-

algorithm

proposed

in

-LSB-

-RSB-

each

processor

computes

how

many

times

all

the

candidates

appear

in

the

locally

stored

portion

of

the

database

transactions

This

is

done

by

building

the

entire

hash

tree

that

corresponds

to

all

the

candidates

and

then

performing

a

single

pass

over

the

locally

stored

portion

of

the

database

transactions

to

collect

the

counts

Then

all

the

processors

participate

in

a

global

reduction

operation

-LSB-

-RSB-

to

get

the

global

counts

of

the

candidates

This

algorithm

is

illustrated

in

Figure

Note

that

since

each

processor

needs

to

build

a

hash

tree

for

all

the

candidates

these

hash

trees

are

identical

at

each

processor

This

algorithm

has

been

shown

to

scale

linearly

and

have

excellent

speedup

and

sizeup

behavior

-LSB-

-RSB-

This

is

because

each

processor

can

compute

the

counts

independently

of

the

other

processors

and

needs

to

communicate

with

the

other

processors

only

once

at

the

end

of

the

computation

step

However

this

algorithm

works

well

only

when

the

hash

trees

can

fit

into

the

main

memory

of

each

processor

If

the

number

of

candidates

is

large

then

the

hash

tree

does

not

fit

into

the

main

memory

In

this

case

this

algorithm

has

to

partition

the

hash

tree

and

compute

the

counts

by

scanning

the

database

per

partition

Note

that

the

number

of

candidates

increases

as

either

the

number

of

distinct

items

in

the

database

increases

or

as

the

minimum

support

level

of

the

association

rules

decreases

Thus

the

CD

algorithm

is

effective

for

small

number

of

distinct

items

and

a

high

minimum

support

level

Data

Distribution

Algorithm

The

Data

Distribution

-LRB-

DD

-RRB-

algorithm

-LSB-

-RSB-

addresses

the

memory

problem

of

the

CD

algorithm

by

partitioning

the

candidate

set

among

the

processors

This

partitioning

is

done

in

a

round

robin

fashion

Now

each

processor

is

responsible

for

computing

the

frequencies

of

its

locally

stored

subset

of

the

candidate

set

In

order

to

do

that

each

processor

needs

to

scan

its

locally

stored

portion

of

database

transactions

as

well

as

the

portions

of

the

database

transactions

assigned

to

the

other

processors

In

the

DD

algorithm

this

is

done

by

having

each

processor

receive

the

portions

of

the

transactions

stored

in

the

other

processors

according

to

the

following

fashion

Each

processor

Pi

allocates

P

buffers

-LRB-

each

one

page

long

and

for

each

processor

-RRB-

The

ith

buffer

is

used

to

store

transactions

from

the

locally

stored

database

and

the

remaining

buffers

are

used

to

store

transactions

from

the

other

processors

such

that

buffer

j

will

store

transactions

from

processor

Pj

Now

each

processor

checks

the

P

buffers

to

see

which

one

contains

data

Let

k

be

this

buffer

Each

processor

processes

the

transactions

in

this

buffer

and

updates

the

frequencies

of

its

own

candidate

subset

based

on

these

transactions

If

this

kth

buffer

corresponds

to

the

buffer

that

stores

local

transactions

it

is

sent

to

all

the

other

processors

and

a

new

page

is

read

from

the

local

database

If

the

kth

buffer

corresponds

to

a

buffer

that

stores

transactions

from

another

processor

then

it

is

cleared

and

a

receive

request

is

issued

This

continues

until

every

processor

has

processed

the

entire

database

transactions

After

the

collection

of

count

information

is

done

each

processor

finds

the

frequent

itemsets

from

the

candidate

set

and

these

frequent

sets

are

sent

to

every

other

processor

using

an

all

-LCB-

to

-LCB-

all

broadcast

operation

-LSB-

-RSB-

Figure

shows

the

high

level

operations

of

the

algorithm

Note

that

each

processor

has

different

set

of

candidates

in

the

candidate

hash

tree

This

algorithm

exploits

the

total

available

memory

better

by

partitioning

the

candidate

set

As

the

number

of

processors

increases

the

number

of

candidates

that

the

algorithm

can

handle

also

increases

However

as

reported

in

-LSB-

-RSB-

the

performance

of

this

algorithm

is

significantly

worse

than

the

CD

algorithm

The

response

time

of

this

algorithm

is

to

times

higher

than

that

of

the

CD

algorithm

-LSB-

-RSB-

The

problem

lies

with

communication

cost

of

moving

data

around

and

with

the

extra

work

that

is

performed

in

processing

all

the

transactions

The

communication

pattern

of

this

algorithm

causes

two

problems

First

during

each

pass

of

the

algorithm

each

processor

sends

to

all

the

other

processors

the

portion

of

the

database

that

resides

locally

In

particular

each

processor

reads

the

locally

stored

portion

of

the

database

one

page

at

a

time

and

sends

it

to

all

the

other

processors

by

issuing

P

send

operations

Similarly

each

processor

issues

a

receive

operation

from

each

other

processor

in

order

to

receive

these

pages

If

the

interconnection

network

of

the

underlying

parallel

computer

was

fully

connected

-LRB-

i.e.

there

is

a

direct

link

between

all

pairs

of

processors

and

each

processor

can

receive

data

on

all

incoming

links

simultaneously

-RRB-

then

this

communication

pattern

will

lead

to

a

very

good

performance

In

particular

if

O

-LRB-

NP

-RRB-

is

the

size

of

the

database

assigned

locally

to

each

processor

the

amount

of

time

spent

in

the

communication

will

be

O

-LRB-

N

-RRB-

However

on

all

realistic

parallel

computers

the

processors

are

connected

via

Proc

Proc

Proc

Proc

N/P

N/P

N/P

M/P

M/P

M/P

M/P

Local

Data

Remote

Data

Candidate

Hash

Tree

CountCount

Local

Data

Remote

Data

Candidate

Hash

Tree

CountCount

Local

Data

Remote

Data

Candidate

Hash

Tree

CountCount

Local

Data

Remote

Data

Candidate

Hash

Tree

CountCount

-LCB-

A

B

-RCB-

-LCB-

A

D

-RCB-

-LCB-

A

C

-RCB-

-LCB-

A

E

-RCB-

-LCB-

B

C

-RCB-

-LCB-

B

D

-RCB-

-LCB-

B

E

-RCB-

-LCB-

C

D

-RCB-

-LCB-

C

E

-RCB-

-LCB-

C

F

-RCB-

-LCB-

D

E

-RCB-

-LCB-

E

F

-RCB-

N/PData

N

number

of

data

items

M

size

of

candidate

set

P

number

of

processors

Data

Data

Broadcast

Broadcast

Broadcast

All-to-all

Broadcast

Figure

Data

Distribution

-LRB-

DD

-RRB-

Algorithm

a

sparser

networks

-LRB-

such

as

or

hypercube

-RRB-

and

a

processor

can

receive

data

from

-LRB-

or

send

data

to

-RRB-

only

one

other

processor

at

a

time

On

such

machines

this

communication

patterns

will

take

significantly

more

time

than

O

-LRB-

N

-RRB-

because

of

contention

Note

that

the

locally

stored

portions

of

the

database

can

be

sent

to

all

the

other

processors

by

using

the

all

-LCB-

to

-LCB-

all

broadcast

described

in

-LSB-

-RSB-

This

operation

does

not

suffer

from

the

contentions

problems

and

its

performance

is

O

-LRB-

N

-RRB-

Second

if

we

look

at

the

size

of

the

candidate

sets

as

a

function

of

the

number

of

passes

of

the

algorithm

we

see

that

in

the

first

few

passes

the

size

of

the

candidate

sets

increases

and

after

that

it

decreases

In

particular

during

the

last

several

passes

of

the

algorithm

there

are

only

a

small

number

of

items

in

the

candidate

sets

However

each

processor

in

the

DD

algorithm

still

sends

the

locally

stored

portions

of

the

database

to

all

the

other

processors

Thus

even

though

the

computation

decreases

the

amount

of

communication

remains

the

same

The

extra

work

is

introduced

due

to

the

fact

that

every

transaction

has

to

go

through

each

hash

tree

located

at

different

processors

The

amount

of

work

required

to

process

a

single

transaction

in

a

partitioned

hash

tree

is

not

proportionally

reduced

with

the

reduction

in

the

number

of

candidates

In

Figure

of

CD

only

NP

number

of

transactions

go

through

hash

tree

of

M

candidates

where

in

Figure

of

DD

N

number

of

transactions

have

to

go

through

hash

tree

of

MP

candidates

If

the

amount

of

work

required

for

each

transaction

to

be

checked

against

the

hash

tree

of

MP

number

of

candidates

is

of

that

of

hash

tree

of

M

number

of

candidates

there

is

no

extra

work

However

for

this

to

be

true

the

average

depth

of

hash

tree

has

to

be

reduced

to

and

the

average

number

of

candidates

in

the

leaf

nodes

has

to

be

reduced

to

in

this

algorithm

This

does

not

happen

in

the

hash

tree

scheme

proposed

in

-LSB-

-RSB-

To

see

this

consider

a

hash

tree

with

single

candidate

at

the

leaf

node

and

with

branching

factor

of

B.

By

reducing

the

number

of

candidates

to

of

M

will

reduce

the

depth

of

the

hash

tree

by

only

logB

P

With

B

P

-LRB-

which

would

be

the

most

likely

-RRB-

the

logB

P

You

may

change

the

assumption

such

that

the

number

of

candidates

at

the

leaf

node

is

greater

than

P

The

number

of

candidate

at

the

leaf

goes

down

to

but

the

depth

of

the

tree

does

not

change

In

most

of

real

cases

the

hash

tree

will

be

in

between

these

two

extreme

cases

However

the

amount

of

work

per

transactions

will

not

go

down

to

of

the

original

hash

tree

with

M

candidates

Intelligent

Data

Distribution

Algorithm

We

developed

the

Intelligent

Data

Distribution

-LRB-

IDD

-RRB-

algorithm

which

tackles

the

problem

of

communication

overhead

and

extra

work

in

the

DD

algorithm

We

changed

the

communication

pattern

of

moving

data

around

so

that

an

all

-LCB-

to

-LCB-

all

broadcast

is

performed

The

communication

overhead

of

this

approach

is

minimized

on

any

practical

parallel

architectures

that

can

embedded

in

a

ring

-LSB-

-RSB-

Figure

shows

the

pseudo

code

for

this

data

movements

In

our

algorithm

the

processors

form

a

logical

ring

and

each

processor

determines

its

right

and

left

neighbor

processors

Each

processor

has

one

send

buffer

-LRB-

SBuf

-RRB-

and

one

receive

buffer

-LRB-

RBuf

-RRB-

Initially

the

SBuf

is

filled

with

one

block

of

local

data

Then

each

processor

initiates

asynchronous

send

operation

to

the

right

neighbor

processor

with

SBuf

and

asynchronous

receive

operation

to

the

left

neighbor

processor

with

RBuf

While

these

asynchronous

operations

are

proceeding

each

processor

processes

the

transactions

in

SBuf

and

collects

the

counts

of

the

candidates

assigned

to

the

processor

After

this

operation

each

processor

waits

until

this

asynchronous

operations

have

completed

Then

the

roles

of

SBuf

and

RBuf

are

switched

and

the

above

operations

continue

for

P

times

Compared

to

DD

where

all

the

processors

send

data

to

all

other

processors

we

perform

only

a

point

to

point

communication

between

neighbors

thus

eliminating

any

communication

contention

Recall

that

for

DD

the

communication

overhead

of

data

movements

dominates

the

computation

work

in

the

later

passes

of

the

process

We

eliminate

this

problem

by

switching

to

the

CD

algorithm

in

the

later

passes

of

the

algorithm

When

the

total

number

of

candidates

falls

below

a

threshold

we

switch

to

CD

algorithm

Note

that

switching

to

the

CD

method

does

not

cause

any

communication

or

computation

overhead

since

each

processor

can

independently

determine

when

to

switch

provided

that

the

threshold

parameter

is

globally

known

The

good

choice

of

the

parameter

could

be

the

maximum

number

of

candidates

that

single

processor

can

have

in

the

main

memory

To

eliminate

the

extra

work

in

processing

the

whole

data

transactions

we

partition

candidates

based

on

the

first

item

of

the

candidates

Figure

shows

the

high

level

picture

of

the

algorithm

In

this

scheme

all

the

candidates

starting

with

the

same

item

belongs

to

one

processor

For

example

processor

in

Figure

has

all

the

candidates

starting

with

item

A

and

C

while

processor

has

all

the

candidates

starting

with

B

and

E.

Each

processor

keeps

the

first

items

of

the

candidates

it

has

in

a

bit

map

In

Apriori

algorithm

at

the

root

level

of

hash

tree

every

item

in

a

transaction

is

hashed

and

checked

against

the

hash

tree

In

our

algorithm

at

root

level

each

processor

checks

every

item

of

the

transaction

against

the

bit

while

-LRB-

done

-RRB-

f

FillBuffer

-LRB-

fd

SBuf

-RRB-

for

-LRB-

k

k

k

-RRB-

f

send/receive

data

in

non-blocking

pipeline

MPI

Irecv

-LRB-

RBuf

left

-RRB-

MPI

Isend

-LRB-

SBuf

right

-RRB-

process

transactions

in

SBuf

and

update

hash

tree

Subset

-LRB-

HTree

SBuf

-RRB-

MPI

Waitall

-LRB-

-RRB-

swap

two

buffers

tmp

SBuf

SBuf

RBuf

RBuf

tmp

g

process

transactions

in

SBuf

and

update

hash

tree

Subset

-LRB-

HTree

SBuf

-RRB-

g

Figure

Pseudo

Code

for

Data

Movements

map

to

see

if

the

processor

contains

candidates

starting

with

the

item

of

the

transaction

If

the

processor

does

not

contain

the

candidates

starting

with

the

item

the

processing

steps

involved

with

that

item

as

the

first

item

in

the

candidate

can

be

skipped

This

results

in

less

amount

of

transaction

data

going

through

the

hash

tree

and

thus

has

less

computation

For

example

let

-LRB-

A

B

C

D

E

F

G

H

-RRB-

be

a

transaction

that

processor

is

processing

in

the

subset

function

discussed

in

Section

at

the

top

level

of

the

function

processor

will

only

proceed

with

item

A

and

C

When

the

page

containing

this

transaction

is

shifted

to

processor

the

processor

will

only

process

items

starting

with

B

and

E.

Compare

this

approach

to

the

DD

algorithm

where

all

the

processors

have

to

process

all

the

items

of

the

transaction

at

the

root

level

of

the

hash

tree

For

instance

assume

that

processor

has

the

same

candidate

hash

tree

in

DD

then

it

will

not

update

any

counts

of

candidates

with

transaction

items

starting

with

B

D

E

F

G

and

H

but

it

can

find

that

out

only

after

processing

them

all

to

the

leaves

of

the

hash

tree

Our

approach

effectively

cuts

down

the

work

involved

with

subset

function

to

of

the

serial

work

by

partitioning

the

candidate

set

intelligently

and

using

bit

maps

to

prune

away

extra

computation

The

intelligent

partitioning

of

the

candidate

set

used

in

IDD

requires

our

algorithm

to

have

a

good

load

balancing

One

of

the

criteria

of

a

good

partitioning

involved

here

is

to

have

an

equal

number

of

candidates

in

all

the

processors

This

gives

about

the

same

size

hash

tree

in

all

the

processors

and

thus

provides

good

load

balancing

among

processors

Note

that

in

the

DD

algorithm

this

was

accomplished

by

distributing

candidates

in

round

robin

fashion

Another

criteria

is

to

have

each

processors

have

a

mixed

bag

of

the

candidates

This

will

help

to

prevent

load

imbalance

due

to

the

skew

in

data

For

instance

consider

a

data

Count

Count

Candidate

Hash

Tree

Count

Count

Candidate

Hash

Tree

Count

Count

Candidate

Hash

Tree

Count

Count

Candidate

Hash

Tree

A

C

Bit

Map

Proc

Proc

Proc

Proc

N/P

N/P

N/P

Local

Data

Remote

Data

Local

Data

Remote

Data

Local

Data

Remote

Data

Local

Data

Remote

Data

N/PData

DataData

N

number

of

data

items

M

size

of

candidate

set

P

number

of

processors

Shift

Shift

Shift

Data

Shift

Data

Shift

M/P

M/P

M/P

M/P

-LCB-

B

D

-RCB-

-LCB-

B

C

-RCB-

-LCB-

E

F

-RCB-

-LCB-

D

E

-RCB-

-LCB-

D

G

-RCB-

-LCB-

D

F

-RCB-

-LCB-

G

J

-RCB-

-LCB-

F

G

-RCB-

-LCB-

G

I

-RCB-

-LCB-

A

B

-RCB-

-LCB-

A

C

-RCB-

-LCB-

C

E

-RCB-

Bit

Map

Bit

Map

Bit

Map

B

E

F

GD

All-to-all

Broadcast

Figure

Intelligent

Data

Distribution

-LRB-

IDD

-RRB-

Algorithm

base

with

distinct

items

numbered

from

to

and

that

the

data

base

transactions

have

more

data

items

numbered

with

to

If

we

partition

the

candidates

between

two

processors

and

assign

all

the

candidates

starting

with

items

to

to

processor

and

candidates

starting

with

items

to

to

processor

P

then

there

would

be

more

work

for

processor

P

We

use

bin

packing

to

accomplish

the

load

balancing

Each

processor

generates

all

the

possible

candidates

and

just

keeps

the

counts

of

candidates

starting

with

each

items

and

does

not

save

the

candidates

This

-LRB-

first-item

count

-RRB-

pairs

are

stored

in

an

array

The

array

elements

are

randomly

swapped

around

to

give

a

good

mix

of

candidates

For

each

processor

we

keep

how

many

candidates

are

assigned

to

each

of

them

and

a

bit

map

to

keep

track

of

which

first

item

of

the

candidate

sets

belong

to

the

processor

We

pick

one

pair

at

a

time

from

the

array

and

give

it

to

the

processor

with

the

least

number

of

the

candidates

assigned

All

the

processors

update

the

count

for

this

processor

and

the

assigned

processor

sets

bit

map

for

the

first

item

of

the

candidates

This

bit

map

is

used

in

the

generation

of

candidates

that

belong

to

the

processor

and

is

also

used

in

the

top

level

of

hash

tree

as

discussed

before

At

the

end

of

the

bin

packing

process

we

perform

additional

shuffling

of

candidates

to

reduce

the

variance

in

the

number

of

candidates

assigned

We

were

able

to

achieve

less

than

of

load

imbalance

with

the

bin

packing

method

described

here

Hybrid

Algorithm

The

IDD

algorithm

exploits

the

total

system

memory

while

minimizing

the

communication

overhead

involved

However

as

the

number

of

processors

increases

this

algorithm

runs

into

a

problem

The

cause

of

the

problem

is

decreasing

amount

of

work

per

processor

as

the

number

of

processors

increases

The

average

number

of

candidates

assigned

to

each

processor

is

MP

where

M

is

the

number

of

total

candidates

As

more

number

of

processors

are

used

the

number

of

candidates

assigned

to

each

processor

decreases

This

has

two

implications

First

with

fewer

number

of

candidates

per

processor

it

is

much

more

difficult

to

have

a

good

load

balancing

Second

the

smaller

number

of

candidates

gives

a

smaller

hash

tree

and

less

computation

work

per

data

in

SBuf

of

Figure

The

amount

of

computation

might

be

less

than

the

communication

involved

This

will

be

an

even

more

serious

problem

in

a

system

that

can

not

perform

asynchronous

communication

The

Hybrid

Distribution

-LRB-

HD

-RRB-

algorithm

addresses

the

above

problems

by

dynamically

partitioning

the

candidate

set

into

a

big

enough

section

and

assign

a

group

of

processors

to

each

partition

The

same

parameter

that

was

used

to

determine

whether

to

switch

to

CD

algorithm

can

be

used

to

decide

the

size

of

the

partition

in

this

algorithm

For

example

let

this

parameter

be

C

If

the

total

number

of

candidates

M

is

less

than

C

it

switches

to

CD

algorithm

Otherwise

find

out

the

number

of

processor

groups

G

dM

Ce

and

form

a

logical

G

PG

processor

mesh

configuration

Figure

shows

processor

mesh

configuration

In

the

first

step

we

partition

hash

tree

and

update

hash

trees

according

to

the

IDD

algorithm

among

the

processors

in

the

same

columns

of

the

processor

mesh

Note

that

processors

in

the

same

row

have

the

exactly

same

candidates

and

candidate

sets

along

the

each

column

partition

the

total

candidate

set

At

the

end

of

the

operation

each

processor

has

complete

candidate

counts

for

all

the

data

of

the

processors

of

the

same

column

In

the

second

step

perform

reduction

operation

-LSB-

-RSB-

along

the

row

such

that

the

processor

in

the

first

column

of

the

same

row

has

the

total

counts

for

the

candidates

in

the

same

row

processors

In

the

third

step

all

the

processors

in

the

first

column

generate

frequent

set

from

the

candidate

set

and

perform

all

-LCB-

to

-LCB-

all

broadcast

operation

along

the

first

column

of

the

processor

mesh

Then

the

processors

in

the

first

column

broadcast

the

full

frequent

sets

to

the

processors

along

the

same

row

using

one

-LCB-

to

-LCB-

all

broadcast

operation

-LSB-

-RSB-

At

this

point

all

the

processors

have

the

frequent

sets

and

ready

to

proceed

to

the

next

pass

The

second

and

third

step

combines

the

global

reduction

step

of

the

CD

algorithm

and

all

-LCB-

to

-LCB-

all

broadcast

step

of

the

IDD

algorithm

This

algorithm

inherits

all

the

good

features

of

the

IDD

algorithm

and

also

provides

good

load

balance

and

enough

computation

work

by

maintaining

minimum

number

of

candidates

per

processor

In

the

same

time

the

amount

of

data

movement

in

this

algorithm

has

been

cut

down

to

of

the

IDD

Scalability

Analysis

We

will

concentrate

on

the

subset

function

described

in

Section

for

analysis

because

the

majority

of

computation

of

the

algorithm

is

performed

in

subset

function

Table

describes

the

symbols

used

in

this

section

Data

Shift

Data

Shift

Data

Shift

Data

Shift

Data

Shift

Data

Shift

Data

Shift

Data

Shift

Candidate

Hash

Tree

Candidate

Hash

Tree

Candidate

Hash

Tree

G

F

H

I

F

H

F

G

E

F

Data

Shift

Data

Shift

Data

Shift

Candidate

Hash

Tree

Candidate

Hash

Tree

Candidate

Hash

Tree

D

E

G

F

B

C

H

I

C

D

F

H

F

G

Candidate

Hash

Tree

Candidate

Hash

Tree

Candidate

Hash

Tree

D

E

B

C

H

I

C

D

F

H

A

B

D

E

B

C

C

D

A

B

G

F

E

F

F

G

A

B

E

F

Candidate

Hash

Tree

Candidate

Hash

Tree

Candidate

Hash

Tree

A

B

D

E

G

F

B

C

H

I

F

H

F

G

E

F

C

D

Candidate

Hash

Tree

G

F

Candidate

Hash

Tree

D

E

G

F

Candidate

Hash

Tree

D

E

A

B

D

E

A

B

G

F

A

B

Candidate

Hash

Tree

A

B

D

E

G

F

Candidate

Hash

Tree

H

I

E

F

Candidate

Hash

Tree

B

C

H

I

Candidate

Hash

Tree

B

C

H

I

B

C

E

F

E

F

Candidate

Hash

Tree

Candidate

Hash

Tree

F

H

F

G

Candidate

Hash

Tree

C

D

F

H

F

G

Candidate

Hash

Tree

C

D

F

H

C

D

F

G

Candidate

Hash

Tree

B

C

E

F

H

I

C

D

F

G

F

H

Frequent

Item

Set

F

H

H

I

Frequent

Item

Set

F

H

H

I

Frequent

Item

Set

F

H

H

I

Frequent

Item

Set

F

H

H

I

Frequent

Item

Set

F

H

H

I

Frequent

Item

Set

F

H

H

I

Frequent

Item

Set

F

H

H

I

Frequent

Item

Set

F

H

H

I

Frequent

Item

Set

F

H

H

I

Frequent

Item

Set

F

H

H

I

Frequent

Item

Set

F

H

H

I

Frequent

Item

Set

F

H

H

I

Data

Shift

Step

Reduction

Operation

Along

the

Rows

Follwed

by

One-to-all

Broadcast

Operation

Along

the

Rows

Step

All-to-all

Broadcast

Operation

Along

the

First

Column

All-to-all

Broadcast

Step

Partitioning

of

Candidate

Sets

and

Data

Movement

Along

the

Columns

Figure

Hybrid

Distribution

-LRB-

HD

-RRB-

Algorithm

in

Processor

Mesh

-LRB-

G

P

-RRB-

symbol

definition

N

Total

number

of

transactions

P

Number

of

processors

M

Total

number

of

candidates

k

Pass

number

in

Apriori

algorithm

l

Average

number

of

items

in

a

transaction

B

Branching

factor

of

candidate

hash

tree

h

Average

number

of

hash

function

calls

per

potential

candidate

c

Average

number

of

candidates

at

the

leaf

node

cmax

Maximum

number

of

candidates

allowed

at

the

leaf

node

Table

Symbols

used

in

the

analysis

A

transaction

with

size

l

at

pass

k

has

l

k

potential

candidates

to

check

against

the

candidate

hash

tree

For

each

potential

candidate

there

is

a

traversal

cost

h

and

cost

involved

with

checking

whether

the

potential

candidate

is

at

the

leaf

node

reached

The

major

cost

of

traversal

is

the

calls

to

hash

function

and

it

is

bounded

by

the

depth

of

the

hash

tree

Note

that

the

depth

of

a

hash

tree

at

pass

k

is

bounded

by

k

The

cost

of

checking

at

the

leaf

is

directly

proportional

to

the

average

number

of

candidates

in

the

leaf

node

So

the

complexity

of

the

subset

function

in

serial

algorithm

with

N

transaction

at

pass

k

is

N

l

k

-LRB-

hserial

cserial

-RRB-

with

the

following

two

cases

Case

k

logB

M

cmax

ie

the

tree

is

fully

expanded

to

depth

k

and

leaf

nodes

contain

more

than

cmax

candidates

Here

we

have

hserial

k

cserial

M

Bk

Case

k

logB

M

cmax

ie

the

tree

is

not

fully

expanded

to

depth

k.

Here

we

have

hserial

logB

M

cmax

cserial

cmax

The

communication

cost

of

the

CD

algorithm

is

O

-LRB-

M

-RRB-

due

to

the

global

reduction

operation

of

the

candidate

counts

The

computation

per

a

potential

candidate

of

a

transaction

is

the

same

as

the

serial

algorithm

So

the

computation

per

processor

with

NP

number

of

transactions

is

N

P

l

k

-LRB-

hserial

cserial

-RRB-

The

amount

of

computation

increases

either

by

increasing

M

with

lower

minimum

support

or

by

increasing

N

With

fixed

M

as

the

number

of

processors

increases

we

can

increase

N

to

increase

the

computation

per

processor

So

the

CD

algorithm

scales

with

N

However

with

fixed

N

CD

does

not

scale

with

M

The

major

communication

cost

of

the

DD

algorithm

is

O

-LRB-

N

-RRB-

due

to

the

all

-LCB-

to

-LCB-

all

broadcast

operations

of

moving

data

among

the

processors

The

computation

per

processor

in

the

DD

algorithm

is

N

l

k

-LRB-

hdata

cdata

-RRB-

with

the

following

two

cases

Case

k

logB

M

P

cmax

ie

the

tree

is

fully

expanded

to

depth

k

and

leaf

nodes

contain

more

than

cmax

candidates

Here

we

have

hdata

k

cdata

M

PBk

cserial

P

Compared

to

the

CD

the

traversal

cost

per

a

possible

candidate

is

the

same

and

the

checking

cost

at

the

leaf

is

of

the

CD

With

P

times

more

data

than

the

CD

the

DD

has

more

computation

work

because

the

overall

cost

for

traversing

is

P

times

more

than

that

of

the

CD

Case

k

logB

M

P

cmax

ie

the

tree

is

not

fully

expanded

to

depth

k.

Here

we

have

hdata

logB

M

P

cmax

logB

M

cmax

logB

P

cdata

cmax

cserial

In

this

case

the

checking

cost

per

a

possible

candidate

is

about

the

same

as

that

of

the

CD

and

the

upper

bound

of

the

traversal

cost

is

decreased

by

logB

P

which

is

less

than

for

B

P

With

P

times

more

data

the

DD

has

almost

P

times

more

computation

work

for

checking

cost

at

the

leaf

and

close

to

P

times

more

cost

for

traversing

Compared

to

the

case

above

the

performance

in

this

case

is

worse

in

general

This

shows

that

the

DD

algorithm

works

better

with

larger

number

of

candidates

The

DD

algorithm

does

not

scale

with

N

due

to

the

communication

overhead

O

-LRB-

N

-RRB-

However

with

fixed

N

as

the

number

of

processors

increases

we

can

increase

M

to

increase

cdata

in

case

and

the

upper

bound

of

hdata

in

case

above

So

DD

scales

with

M

The

communication

cost

of

the

IDD

is

the

same

as

the

DD

However

the

computation

cost

is

roughly

of

the

DD

because

of

the

intelligent

partitioning

of

candidates

set

and

the

use

of

bitmap

to

prune

at

the

root

of

the

hash

tree

So

the

computation

per

processor

is

N

l

k

P

-LRB-

hdata

cdata

-RRB-

N

P

l

k

-LRB-

hdata

cdata

-RRB-

Unlike

the

DD

algorithm

the

amount

of

computation

work

per

processor

is

about

the

same

as

that

of

the

CD

However

P

must

be

relatively

small

to

have

a

good

load

balance

If

P

becomes

large

with

fixed

M

the

problem

of

load

imbalance

discussed

in

Section

makes

some

processors

work

on

more

than

of

items

in

a

transaction

at

the

root

of

the

hash

tree

IDD

scales

better

than

DD

due

to

the

less

computation

work

but

the

asymptotic

scalability

behavior

is

the

same

as

that

of

DD

For

the

HD

algorithm

with

G

PG

processor

partitions

the

communication

cost

is

O

-LRB-

GNP

-RRB-

for

data

movement

and

O

-LRB-

MG

-RRB-

for

the

reduction

operations

The

total

number

of

transactions

each

processor

has

to

process

is

GNP

and

only

of

the

possible

candidates

in

a

transaction

will

go

through

the

hash

tree

So

the

computation

per

processor

is

GN

P

l

k

G

-LRB-

hhybrid

chybrid

-RRB-

N

P

l

k

-LRB-

hhybrid

chybrid

-RRB-

We

can

show

that

HD

scales

with

both

N

and

M

With

fixed

M

as

we

increase

P

we

can

increase

N

to

increase

the

amount

of

computation

Note

also

that

the

communication

cost

for

data

movement

stays

the

same

as

both

P

and

N

increases

With

fixed

N

as

we

increase

P

we

can

increase

the

amount

of

computation

by

increasing

M

and

thus

increasing

hhybrid

or

chybrid

As

M

increases

the

number

of

partitions

of

candidate

set

increases

i.e.

G

increases

The

communication

cost

for

data

movement

stays

the

same

as

both

P

and

G

increase

and

the

communication

cost

for

the

reduction

operations

stays

the

same

as

both

M

and

G

increase

So

the

overall

communication

cost

overhead

does

not

change

Experimental

Results

We

implemented

our

parallel

algorithms

on

a

-LCB-

processor

Cray

parallel

computer

Each

processor

on

the

is

a

Dec

Alpha

-LRB-

-RRB-

and

has

of

memory

The

processors

are

interconnected

via

a

three

dimensional

torus

network

that

has

a

peak

unidirectional

bandwidth

of

per

second

and

a

small

latency

For

communication

we

used

the

message

passing

interface

-LRB-

MPI

-RRB-

Our

experiments

have

shown

that

for

we

obtain

a

bandwidth

of

and

an

effective

startup

time

of

microseconds

We

generated

a

synthetic

dataset

using

a

tool

provided

by

-LSB-

-RSB-

and

described

in

-LSB-

-RSB-

The

parameters

for

the

data

set

chosen

are

average

transaction

length

of

and

average

size

of

frequent

item

sets

of

Data

sets

with

transactions

-LRB-

KB

-RRB-

were

generated

for

different

processors

Due

to

the

disk

limitations

of

the

system

we

have

60 80 100 120

Response

time

-LRB-

sec

-RRB-

Number

of

processors

count

intelligent

data

hybrid

Figure

Scaleup

result

with

transactions

and

minimum

support

kept

the

small

transactions

in

the

buffer

and

read

the

transactions

from

the

buffer

instead

of

from

the

actual

disks

For

larger

data

sets

in

the

experiment

we

read

the

same

data

set

multiple

times

We

performed

scaleup

a

test

with

transactions

per

processor

with

minimum

support

of

We

could

not

use

lower

minimum

support

because

the

CD

algorithm

ran

out

of

main

memory

For

this

experiment

in

the

IDD

and

HD

algorithms

we

have

set

the

minimum

number

of

candidates

for

switching

to

the

CD

algorithm

very

low

to

show

the

validity

of

our

approaches

With

support

both

algorithms

switched

to

CD

algorithm

in

pass

of

total

passes

and

of

the

overall

response

time

of

the

serial

code

was

spent

in

the

first

passes

These

scaleup

results

are

shown

in

Figure

As

noted

in

-LSB-

-RSB-

the

CD

algorithm

scales

very

well

Looking

at

the

performance

obtained

by

IDD

we

see

that

its

response

time

increases

as

we

increase

the

number

of

processors

This

is

due

to

the

load

balancing

problem

we

discussed

in

Section

where

the

number

of

candidates

per

processor

decreases

as

the

number

of

processors

increases

However

the

performance

achieved

by

IDD

is

much

better

than

that

of

the

DD

algorithm

-LSB-

-RSB-

In

particular

DD

requires

to

times

more

time

than

CD

on

processors

-LSB-

-RSB-

whereas

IDD

is

only

slower

In

fact

even

on

processors

IDD

is

only

times

slower

than

CD

Looking

at

the

performance

of

the

HD

algorithm

we

see

that

response

time

remains

almost

constant

as

we

increase

the

number

of

processors

while

keeping

the

number

of

transactions

per

processor

and

the

minimum

support

fixed

Comparing

against

CD

we

see

that

HD

actually

performs

better

as

the

number

of

processors

increases

Its

performance

on

also

performed

similar

experiments

on

an

IBM

in

which

the

entire

database

resided

on

disks

Our

experiments

show

that

the

I/O

requirements

do

not

change

the

relative

performance

of

the

various

schemes

200 400 600 800

Response

time

-LRB-

sec

-RRB-

Number

of

transactions

per

processor

-LRB-

K

-RRB-

count

intelligent

data

hybrid

Figure

Sizeup

result

with

processors

and

minimum

support

processors

is

better

than

CD

This

performance

advantage

of

HD

over

CD

is

because

the

number

of

processors

involved

in

global

reduction

operation

of

counts

is

much

less

in

HD

than

CD

We

measured

how

our

algorithms

perform

as

we

increase

the

number

of

transactions

per

node

from

-LRB-

MB

-RRB-

to

-LRB-

MB

-RRB-

For

these

experiments

we

fixed

the

number

of

processors

at

and

the

minimum

support

at

These

results

are

shown

in

Figure

From

this

figure

we

can

see

that

CD

and

HD

perform

almost

identically

For

both

of

the

algorithms

the

response

time

increases

linearly

with

the

number

of

transactions

IDD

also

scales

linearly

but

because

of

its

load

imbalance

problem

its

performance

is

somewhat

worse

Our

experiments

so

far

have

shown

that

the

performance

of

HD

and

CD

are

quite

comparable

However

the

real

advantage

of

HD

-LRB-

and

IDD

-RRB-

over

CD

is

that

they

do

not

require

the

whole

hash

tree

to

reside

on

each

processor

and

thus

better

exploit

the

available

memory

This

allows

use

of

the

Apriori

algorithm

with

smaller

support

level

To

verify

this

we

performed

the

experiments

in

which

we

fix

the

number

of

transactions

per

processor

to

and

successively

decreased

the

minimum

support

level

These

experiments

for

and

processors

are

shown

in

Figures

and

respectively

A

number

of

interesting

observations

can

be

made

from

these

results

First

both

IDD

and

HD

successfully

ran

using

lower

support

levels

than

CD

In

particular

IDD

and

HD

ran

down

to

a

support

level

of

on

processors

and

on

processors

In

contrast

CD

could

only

run

down

to

a

support

level

of

and

ran

out

of

memory

for

the

lower

supports

The

difference

between

the

smaller

support

levels

on

and

processors

is

due

to

the

fact

that

the

algorithms

can

exploit

the

aggregate

memory

of

the

larger

number

of

processors

The

second

thing

to

notice

is

that

HD

performs

better

than

IDD

both

on

and

processors

and

the

relative

performance

of

IDD

compared

to

HD

get

worse

as

the

number

Response

time

-LRB-

sec

-RRB-

Minimum

support

-LRB-

-RRB-

-LRB-

K

-RRB-

-LRB-

K

-RRB-

-LRB-

K

-RRB-

-LRB-

K

-RRB-

count

intelligent

data

hybrid

simple

hybrid

Figure

Response

time

on

processors

with

transactions

as

the

minimum

support

varies

of

processors

increases

As

discussed

earlier

this

performance

difference

is

due

to

load

imbalance

As

the

number

of

processors

increases

this

load

imbalance

gets

worse

However

on

processors

IDD

is

worse

than

HD

for

support

level

but

only

worse

for

support

of

This

is

because

as

the

support

level

decreases

the

number

of

candidates

-LRB-

shown

in

parenthesis

in

Figures

and

-RRB-

increase

which

improves

the

load

balance

Figures

and

also

shows

the

performance

of

a

simple

hybrid

algorithm

obtained

by

combining

CD

and

IDD

In

this

scheme

in

each

pass

of

the

Apriori

algorithm

we

perform

CD

if

the

hash

table

can

fit

in

the

memory

of

each

processors

or

IDD

if

it

can

not

As

we

can

see

from

these

results

this

simple

hybrid

algorithm

performs

worse

than

HD

In

particular

the

relative

performance

of

this

scheme

compare

to

HD

gets

worse

as

the

number

of

processors

increases

For

example

for

a

support

level

of

it

is

worse

on

processors

and

worse

on

processors

Thus

the

HD

algorithm

by

gradually

adjusting

the

subsets

of

processors

that

perform

IDD

and

CD

achieves

better

performance

This

is

because

of

the

following

two

reasons

First

the

candidate

set

is

split

among

a

fewer

number

of

processors

which

minimizes

load

imbalance

and

second

the

reduction

operation

to

obtain

the

counts

in

CD

are

performed

among

fewer

processors

decreasing

the

communication

overhead

In

another

experiment

we

varied

the

number

of

processors

from

to

and

measured

how

low

we

can

go

with

minimum

support

for

our

algorithms

Table

shows

the

results

The

result

shows

that

as

we

have

more

processors

our

algorithms

can

handle

more

candidates

generated

due

to

lower

minimum

support

Table

shows

how

the

HD

algorithm

chose

the

processor

configuration

based

on

the

number

of

candidates

at

each

pass

with

processors

and

minimum

support

Response

time

-LRB-

sec

-RRB-

Minimum

support

-LRB-

-RRB-

-LRB-

K

-RRB-

-LRB-

K

-RRB-

-LRB-

K

-RRB-

-LRB-

K

-RRB-

-LRB-

K

-RRB-

count

intelligent

data

hybrid

simple

hybrid

Response

time

-LRB-

sec

-RRB-

Minimum

support

-LRB-

-RRB-

-LRB-

K

-RRB-

-LRB-

K

-RRB-

-LRB-

K

-RRB-

-LRB-

K

-RRB-

-LRB-

K

-RRB-

count

intelligent

data

hybrid

simple

hybrid

Figure

Response

time

on

processors

with

transactions

as

the

minimum

support

varies

Number

of

processors

Successful

down

to

Ran

out

of

memory

at

Table

Minimum

support

-LRB-

-RRB-

reachable

with

different

number

of

processors

in

our

algorithms

Pass

Configuration

No

of

Cand

Table

Processor

configuration

and

number

of

candidates

of

the

HD

algorithm

with

processors

and

minimum

support

for

each

pass

Note

that

configuration

is

the

same

as

the

DD

algorithm

and

is

the

same

as

the

CD

algorithm

The

total

pass

was

and

all

passes

after

had

configuration

Conclusion

In

this

paper

we

proposed

two

parallel

algorithms

for

mining

association

rules

The

IDD

algorithm

utilizes

total

main

memory

available

more

effectively

than

the

CD

algorithm

This

algorithms

improves

over

the

DD

algorithm

which

has

high

communication

overhead

and

has

extra

work

The

communication

overhead

was

reduced

using

a

better

data

movement

communication

mechanism

and

extra

work

was

reduced

by

partitioning

the

candidate

set

intelligently

and

using

bit

maps

to

prune

away

extra

computation

However

as

the

number

of

processors

available

increases

the

efficiency

of

this

algorithm

decreases

unless

the

amount

of

work

is

increased

by

having

more

number

of

candidates

The

HD

combines

advantages

of

the

CD

and

IDD

This

algorithm

partitions

candidate

sets

just

like

the

IDD

to

exploit

the

aggregate

main

memory

but

dynamically

determines

the

number

of

partitions

such

that

the

partitioned

candidate

set

fits

into

the

main

memory

of

each

processor

and

each

processor

has

enough

number

of

candidates

for

computation

It

also

exploits

the

advantage

of

the

CD

by

just

exchanging

counts

information

and

moving

around

the

minimum

number

of

transactions

among

the

smaller

subset

of

processors

The

experimental

results

on

a

-LCB-

processor

Cray

parallel

machine

show

that

the

HD

algorithm

scales

just

as

well

as

the

CD

It

also

exploits

the

aggregate

main

memory

better

and

thus

able

to

find

out

more

association

rules

with

much

smaller

minimum

support

with

a

single

scan

of

database

per

pass

The

IDD

algorithm

also

shows

that

it

outperforms

the

DD

algorithm

but

is

not

as

scalable

as

HD

and

CD

Future

works

include

applying

these

algorithms

to

real

data

like

retail

sales

transaction

mail

order

history

database

and

World

Wide

Web

server

logs

-LSB-

-RSB-

to

confirm

the

experimental

results

in

the

real

life

domain

We

plan

on

porting

our

code

to

different

platforms

including

Cray

IBM

and

SGI

SMP

clusters

We

also

plan

on

implementing

our

ideas

in

generalized

association

rules

-LSB-

-RSB-

and

sequential

patterns

-LSB-

-RSB-

References

-LSB-

-RSB-

R.

Agrawal

T.

Imielinski

and

A.

Swami

Mining

association

rules

between

sets

of

items

in

large

databases

In

Proc

of

ACM-SIGMOD

Int

Conf

on

Management

of

Data

Washington

D.C.

-LSB-

-RSB-

R.

Agrawal

and

R.

Srikant

Fast

algorithms

for

mining

association

rules

In

Proc

of

the

VLDB

Conference

pages

-LCB-

Santiago

Chile

-LSB-

-RSB-

R.

Agrawal

and

J.C.

Shafer

Parallel

mining

of

association

rules

Design

implementation

and

experience

Technical

Report

IBM

Research

Division

Almaden

Research

Center

-LSB-

-RSB-

J.

Han

and

Y.

Fu

Discovery

of

multiple

-LCB-

level

association

rules

from

large

databases

In

Proc

of

the

VLDB

Conference

Zurich

Switzerland

-LSB-

-RSB-

M.

A.

W.

Houtsma

and

A.

N.

Swami

Set-oriented

mining

for

association

rules

in

relational

databases

In

Proc

of

the

Int

l

Conf

on

Data

Eng.

pages

-LCB-

Taipei

Taiwan

-LSB-

-RSB-

Vipin

Kumar

Ananth

Grama

Anshul

Gupta

and

George

Karypis

Introduction

to

Parallel

Computing

Algorithm

Design

and

Analysis

Benjamin

Cummings

Addison

Wesley

Redwod

City

-LSB-

-RSB-

B.

Mobasher

N.

Jain

E.H.

Han

and

J.

Srivastava

Web

mining

Pattern

discovery

from

world

wide

web

transactions

Technical

Report

Department

of

Computer

Science

University

of

Minnesota

M

inneapolis

-LSB-

-RSB-

H.

Mannila

H.

Toivonen

and

A.

I.

Verkamo

Discovering

frequent

episodes

in

sequences

In

Proc

of

the

First

Int

l

Conference

on

Knowledge

Discovery

and

Data

Mining

pages

-LCB-

Montreal

Quebec

-LSB-

-RSB-

IBM

Quest

Data

Mining

Project

Quest

synthetic

data

generation

code

-LSB-

-RSB-

R.

Srikant

and

R.

Agrawal

Mining

generalized

association

rules

In

Proc

of

the

VLDB

Conference

pages

-LCB-

Zurich

Switzerland

-LSB-

-RSB-

R.

Srikant

and

R.

Agrawal

Mining

sequential

patterns

Generalizations

and

performance

improvements

In

Proc

of

the

Fifth

Int

l

Conference

on

Extending

Database

Technology

Avignon

France

-LSB-

SAD

-RSB-

M.

Stonebraker

R.

Agrawal

U.

Dayal

E.

J.

Neuhold

and

A.

Reuter

DBMS

research

at

a

crossroads

The

vienna

update

In

Proc

of

the

VLDB

Conference

pages

-LCB-

Dublin

Ireland

-LSB-

-RSB-

A.

Savasere

E.

Omiecinski

and

S.

Navathe

An

efficient

algorithm

for

mining

association

rules

in

large

databases

In

Proc

of

the

VLDB

Conference

pages

-LCB-

Zurich

Switzerland

Contents

Introduction

Basic

Concepts

Parallel

Algorithms

Count

Distribution

Algorithm

Data

Distribution

Algorithm

Intelligent

Data

Distribution

Algorithm

Hybrid

Algorithm

Scalability

Analysis

Experimental

Results

Conclusion

List

of

Figures

Apriori

Algorithm

Count

Distribution

-LRB-

CD

-RRB-

Algorithm

Data

Distribution

-LRB-

DD

-RRB-

Algorithm

Pseudo

Code

for

Data

Movements

Intelligent

Data

Distribution

-LRB-

IDD

-RRB-

Algorithm

Hybrid

Distribution

-LRB-

HD

-RRB-

Algorithm

in

Processor

Mesh

-LRB-

G

P

-RRB-

Scaleup

result

with

transactions

and

minimum

support

Sizeup

result

with

processors

and

minimum

support

Response

time

on

processors

with

transactions

as

the

minimum

sup

port

varies

Response

time

on

processors

with

transactions

as

the

minimum

sup

port

varies

List

of

Tables

Symbols

used

in

the

analysis

Minimum

support

-LRB-

-RRB-

reachable

with

different

number

of

processors

in

our

algorithms

Processor

configuration

and

number

of

candidates

of

the

HD

algorithm

with

processors

and

minimum

support

for

each

pass

Note

that

configuration

is

the

same

as

the

DD

algorithm

and

is

the

same

as

the

CD

algorithm

The

total

pass

was

and

all

passes

after

had

configuration

association rules
data mining
load balance
scalability