Experiences

with

Processor

Reservation

and

Dynamic

QOS

in

Real-Time

Mach

Chen

Lee

Ragunathan

Rajkumar

and

Cliff

Mercer

Department

of

Computer

Science

Carnegie

Mellon

University

Pittsburgh

PA

USA

-LCB-

clee,raj+,cwm@cs.cmu.edu

-RCB-

Abstract

The

RT-Mach

microkernel

supports

a

processor

reserve

abstraction

which

permits

threads

to

specify

their

CPU

resource

requirements

If

admitted

by

the

kernel

it

guarantees

that

the

requested

CPU

demand

is

available

to

the

requestor

We

designed

this

kernel-supported

mechanism

to

be

relatively

simple

based

on

the

microkernel

notion

that

user-level

policies

can

use

this

simple

mechanism

to

build

more

complex

and

powerful

schemes

In

this

paper

we

focus

on

the

needs

of

such

user-level

policies

in

the

form

of

a

dynamic

Quality

of

Service

server

We

seek

three

goals

-LRB-

a

-RRB-

explore

the

necessity

sufficiency

power

and

flexibility

of

the

kernel-supported

reserve

mechanism

-LRB-

b

-RRB-

dynamic

management

of

application

quality

in

real-time

and

multimedia

applications

and

-LRB-

c

-RRB-

investigate

our

ability

to

predict

and

achieve

end-to-end

application

delays

in

realistic

distributed

real-time

and

multimedia

applications

We

use

a

two-pronged

approach

to

accomplish

our

goals

First

we

apply

the

processor

reserve

abstraction

in

a

user-level

dynamic

quality

of

service

server

A

QOS

server

can

allow

applications

to

dynamically

adapt

in

real-time

based

on

system

load

user

input

or

application

requirements

Second

we

apply

the

dynamic

QOS

control

capabilities

to

a

distributed

multimedia

application

whose

threads

have

to

interact

and

coordinate

with

each

other

within

and

across

processor

boundaries

A

new

notion

called

continuous

thread

of

control

is

introduced

to

assist

in

bundling

processor

reserves

Our

experiments

show

that

we

can

indeed

predict

and

achieve

end-to-end

delays

in

a

distributed

multimedia

application

A

summary

of

lessons

learned

and

additional

functionality

needed

is

also

provided

Introduction

The

need

for

real-time

resource

management

is

a

key

element

which

explicitly

distinguishes

multimedia

applications

from

other

traditional

applications

The

Real-Time

Mach

kernel

-LSB-

-RSB-

supports

many

primitives

for

constructing

predictable

and

dynamic

real-time

applications

These

primitives

include

a

wide

choice

of

real-time

scheduling

policies

-LRB-

including

fixed

priority

scheduling

rate-monotonic

scheduling

and

earliest

deadline

scheduling

-RRB-

periodic

and

aperiodic

real-time

threads

with

the

notion

of

deadlines

and

handlers

for

deadline

misses

real-time

synchronization

primitives

-LSB-

-RSB-

and

real-time

inter-process

communication

primitives

which

support

priority

inheritance

-LSB-

-RSB-

virtual

memory

wiring

a

real-time

shell

and

a

network

protocol

server

a

processor

reserve

abstraction

an

integrated

toolset

for

schedulability

analysis

and

real-time

monitoring

of

scheduling

decisions

-LSB-

-RSB-

and

real-time

windowing

services

-LSB-

-RSB-

In

this

paper

we

build

upon

the

processor

reserve

abstraction

to

provide

dynamic

quality

of

service

support

and

to

predict

and

achieve

end-to-end

delays

in

distributed

multimedia

applications

Processor

Reserves

in

Real-Time

Mach

The

Real-Time

Mach

microkernel

supports

an

abstraction

called

processor

capacity

reserves

-LSB-

-RSB-

which

allows

application

threads

to

specify

their

CPU

requirements

in

terms

of

their

timing

constraints

The

kernel

performs

admission

control

using

a

specification

that

consists

of

required

CPU

usage

per

specified

interval

-LRB-

such

as

ms

every

ms

or

ms

every

ms

-RRB-

The

kernel

scheduler

schedules

application

threads

such

that

these

timing

constraints

are

satisfied

The

kernel

allows

multiple

threads

-LRB-

possibly

from

different

tasks

-RRB-

to

be

bound

to

the

same

reserve

The

kernel

enforces

that

application

threads

bound

to

a

reserve

specification

can

not

disrupt

the

timing

behavior

of

other

applications

This

is

achieved

by

ensuring

that

only

the

reserve

specification

is

guaranteed

by

the

kernel

and

beyond

the

stated

usage

level

the

kernel

can

suspend

or

execute

the

demanding

application

threads

at

low

priority

Such

kernel

enforcement

of

each

reserve

provides

a

temporal

protection

barrier

between

applications

a

temporal

domain

analogue

to

the

address

space

protection

of

processes

in

the

spatial

domain

-LSB-

-RSB-

Real-time

applications

with

reserves

and

non-real-time

applications

which

do

not

need

guarantees

reserves

can

co-exist

comfortably

Applications

without

reserves

are

implicitly

bound

by

the

kernel

to

a

default

reserve

which

is

scheduled

only

where

there

are

no

other

threads

bound

to

reserves

ready

to

run

System

calls

are

available

to

provide

feedback

to

applications

about

the

recent

usage

and

guaranteed

information

of

various

reserves

Since

the

kernel

must

monitor

the

execution

time

of

threads

to

enforce

reserves

it

provides

a

built-in

framework

for

measuring

the

amount

of

time

spent

by

threads

overhead

and

CPU

idleness

The

CPU

idleness

is

charged

to

a

kerneldefined

idle

reserve

Reserve

parameters

can

be

dynamically

adjusted

subject

to

the

admission

control

policy

The

timing

behavior

of

reserved

applications

can

therefore

be

changed

dynamically

These

features

of

the

processor

reserve

abstraction

were

intended

to

provide

a

simple

yet

powerful

set

of

mechanisms

for

use

by

real-time

and

multimedia

applications

in

a

context

where

non-real-time

applications

may

also

co-exist

This

abstraction

has

been

tested

previously

in

stand-alone

applications

such

as

a

video

viewer

which

reads

a

file

into

memory

in

non-real-time

-LRB-

background

-RRB-

mode

and

then

displays

frames

in

real-time

mode

Related

Work

Software

developers

have

written

many

multimedia

applications

that

run

acceptably

on

general

purpose

operating

systems

as

long

as

no

other

programs

compete

for

system

resources

-LSB-

-RSB-

With

additional

real-time

operating

system

support

to

carefully

manage

operating

system

resources

these

applications

could

run

even

with

competition

Many

researchers

in

the

distributed

real-time

multimedia

community

have

turned

their

attention

to

end-to-end

performance

guarantees

which

ultimately

include

network

communication

and

end-system

resource

management

-LRB-

including

but

not

limited

to

CPU

network

protocol

stack

memory

file

system

or

server

-RRB-

Much

research

has

been

done

on

the

networking

issues

-LSB-

-RSB-

-LSB-

-RSB-

-LSB-

-RSB-

to

name

a

few

in

which

specific

assumptions

are

made

about

the

endpoints

of

the

distributed

computation

As

that

work

matures

attention

is

turning

to

issues

of

end-system

control

as

well

-LSB-

-RSB-

Jeffay

et

al.

-LSB-

-RSB-

employ

hard

real-time

scheduling

theory

in

a

specialized

micro-kernel

to

address

timing

issues

related

to

different

stages

of

live

video

and

audio

processing

Robin

et

al.

-LSB-

-RSB-

designed

a

system

based

on

Chorus

-LSB-

-RSB-

micro-kernel

that

addresses

both

the

network

and

end

host

QOS

control

The

system

uses

an

earliest-deadlinefirst

scheduling

policy

and

a

time-line-based

admission

test

for

guaranteed

class

threads

Nahrstedt

and

Smith

-LSB-

-RSB-

used

AIX

for

their

telerobotic

application

and

showed

that

the

AIX

real-time

priorities

are

not

enough

to

control

protocol

task

behavior

when

used

for

implementation

of

rate-monotonic

or

deadline-based

scheduling

unless

severe

restrictions

are

made

which

includes

only

one

user

allowed

one

multimedia

application

running

on

the

application/transport

protocols

implemented

in

a

single

user

process

with

real-time

priority

etc.

Only

with

these

restrictions

satisfied

can

we

map

rate-monotonic

scheduling

onto

the

real-time

priority

scheme

of

AIX

to

provide

-LRB-

approximate

-RRB-

predictability

for

guaranteed

services

-LSB-

-RSB-

Organization

of

the

Paper

The

rest

of

this

paper

is

organized

as

follows

In

Section

we

outline

the

objectives

of

a

dynamic

QOS

server

on

top

of

RT-Mach

and

describe

its

architecture

In

Section

we

describe

a

real

distributed

multimedia

application

called

RT-Phone

In

Section

we

apply

the

reserve

and

QOS

abstractions

to

RT-Phone

We

also

derive

the

predicted

end-to-end

delay

that

must

be

guaranteed

by

the

use

of

reserves

A

new

concept

named

continuous

thread

of

control

is

introduced

to

bundle

many

threads

to

a

single

reserve

and

a

multi-phase

protocol

to

coordinate

two

QOS

servers

is

presented

We

also

provide

performance

numbers

from

RT-Phone

to

demonstrate

that

we

do

satisfy

the

predicted

end-to-end

application

delay

Finally

Section

presents

a

summary

of

the

lessons

that

we

learned

based

on

RT-Phone

and

our

experiments

Dynamic

Quality

of

Service

Control

The

relatively

simple

mechanisms

associated

with

the

reserve

mechanism

are

designed

to

be

basic

building

blocks

that

can

be

used

to

construct

more

powerful

user-level

schemes

Real-time

and

multimedia

applications

on

RT

Mach

can

not

be

assumed

to

be

static

in

nature

Thus

we

must

allow

a

dynamic

mix

of

concurrent

real-time

and

multimedia

applications

whose

timing

requirements

not

only

vary

widely

but

can

also

change

during

run-time

Instead

of

each

application

designing

its

own

scheme

-LRB-

s

-RRB-

to

use

the

kernel

reserve

support

a

consistent

framework

that

lays

out

the

ground

rules

for

cooperation

and

coordination

between

various

applications

is

highly

desirable

Based

on

this

motivation

we

propose

a

dynamic

user-level

QOS

server

with

two

specific

requirements

First

it

must

be

able

to

meet

the

timing

requirements

of

each

application

independent

of

the

behavior

of

other

applications

Second

it

must

be

able

to

let

an

application

dynamically

adapt

its

own

behavior

based

on

its

own

internal

requirements

user

input

and/or

the

total

load

on

the

system

Quality

Management

and

its

Implications

to

Systems

Support

We

summarize

our

approach

to

quality

management

as

follows

The

quality

of

an

application

can

ultimately

be

defined

only

by

the

application

in

concern

For

example

one

recording

application

may

emphasize

the

reception

of

all

incoming

audio

packets

another

interactive

application

may

emphasize

lower

jitter

and

yet

another

may

emphasize

very

low

end-to-end

delay

An

application-independent

kernel

can

therefore

can

not

institute

direct

support

for

all

possible

notions

of

quality

However

application

quality

requirements

can

and

in

practice

are

eventually

translated

to

the

demands

that

they

place

on

system

resources

From

a

kernel

's

perspective

therefore

we

strive

to

support

flexible

and

powerful

mechanisms

which

can

support

variations

in

the

resource

requirements

of

various

applications

and

also

dynamic

quality

changes

that

may

be

required

of

individual

applications

The

key

theme

behind

our

resource

management

approach

is

not

unlike

that

behind

the

reserve

abstraction

We

aim

to

provide

a

mechanism

between

the

application

and

the

system

layer

wherein

the

application

can

flexibly

specify

its

requirements

to

the

system

layer

and

in

turn

the

system

layer

can

provide

accurate

and

dynamic

feedback

on

the

state

of

the

application

's

resources

individually

and

with

respect

to

the

other

applications

that

are

co-resident

The

QOS

server

we

describe

next

is

based

on

this

primary

theme

The

QOS

Server

The

architecture

of

the

QOS

server

is

presented

in

Figure

Our

implementation

currently

assumes

that

applications

are

cooperative

rather

than

malicious

in

AdmissionControl

Display

DynamicQualityControlClientQuality

AttributesDatabaseQOSInterfaceClient

QOSServer

QOSInterfaceClientProcessorReservation

Real-Time

Mach

Figure

The

Architecture

of

the

Dynamic

QOS

Server

The

QOS

server

is

based

on

the

following

complementary

aspects

QOS

attributes

are

used

by

application

threads

to

specify

their

acceptable

quality

levels

to

the

QOS

server

An

application

can

choose

from

one

of

many

quality

adjustment

policies

that

the

QOS

server

can

use

on

it

if

the

quality

guarantee

given

to

the

application

has

to

be

assumption

can

be

eliminated

by

forcing

the

QOS

server

to

be

the

sole

interface

to

the

kernel

's

reservation

mechanisms

similar

to

the

user-level

UX

server

on

Mach

upgraded

or

downgraded

In

other

words

this

determines

how

an

application

wants

to

be

treated

when

its

current

resource

allocation

has

to

be

altered

Included

in

this

policy

choice

is

an

application

quality

adjustment

priority

which

determines

the

order

by

which

the

QOS

server

selects

the

beneficiary

-LRB-

victim

-RRB-

of

a

server

decision

to

upgrade

-LRB-

downgrade

-RRB-

the

quality

of

one

or

more

applications

One

of

many

admission

control

adjustment

policies

is

used

by

the

QOS

server

to

admit

as

many

new

application

requests

as

possible

without

violating

the

needs

of

currently

registered

applications

One

of

many

overrun

control

policies

is

used

by

the

QOS

to

satisfy

the

needs

of

registered

applications

whose

current

CPU

demand

exceeds

their

actual

allocation

This

is

done

by

the

QOS

server

exploiting

the

slack

that

may

be

available

from

applications

not

using

their

current

resource

allocation

These

components

of

the

QOS

server

are

described

in

greater

detail

next

QOS

Attributes

Application

threads

can

submit

requests

to

the

QOS

server

for

resource

allocation

specifying

QOS

attributes

The

kernel

reservation

mechanism

requires

a

fixed

-LRB-

worst-case

-RRB-

computation

time

and

a

fixed

period

but

the

QOS

server

relaxes

this

constraint

to

be

more

flexible

and

dynamic

QOS

attributes

include

the

minimum

maximum

and

mostdesired

levels

of

the

computation

time

as

well

as

the

minimum

maximum

and

most-desired

levels

of

the

period

In

other

words

if

an

application

chooses

a

range

of

acceptable

computation

times

-LRB-

or

periods

-RRB-

the

QOS

server

is

free

to

pick

a

legal

value

in

this

range

The

QOS

server

tries

to

provide

to

each

application

the

maximum

quality

value

in

this

range

but

will

not

go

below

the

minimum

requested

value

-LRB-

once

an

application

is

admitted

-RRB-

Quality

Adjustment

A

quality

adjustment

policy

for

each

request

indicates

how

the

QOS

server

should

pick

from

the

possibly

wide

range

of

possibilities

for

the

-LCB-

computation

time

period

-RCB-

specified

in

an

application

's

QOS

attribute

An

Adjust-Computation-Time-First

policy

adjusts

the

computation

time

first

keeping

the

period

constant

-LRB-

later

adjusting

the

period

if

needed

-RRB-

A

Keep-Period-Constant

policy

is

obtained

by

specifying

the

minimum

maximum

and

desired

values

of

the

period

to

be

the

same

This

option

can

be

utilized

by

applications

which

can

pick

from

different

algorithms

to

process

their

data

-LRB-

e.g.

choose

a

simple

but

less

accurate

algorithm

versus

a

complex

but

more

accurate

algorithm

-RRB-

or

pick

different

paths

of

the

same

algorithm

based

on

the

available

time

-LRB-

e.g.

choose

not

to

decode

some

blocks

in

JPEG

decoding

of

a

video

frame

-RRB-

An

Adjust-Period-First

policy

adjusts

the

period

first

keeping

the

computation

time

constant

-LRB-

later

adjusting

the

computation

time

if

needed

-RRB-

A

Keep-Computation-Time-Constant

policy

is

obtained

by

the

application

by

specifying

the

minimum

maximum

and

desired

values

of

the

computation

time

to

be

the

same

This

policy

can

be

useful

for

applications

which

can

tolerate

changes

to

their

rate

of

execution

but

not

their

computation

time

per

instance

-LRB-

e.g.

capture

a

video

frame

and

compress

it

using

JPEG

but

the

rate

of

capture

can

be

varied

-RRB-

A

Keep-Reservation-Constant

policy

is

obtained

by

specifying

the

minimum

maximum

and

desired

values

of

the

computation

time

to

be

the

same

and

similarly

for

the

period

A

Step-wise

Adjustment

policy

is

available

for

the

computation

time

-LRB-

period

-RRB-

in

which

the

computation

time

-LRB-

period

-RRB-

will

be

adjusted

only

in

discrete

steps

the

size

of

which

is

specified

by

the

application

A

Negotiation

Policy

tells

the

QOS

server

to

notify

the

client

when

an

adjustment

needs

to

be

made

-LRB-

along

with

information

about

the

maximum

available

reservation

at

the

time

-RRB-

and

the

client

can

negotiate

the

computation

time

and

reservation

parameters

of

the

new

adjustment

This

policy

leaves

the

actual

choice

of

resource

allocation

parameters

to

the

application

and

can

therefore

be

used

by

those

applications

which

can

not

be

satisfied

by

any

of

the

other

policies

This

policy

is

particularly

useful

for

real-time

applications

such

as

feedback

control

where

controllers

may

only

be

defined

in

the

application

for

some

specific

values

of

the

The

chosen

quality

adjustment

policy

for

an

application

is

applied

when

the

QOS

server

decides

to

alter

its

resource

allocation

Such

decisions

are

made

when

the

server

exercises

its

admission

control

policy

and

dynamic

quality

control

policy

These

are

discussed

in

subsequent

sections

In

addition

to

the

above

quality

adjustment

policies

a

quality

adjustment

priority

associated

with

each

reserve

specifies

the

global

priority

at

which

the

application

's

quality

will

be

adjusted

This

means

that

if

adjustments

to

reservations

allocated

to

applications

were

to

be

made

by

the

QOS

server

an

application

with

a

lower

adjustment

priority

would

be

upgraded

after

-LRB-

and

downgraded

before

-RRB-

an

application

with

a

higher

adjustment

priority

An

application

with

reserves

can

submit

a

dynamic

on-line

request

to

change

its

prior

status

requesting

a

higher

allocation

and/or

changing

its

quality

adjustment

policy

The

request

is

not

guaranteed

to

be

honored

however

This

allows

a

QOS

client

to

change

its

behavior

dynamically

based

on

user

input

-LRB-

or

changing

internal

application

requirements

-RRB-

feedback

control

application

for

example

can

use

a

PD

controller

at

some

pre-defined

high

frequencies

a

PID

controller

at

some

pre-defined

medium

frequencies

and

a

fuzzy/adaptive

controller

at

relatively

low

frequencies

Admission

Control

Policies

The

QOS

server

tries

to

provide

the

maximum

reservation

available

in

the

system

based

on

the

application

requirement

and

the

system

load

at

the

time

of

request

The

reservation

parameters

of

existing

QOS

clients

may

be

modified

if

necessary

to

admit

a

new

client

An

admission

control

policy

allows

the

QOS

server

to

decide

whether

to

accept

a

new

request

and

at

what

level

of

quality

Six

different

admission

control

policies

are

available

New-Minimum

Existing-Minimum

Both

incoming

and

current

applications

can

be

pushed

down

to

their

minimum

acceptable

quality

levels

New-Minimum

Existing-Desired

The

incoming

application

can

be

pushed

down

to

its

minimum

quality

level

but

current

applications

must

not

be

pushed

farther

below

their

desired

levels

New-Minimum

Existing-Actual

The

incoming

application

can

be

pushed

down

to

its

minimum

quality

level

but

current

applications

must

not

be

pushed

farther

below

their

current

allocation

levels

The

remaining

policies

New-Desired

Existing-Minimum

New-Desired

Existing-Desired

and

New-Desired

Existing-Actual

are

counterparts

to

the

above

but

the

incoming

request

must

be

accepted

at

its

desired

quality

level

Dynamic

Quality

Control

Policies

The

QOS

server

polls

the

kernel

at

periodic

intervals

-LRB-

currently

every

seconds

-RRB-

to

determine

how

much

of

each

guarateed

reservation

is

being

under-used

-LRB-

signaling

an

under-run

-RRB-

and

how

many

threads

have

their

usage

exceeding

their

reservations

-LRB-

signaling

an

over-run

which

is

executed

in

background

mode

and

charged

to

a

default

reserve

in

the

current

version

of

RT-Mach

-RRB-

Based

on

this

information

the

QOS

server

may

decide

and

notify

clients

that

their

reservations

are

being

reduced

or

increased

so

that

they

can

adapt

their

behavior

One

of

four

different

overrun

control

policies

can

be

chosen

These

policies

are

similar

in

nature

to

the

admission

control

policies

discussed

above

The

QOS

Server

Threads

As

illustrated

in

Figure

the

QOS

server

implementation

consists

of

threads

The

admission

control

thread

determines

if

new

QOS

requests

can

be

granted

The

dynamic

quality

control

thread

is

the

one

which

periodically

checks

the

kernel

for

status

information

regarding

the

reserve

usage

of

various

registered

applications

The

display

thread

is

an

X-client

which

graphically

depicts

the

granted

reservations

to

QOS

clients

the

actual

usage

of

reservations

the

quality

adjustment

policy

for

each

QOS

client

and

the

admission

and

overrun

policies

of

the

QOS

server

itself

The

Client

Interface

to

the

QOS

Server

QOS

clients

access

the

QOS

server

using

library

calls

to

register/unregister

their

presence

as

well

as

to

request

and

alter

their

quality

attributes

The

QOS

server

notifies

each

QOS

client

of

any

changes

in

resource

allocation

using

a

communication

port

registered

specified

by

the

application

at

the

time

of

registration

The

RT-Phone

Teleconferencing

Application

In

order

to

study

the

necessity

sufficiency

power

and

performance

of

the

reservation

and

QOS

mechanisms

we

implemented

a

distributed

teleconferencing

application

across

the

network

We

describe

this

application

named

RT-Phone

below

followed

by

a

description

of

the

application

architecture

Our

QOS

experiments

with

the

application

will

be

described

in

the

next

section

Figure

The

RT-Phone

user

interface

on

starting

up

RT-Phone

RT-Phone

presents

a

telephone-pad-like

Motif-based

graphical

interface

running

on

top

of

RT-Mach

The

initial

state

when

RT-Phone

starts

is

presented

in

Figure

A

caller

initiates

communication

with

a

remote

workstation

running

RT-Phone

by

first

clicking

on

the

handset

as

illustrated

in

Figure

The

dialer

corresponding

to

Hostname

and

other

speed-dial

buttons

becomes

enabled

when

the

handset

is

off

the

hook

The

caller

then

specifies

a

destination

host

-LRB-

using

a

speed

button

or

by

specifying

a

specific

remote

hostname

in

a

dialog

box

-RRB-

and

a

connection

request

is

sent

to

the

remote

machine

This

connection

request

is

displayed

on

a

caller-id

window

as

shown

on

the

top

right

window

in

Figure

The

callee

completes

establishment

of

the

two-way

connection

by

clicking

on

his/her

handset

button

At

this

point

a

audio

conversation

can

take

place

on

the

network

A

real-time

video

stream

can

also

be

transmitted

when

the

Figure

The

RT-Phone

user

interface

in

the

off-the-hook/ready-to-dial

state

Figure

The

RT-Phone

user

interface

after

a

network

phone

connection

is

established

connection

gets

established

but

in

this

paper

we

shall

confine

our

discussion

to

the

duplex

audio

streams

only

Controls

are

available

for

the

user

to

set

the

volume

level

of

the

speaker

and/or

the

microphone

to

block

incoming

connection

requests

from

specific

hosts

mute

microphone

input

to

clear

the

caller-id

field

and

to

modify

the

quality

of

the

audio

streams

transmitted

-LRB-

this

will

be

discussed

in

more

detail

in

Section

-RRB-

These

features

are

generally

only

a

subset

of

more

sophisticated

tools

such

as

the

EtherPhone

system

-LSB-

-RSB-

and

were

custom-designed

only

to

exercise

relevant

portions

of

our

QOS

and

reservation

mechanisms

The

Architecture

of

the

RT-Phone

Application

DisplayInterfaceConnectionControlAudioSend

AudioReceiveAudioServer

Real-Time

MachAudio

device

driver

HardwareP

AS

PAS

network

From

network

to/from

networkRT-PhoneQOSServer

OtherClients

Figure

The

Architecture

of

RT-Phone

at

each

end-point

The

architecture

of

RT-Phone

at

each

node

is

presented

in

Figure

It

is

in

many

ways

similar

to

traditional

teleconferencing

applications

-LRB-

for

example

PictureTel

cu/c-me

on

the

Mac

-LSB-

-RSB-

-RRB-

but

it

is

also

quite

different

in

many

other

ways

For

example

a

distinct

user-level

audio

server

process

hides

the

details

of

the

system

's

sound

card

by

providing

a

higher

level

Processor

reserves

provide

a

mutual

temporal

barrier

between

RT-Phone

and

other

applications

In

addition

the

same

processor

reserve

is

applied

across

task

boundaries

Finally

we

allow

the

quality

of

the

audio

streams

to

be

modified

dynamically

This

application

runs

on

Real-Time

Mach

on

two

processors

networked

using

Ethernet

The

dark

directional

arrows

of

Figure

represent

the

data

flow

in

the

application

A

User-Level

Audio

Server

-LRB-

ULAS

-RRB-

-LSB-

-RSB-

provides

a

high-level

programming

interface

to

audio

cards

by

hiding

the

details

and

device

driver

calls

for

the

specific

sound

card

in

use

For

example

in

RT-Phone

we

use

SoundBlaster-compatible

Pro-Audio

Spectrum

-LRB-

PAS

-RRB-

audio

cards

This

ULAS

interface

allows

the

choice

of

an

audio

sampling

rate

the

choice

between

samples

and

samples

and

the

choice

of

a

buffer

size

which

is

used

to

store

audio

samples

until

ready

for

input

and

output

Three

threads

in

the

audio-server

deal

with

control

of

these

configuration

parameters

audio

data

input

from

the

microphone

and

audio

data

output

to

the

speaker

respectively

Although

the

PAS

card

supports

bi-directional

audio

-LRB-

cap

AudioFile

utility

supports

a

similar

server

as

well

ture

and

output

-RRB-

it

can

not

do

both

simultaneously

-LRB-

in

duplex

mode

-RRB-

Hence

the

user-level

audio-servers

can

only

do

either

audio

input

or

output

at

any

given

time

We

therefore

use

two

PAS

cards

and

two

audio

servers

one

for

continuous

audio

sampling

and

another

for

continuous

audio

output

respectively

A

periodic

interrupt

based

on

the

sampling

rate

and

buffer

size

triggers

the

input

audio

server

to

copy

the

buffer

into

its

requesting

client

an

audio-send

thread

within

RT-Phone

A

shared

memory

buffer

is

used

between

the

audio

server

and

the

RT-Phone

thread

The

audio-send

thread

then

transmits

the

data

to

the

remote

callee

across

the

network

An

audio-receive

thread

receives

the

audio

data

and

outputs

it

to

its

speaker

through

the

use

of

the

output

A

mirror

image

of

the

same

streams

transmits

audio

from

the

callee

to

the

caller

Reserves

QOS

and

End-to-End

Delays

in

RT-Phone

In

this

section

we

derive

our

predicted

end-to-end

delay

of

the

RT-Phone

application

described

in

the

previous

section

We

also

describe

the

application

of

the

reserve

and

dynamic

quality

schemes

to

RT-Phone

Finally

we

provide

performance

measurements

of

RT-Phone

which

show

that

we

indeed

satisfy

the

predicted

end-to-end

delays

The

Quality

Parameters

of

RT-Phone

There

are

five

quality

parameters

associated

with

RT-Phone

end-to-end

delay

jitter

audio

sampling

rate

audio

sample

size

and

audio

drop

rate

In

this

paper

we

focus

on

making

the

audio

drop

rate

being

by

allocating

and

guaranteeing

necessary

resources

and

satisfying

an

acceptable

end-to-end

delay

requirement

for

different

values

of

audio

sampling

rate

sample

size

and

the

value

of

T

in

the

pipeline

stage

Based

on

requirements

used

in

the

telephony

domain

we

require

that

all

end-to-end

delays

be

less

than

ms.

The

Guaranteed

End-To-End

Delay

Stage

Stage

Stage

Stage

Stage

Processor

T

T

TNetworkDelay

Figure

The

Audio

Stream

Pipeline

Stages

in

the

End-To-End

Delay

The

pipeline

stages

involved

in

the

audio

transmit

path

of

Figure

are

illustrated

in

Figure

It

consists

of

stages

Stage

represents

the

filling

of

its

DMA

buffer

by

the

control

is

a

critical

issue

to

be

dealt

with

in

this

context

but

is

beyond

the

scope

of

this

paper

-LRB-

PAS

-RRB-

sound

card

This

delay

is

determined

by

the

parameters

chosen

for

the

sampling

rate

sample

size

and

buffer

size

on

the

sound

card

Stage

represents

the

processing

of

the

data

-LRB-

receive

audio

data

from

server

and

protocol

processing

for

transmit

-RRB-

Stage

represents

the

network

propagation

delay

Stage

represents

the

processing

on

the

receiver

side

-LRB-

protocol

processing

for

receive

and

send

to

PAS

card

through

the

audio

server

-RRB-

Finally

stage

is

the

actual

playing

back

of

the

audio

samples

by

the

hardware

If

T

represents

the

hardware

buffering

delay

for

stage

-LRB-

and

the

same

for

stage

-RRB-

we

let

stages

and

have

the

worst-case

timing

constraint

of

T

This

is

done

by

assigning

reserves

to

these

processor

with

the

reserve

period

T

such

that

the

kernel

can

guarantee

that

the

processing

in

these

two

stages

is

completed

within

T

units

The

end-toend

delay

for

an

audio

sample

is

therefore

given

by

worst-case

end-to-end

delay

network

propagation

delay

It

must

be

noted

that

the

delay

component

is

not

Consider

an

audio

sample

which

is

sampled

by

the

hardware

in

Stage

This

sample

at

offset

t

from

the

beginning

of

the

stage

will

be

played

at

the

same

offset

t

in

stage

The

number

of

T

stages

inbetween

these

two

points

is

and

not

Since

network

communications

is

not

a

focus

of

this

paper

we

use

a

dedicated

network

which

results

in

negligible

propogation

delay

e.g.

a

typical

bytes

output

every

ms

at

sampling

with

byte/sample

takes

ms

at

Mbps

to

transmit

but

take

ms

to

collect

Bundling

of

Threads

to

a

Reserve

The

audio-send

thread

is

normally

blocked

waiting

for

data

arrival

from

the

audio-server

Consider

the

flow

of

control

from

the

arrival

of

the

hardware

buffer

full

interrupt

through

the

user-level

audio-server

to

the

audio-send

thread

Logically

a

single

consecutive

action

takes

place

from

the

audio

server

which

makes

data

available

to

the

audio-send

thread

which

then

processes

it

and

transmits

it

across

the

network

The

timing

constraint

is

that

this

reception

from

hardware

and

succeeding

transmission

must

complete

by

the

interval

T

We

define

the

notion

of

a

continuous

thread

of

control

as

comprising

those

segments

of

code

where

-LRB-

a

-RRB-

the

flow

of

control

is

sequential

and

-LRB-

b

-RRB-

a

strict

precedence

constraint

exists

between

the

segments

The

former

means

that

no

two

segments

can

run

concurrently

The

latter

means

that

a

segment

can

start

only

after

and

soon

after

its

previous

segment

-LRB-

if

any

-RRB-

has

completed

These

segments

of

code

can

transcend

task

boundaries

but

not

resource

boundaries

such

as

a

CPU

Under

this

definition

a

normal

OS

thread

is

always

a

continuous

thread

of

control

since

the

flow

of

control

is

sequential

multiple

portions

of

the

thread

can

not

be

active

at

the

same

time

This

notion

of

a

continuous

thread

of

control

is

rather

useful

in

the

context

of

a

processor

reserve

which

can

be

bound

to

multiple

threads

simultaneously

as

stated

in

Section

All

OS

threads

-LRB-

or

segments

-RRB-

comprising

a

continuous

thread

of

control

typically

have

a

single

timing

constraint

from

the

first

segment

to

its

last

segment

A

single

processor

reserve

may

then

be

usefully

bound

to

all

OS

threads

comprising

the

continuous

thread

of

control

and

the

reserve

can

be

used

to

satisfy

the

timing

constraint

of

that

continuous

thread

of

control

Based

on

this

notion

the

input

thread

in

the

audio-server

and

the

audio-send

thread

are

bound

to

the

same

reserve

with

a

period

of

T.

Recall

that

T

is

determined

by

the

sound

card

configuration

parameters

sampling

rate

sample

size

and

buffer

size

Similarly

the

audio-receive

thread

and

the

output

thread

in

the

audio-server

can

be

bound

to

another

reserve

with

a

period

of

T

Our

reserve

admission

control

in

RT-Mach

uses

results

from

real-time

scheduling

theory

-LSB-

-RSB-

which

can

guarantee

the

timing

constraints

of

tasks

with

different

periods

and

computation

times

Hence

the

reserve

period

for

one

direction

need

not

be

the

same

as

the

reserve

period

for

the

opposite

direction

In

other

words

the

quality

of

the

two

output

streams

at

the

two

speaker

devices

can

be

completely

independent

of

one

another

Performance

Measurements

We

measured

the

end-to-end

delays

actually

encountered

in

RT-Phone

to

validate

that

the

predicted

delays

are

indeed

achievable

by

the

application

of

reserves

and

the

dynamic

QOS

server

The

measurement

scheme

and

the

actual

measurements

are

provided

below

Network

RT-Mach

PCMeasuringEnd-to-End

DelayR

T-Mach

PCrunningRT-Phone

RT-Mach

PCrunningRT-Phone

Figure

Measurement

of

Audio

End-to-End

Delay

RT-Phone

was

run

between

two

machines

on

a

network

as

illustrated

in

Figure

The

end-to-end

delay

measurements

were

made

using

a

third

machine

The

measuring

machine

generated

a

sharp

sound

pulse

on

one

machine

running

RT-Phone

and

then

recorded

the

source

pulse

and

the

transmitted

output

from

the

destination

machine

running

RT-Phone

By

measuring

the

timestamp

of

the

generated

sound

and

the

timestamp

at

which

the

transmitted

pulse

is

heard

the

end-to-end

delay

from

the

users

's

true

reception

is

determined

to

a

very

high

degree

of

accuracy

-LRB-

within

a

few

ms

-RRB-

Note

that

the

audio

stream

in

the

direction

opposite

to

the

direction

being

measured

is

active

while

the

measurement

is

being

done

but

this

is

not

illustrated

in

the

figure

Figures

and

present

measured

performance

numbers

from

RT-Phone

running

on

Real-Time

Mach

Figure

presents

the

variation

of

the

end-to-end

delays

between

the

two

conversation

endpoints

as

the

reservation

period

used

for

processing

the

audio

samples

is

varied

This

is

repeated

for

various

audio

sampling

rates

Figure

presents

the

change

in

the

CPU

load

-LRB-

on

PCs

-RRB-

as

the

reservation

period

and

the

sampling

frequencies

are

varied

As

can

be

expected

when

the

reservation

period

increases

the

end-to-end

delay

increases

proportionally

It

can

also

be

seen

that

the

worst-case

end-to-end

delay

bound

of

-LRB-

Reservation

Period

-RRB-

is

satisfied

In

fact

the

plotted

numbers

represent

the

best-case

end-to-end

delay

given

by

the

sum

of

The

delay

to

fill

up

the

DMA

buffer

at

the

audio

source

The

processing

time

of

the

audio

data

buffer

by

the

input

audio

server

and

the

audio

send

thread

The

network

propagation

time

The

processing

time

of

the

received

audio

data

by

the

audio

receive

thread

and

the

output

audio

receiver

and

in

addition

any

preemption

time

from

the

audio

stream

for

the

opposite

direction

End-to-End

Delay

-LRB-

ms

-RRB-

Period

-LRB-

ms

-RRB-

End-to-End

Delay

as

Function

of

Period

kHz

sampling

kHz

sampling

kHz

sampling

kHz

sampling

kHz

sampling

Figure

The

End-To-End

Delay

w

varying

reservation

periods

and

audio

sampling

rates

As

the

audio

sampling

rate

increases

the

processing

times

increase

slightly

resulting

in

a

corresponding

increase

in

the

end-to-end

delay

However

the

most

significant

overhead

CPU

Load

-LRB-

-RRB-

Period

-LRB-

ms

-RRB-

CPU

Load

kHz

sampling

kHz

sampling

kHz

sampling

kHz

sampling

kHz

sampling

Figure

The

CPU

Load

with

different

reservation

periods

and

audio

sampling

rates

for

threads

with

small

periods

is

packet

sending

and

receiving

overhead

In

addition

for

small

reservation

periods

timers

are

set

and

reset

frequently

resulting

in

higher

overhead

Finally

context-switching

overhead

increases

with

sampling

rate

One

can

draw

the

following

conclusions

from

the

performance

numbers

First

for

a

given

reservation

period

in

use

the

audio

sampling

rate

has

very

little

impact

on

both

the

end-to-end

delay

and

the

CPU

load

For

example

when

the

reservation

period

used

is

ms

the

end-to-end

delay

is

ms

and

ms

for

KHz

sampling

and

KHz

sampling

respectively

and

the

total

CPU

load

increases

from

to

This

testifies

to

the

fact

that

data

processing

delays

-LRB-

including

communication

packet

processing

-RRB-

dominate

data

sizes

Secondly

the

CPU

load

increases

non-linearly

with

shorter

reservation

periods

as

seen

in

Figure

indicating

that

the

much

shorter

end-to-end

delays

-LRB-

below

the

knee

of

the

curve

around

ms

-RRB-

are

obtained

at

a

disproportionately

higher

cost

One

would

like

to

be

towards

the

left-end

of

Figure

and

the

right-end

of

Figure

A

reasonable

compromise

for

the

current

hardware

configuration

lies

around

a

reservation

period

of

ms.

QOS

Interface

for

Changing

Quality

Parameters

Dynamically

RT-Phone

also

has

a

user-interface

to

change

the

quality

of

the

received

audio

stream

-LRB-

from

the

remote

participant

-RRB-

and

is

illustrated

in

Figure

When

such

a

quality

change

is

requested

the

QOS

manager

of

both

the

local

and

remote

nodes

are

contacted

to

modify

the

reservations

they

need

for

the

new

quality

setting

If

granted

the

audio-server

is

requested

to

change

to

the

new

settings

The

sender

must

upgrade

quality

only

after

the

receiver

upgrades

its

reservation

-LRB-

and

be

ready

for

the

added

incoming

traffic

-RRB-

The

Figure

The

interface

to

set

system

quality

parameters

QOS

ServerA

QOS

ServerB

Processor

A

Processor

B

NetPhoneA

NetPhoneB

Figure

The

Multi-Phase

Protocol

to

dynamically

adjust

QOS

during

teleconferencing

actual

quality

change

must

be

effected

by

a

multi-phase

protocol

illustrated

in

Figure

Similarly

if

the

sender

degrades

quality

the

receiver

must

downgrade

its

reservation

only

after

the

sender

does

Many

subtleties

underlie

such

dynamic

transitions

of

network

traffic

and

processing

requirements

For

example

it

is

possible

that

the

old

resource

allocation

and

the

new

allocation

are

completely

schedulable

-LRB-

i.e.

all

their

timing

constraints

can

be

satisfied

with

these

resources

-RRB-

independently

but

timing

constraints

can

be

violated

during

the

transition

-LSB-

-RSB-

While

this

may

not

be

a

major

concern

in

non-critical

multimedia

applications

this

loss

of

schedulability

must

not

lead

to

all

subsequent

deadlines

to

be

missed

or

synchronicity

to

be

last

-LRB-

say

between

independently

arriving

audio

and

video

streams

-RRB-

Of

critical

importance

are

the

times

at

which

changes

to

reservation

come

into

effect

the

times

at

which

changes

to

audio

card

set

tings

come

into

effect

and

the

need

for

continued

synchronization

between

the

sender

and

receiver

Lessons

Learned

and

Future

Work

We

have

learnt

a

number

of

lessons

based

on

our

experiences

with

the

RT-Phone

application

in

the

context

of

support

for

processor

reservation

and

dynamic

QOS

control

in

RT-Mach

The

current

processor

reserve

mechanism

seems

to

provide

a

sufficiently

general

meanss

that

can

support

both

dynamic

real-time

and

multimedia

applications

However

the

programming

abstraction

may

have

to

be

raised

with

middleware

libraries

which

map

applicationlevel

QOS

parameters

-LRB-

such

as

sampling

rate

-RRB-

to

kernellevel/QOS-server-level

parameters

-LRB-

such

as

computation

time

and

period

-RRB-

The

reserve

enforcement

already

yields

a

built-in

framework

for

measuring

computation

time

-LRB-

which

is

dependent

on

the

actual

hardware

being

used

-RRB-

The

period

must

be

determined

based

on

the

end-to-end

delays

of

the

application

More

precise

control

over

when

a

reservation

change

will

be

effected

would

be

very

desirable

In

addition

mechanisms

to

synchronize

the

start

times

of

reserves

which

are

active

in

different

processors

may

be

critical

for

networked

applications

particularly

when

network

load

can

be

variant

Our

future

work

includes

extending

the

reserve

abstraction

to

address

the

above

requirements

and

to

other

resources

such

as

filesystems

A

more

general

set

of

protocols

is

needed

to

coordinate

changes

in

reserves

across

processors

for

networked

applications

We

are

currently

looking

at

an

audio-mixing

server

to

combine

multiple

incoming

audio

streams

to

a

single

output

device

and

its

complexity

in

terms

of

reservation

requirements

is

much

higher

compared

to

an

audio-server

with

a

single

client

In

addition

we

are

yet

to

test

our

QOS

framework

with

multiple

clients

each

with

a

different

kind

of

dynamic

quality

change

requirements

Acknowledgements

The

authors

would

like

to

thank

Bryan

Hixon

Manish

Modh

and

William

Nagy

for

their

first

prototype

of

the

QOS

server

on

RT-Mach

References

S.

R.

Ahuja

J.

R.

Ensor

and

D.

N.

Horn

The

Rapport

Multimedia

Conferencing

System

Proceedings

of

the

Conference

on

Office

Information

Systems

March

pp.

Rajkumar

R.

The

Advanced

Real-time

Monitor

User

Manual

Real-Time

Mach

Project

Carnegie

Mellon

University

-LRB-

November

-RRB-

A.

Bricker

M.

Gien

M.

Guilllemont

J.

Lipskis

D.

Orr

and

M.

Rozier

Architectural

Issues

in

Microkernel-base

Operating

Systems

the

CHORUS

Experience

Computer

Communication

-LRB-

July

-RRB-

D.

Ferrari

and

D.

Verma

A

Scheme

for

Real-Time

Channel

Establishment

in

Wide

Area

Networks

-LRB-

April

-RRB-

H.

Gajewska

J.

Kistler

M.

S.

Manasse

and

D.

D.

Redell

Argo

A

System

for

Distributed

Collaboration

Proceedings

of

the

Second

ACM

International

Conference

on

Multimedia

Oct.

pp.

K.

Jeffay

D.

L.

Stone

and

F.

D.

Smith

Kernel

Support

for

Live

Digital

Audio

and

Video

-LRB-

Nov.

-RRB-

J.

Kurose

Open

Issues

and

challenges

in

Providing

Quality

of

Service

Guarantees

in

High-Speed

Networks

ACM

Computer

Comm

Review

-LRB-

January

-RRB-

Liu

C.

L.

and

Layland

J.

W.

Scheduling

Algorithms

for

Multiprogramming

in

a

Hard

Real

Time

Environment

JACM

-LRB-

-RRB-

-LRB-

-RRB-

Mercer

C.

W.

Savage

S.

and

Tokuda

H.

Processor

Capacity

Reserves

OS

Support

for

Multimedia

Applications

IEEE

International

Conference

on

Multimedia

Computing

Systems

-LRB-

May

-RRB-

C.

W.

Mercer

and

R.

Rajkumar

Design

and

Evaluation

of

a

Real-Time

X

Server

Tech

Rept

ART

Project

Carnegie

Mellon

University

Oct.

K.

Nahrstedt

and

J.

Smith

The

QOS

Broker

IEEE

Multimedia

Spring

-LRB-

-RRB-

Nakajima

T.

Kitayama

T.

Arakawa

H.

and

Tokuda

H.

Integrated

Management

of

Priority

Inversion

in

RT

Mach

to

appear

in

IEEE

Real-Time

Systems

Symposium

-LRB-

December

-RRB-

Zelenka

J.

The

Audio

Server

User

Manual

Real-Time

Mach

Project

Carnegie

Mellon

University

-LRB-

November

-RRB-

Adobe

Premiere

for

Macintosh

Rajkumar

R.

Synchronization

in

Real-Time

Systems

A

Priority

Inheritance

Approach

Kluwer

Academic

Publishers

ISBN

P.

Robin

G.

Coulson

A.

Campbell

G.

Blair

and

M.

Papathomas

Implementing

a

OoS

Controlled

ATM

Based

Communications

System

in

Chorus

Technical

Report

Dept.

of

Computing

Lancaster

University

March

Sha

L.

Rajkumar

R.

Lehoczky

J.P.

Ramamritham

K.

Mode

Changes

in

a

Prioritized

Preemptive

Scheduling

Environment

The

Real-Time

Systems

Journal

-LRB-

December

-RRB-

D.

B.

Terry

and

D.

C.

Swinehart

Managing

Stored

Voice

in

the

Etherphone

System

ACM

Transactions

on

Computer

Systems

-LRB-

Feb.

-RRB-

Also

available

in

Xerox

PARC

report

May

Tokuda

H

The

Real-Time

Mach

Operating

System

Carnegie

Mellon

University

Pittsburgh

PA

September

H.

M.

Vin

and

P.

T.

Zellweger

and

D.

C.

Swinehart

and

P.

V.

Rangan

Multimedia

Conferencing

in

the

Etherphone

Environment

IEEE

Computer

-LRB-

Oct.

-RRB-

A.

Vogel

B.

Kerherve

G.

Bochmann

and

J.

Gecsei

Distributed

Multimedia

and

QOS

A

Survey

IEEE

Multimedia

-LRB-

-RRB-

L.

Zhang

S.

Deering

D.

Estrin

S.

Shenker

and

D

Zappala

RSVP

A

New

Resource

Reservation

Protocol

IEEE

Network

-LRB-

September

-RRB-

Table

of

Contents

Introduction

Processor

Reserves

in

Real-Time

Mach

Related

Work

Organization

of

the

Paper

Dynamic

Quality

of

Service

Control

Quality

Management

and

its

Implications

to

Systems

Support

The

QOS

Server

QOS

Attributes

Quality

Adjustment

Admission

Control

Policies

Dynamic

Quality

Control

Policies

The

QOS

Server

Threads

The

Client

Interface

to

the

QOS

Server

The

RT-Phone

Teleconferencing

Application

RT-Phone

The

Architecture

of

the

RT-Phone

Application

Reserves

QOS

and

End-to-End

Delays

in

RT-Phone

The

Quality

Parameters

of

RT-Phone

The

Guaranteed

End-To-End

Delay

Bundling

of

Threads

to

a

Reserve

Performance

Measurements

QOS

Interface

for

Changing

Quality

Parameters

Dynamically

Lessons

Learned

and

Future

Work

Acknowledgements

References

List

of

Figures

Figure

The

Architecture

of

the

Dynamic

QOS

Server

Figure

The

RT-Phone

user

interface

on

starting

up

Figure

The

RT-Phone

user

interface

in

the

off-the-hook/ready-to-dial

state

Figure

The

RT-Phone

user

interface

after

a

network

phone

connection

is

established

Figure

The

Architecture

of

RT-Phone

at

each

end-point

Figure

The

Audio

Stream

Pipeline

Stages

in

the

End-To-End

Delay

Figure

Measurement

of

Audio

End-to-End

Delay

Figure

The

End-To-End

Delay

w

varying

reservation

periods

and

audio

sampling

rates

Figure

The

CPU

Load

with

different

reservation

periods

and

audio

sampling

rates

Figure

The

interface

to

set

system

quality

parameters

Figure

The

Multi-Phase

Protocol

to

dynamically

adjust

QOS

during

teleconferencing

O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

audio conferencing
end-to-end latencies
real-time mach
reserve bundling


Pruning and Grouping Discovered Association Rules H Toivonen M Klemettinen P Ronkainen K H?at?onen H Mannila Department of Computer Science , P.O. Box 26 FIN-00014 University of Helsinki , Finland email : Hannu.Toivonen@Helsinki.FI Abstract Association rules are statements of the form `` for 90 % of the rows of the relation , if the row has value 1 in the columns in set X , then it has 1 also in the columns in set Y '' . Efficient methods exist for discovering association rules from large collections of data . The number of discovered rules can , however , be so large that the rules can not be presented to the user . We show how the set of rules can be pruned by forming rule covers . A rule cover is a subset of the original set of rules such that for each row in the relation there is an applicable rule in the cover if and only if there is an applicable rule in the original set . We also discuss grouping of association rules by clustering , and present some experimental results of both pruning and grouping . 1 Introduction Association rules are an interesting class of database regularities , introduced by Agrawal , Imielinski , and Swami -LSB- AIS93 -RSB- . An association rule is an expression X -RRB- Y , where X and Y are sets of attributes . The intuitive meaning of such a rule is that in the rows of the database where the attributes in X have value true , also the attributes in Y tend to have value true . Efficient methods exist for the discovery of association rules -LSB- MTV94 , AS94 -RSB- . Paradoxically , data mining itself can produce such great amounts of data that there is a new knowledge management problem : there can easily be thousands or even more association rules holding in a data set . For instance , we discovered 2000 rules from a course enrollment database , and almost 30000 rules from a telecommunications network alarm database . In this paper we discuss pruning and grouping of association rules , in order to improve the understandability of the collection of discovered association rules . The rest of this paper is organized as follows . Association rules and there properties are discussed in Section 2 . In Section 3 we present how rule sets can be pruned by forming rule covers . Grouping of association rules is discussed in Section 4 , while Section 5 is a conclusion . 2 Association rules and their properties Let R = fI1 ; I2 ; : : : ; Img be a set of attributes , also called items , over the binary domain f0 ; 1g . The input r = ft1 ; : : : ; tng for the data mining method is a relation over the relation schema fI1 ; I2 ; : : : ; Img , i.e. , a set of binary vectors of size m. Each row can be considered as a set of properties or items -LRB- that is , t -LSB- i -RSB- = 1 , Ii 2 t -RRB- . Let X ? R be a set of attributes and t 2 r a row of the relation . If t -LSB- A -RSB- = 1 for all A 2 X , we write t -LSB- X -RSB- = ? 1 . The set of rows matched by X is m -LRB- X -RRB- = ft 2 r j t -LSB- X -RSB- = ? 1g . An association rule over r is an expression X -RRB- Y , where X ? R and Y ? R n X . Given real numbers -LRB- confidence threshold -RRB- and oe -LRB- support threshold -RRB- , we say that r satisfies X -RRB- Y with respect to and oe , if jm -LRB- XY -RRB- j > = oen and jm -LRB- XY -RRB- j jm -LRB- X -RRB- j > = : That is , at least a fraction oe of the rows of r have 1 's in all attributes of X and Y , and at least a fraction of the rows having a 1 in all attributes of X also have a 1 in all attributes of Y . Given a set of attributes X , we say that X is large -LRB- with respect to the database and the given support threshold oe -RRB- , if jm -LRB- X -RRB- j > = oen : That is , at least a fraction oe of the rows in the relation have 1 's in all the attributes of X . Example 1 Our case study is an enrollment database of courses in computer science . There is a row per student , containing the courses the student has registered for . The discovered association rule Distributed Operating Systems , Introduction to Unix -RRB- Data Communications -LRB- 0.96 , 0.02 -RRB- states that 96 % of the students that have taken courses Distributed Operating Systems and Introduction to Unix , also have taken the course Data Communications , and that 2 % of all the students actually have taken all three courses . 2 Given support threshold oe and confidence threshold , the set of all association rules that hold in a database can be computed efficiently -LSB- MTV94 , AS94 -RSB- . Additionally , one can obtain good approximations to the collection of association rules by using reasonably small samples -LSB- MTV94 -RSB- . The data sets of our case study is representative of reasonable sample size . One of the fundamental problems in data mining is to know what is useful to the user . The thresholds and oe ensure that the discovered rules have enough positive evidence . However , a given relation may satisfy a large number of such association rules . To be useful , a data mining system must manage the large amount of generated information by offering facilities for further rule pruning . Example 2 The course enrollment database consists of registration information of 1066 students who had registered for at least two courses . On average , a row has 5 courses . The total number of courses is 112 . In the data set we discovered 2010 association rules with support threshold oe = :01 -LRB- corresponding to 11 students -RRB- and with no confidence threshold . The rule set contains 420 rules with a confidence of 0.7 or more , and 99 rules with a confidence of 0.9 or more . Raising the support threshold has a more dramatic effect : of those 99 rules with a confidence of 0.9 or more , only 4 rules have a support of at least 0.1 . With support threshold oe = :005 , there are 6715 rules , 592 of which have confidence of 0.9 or more , and 459 of which have confidence of exactly 1 . Thus , raising the confidence threshold to a high value prunes the set considerably , but many rules still may remain . 2 Not all of the rules with high confidence and support are interesting . Some of these rules can correspond to prior knowledge or expectations , refer to uninteresting attributes or attribute combinations , or they can present redundant information . Prior knowledge and expectations are background information that only the user can provide , and so are of course any special interests of the user . One way of specifying such information is to provide structural information about the rule expressions that are interesting . With templates , for instance , the user can explicitly specify both what is interesting and what is not -LSB- KMR +94 -RSB- . In the rest of this paper we present a way for user and domain independent pruning of redundant rules , and discuss grouping of rules . 3 Association rule covers In this section we present rule covers , a domain independent method of reducing the number of rules by elimination of redundancy . Consider a collection ? of rules with the same attribute set Y as the consequent : ? = fXi -RRB- Y j i = 1 ; : : : ; ng : Every rule in ? gives a description of the presence of the attributes Y . Discovered rule sets often are heavily redundant , i.e. several rules describe the same database rows . Given a rule set ? , its subset ? ? ? is a rule cover1 , if -LSB- -LRB- X -RRB- Y -RRB- 2 ? m -LRB- XY -RRB- = -LSB- -LRB- X -RRB- Y -RRB- 2 ? m -LRB- XY -RRB- : I.e. , the rules in a rule cover ? describe Y in all the cases in the database that the original rule set ? does . It is useful to constrain the rule sets and their covers to contain rules with high confidence only : for justification and prediction of the consequent Y , mostly rules of a high confidence are of interest . If one wants to be sure that the pruning methods do not lose any information , one has to assume the following . The data set should be monotonic in the sense that if there is a matching rule for Y with a high confidence , there should not exist a more special rule with a lower confidence . This is to ensure that we do not miss any contradictory information by limiting ourselves to the rules with high confidence only . These restrictions seem to hold in many real world data sets . 3.1 Structural rule cover We first describe a conservative but fast way of pruning , based on the structure of the rules . The method is based on the observation that for all attribute sets X , Y , and Z we have m -LRB- XY Z -RRB- ? m -LRB- XZ -RRB- ; i.e. , a rule X -RRB- Z matches a superset of the database rows matched by the rule XY -RRB- Z . If such rules XY -RRB- Z are removed from a rule cover , the remaining set is also a rule cover . A set of rules ? ? ? is a structural rule cover for ? , if for all rules -LRB- X -RRB- Y -RRB- 2 ? there is no rule -LRB- X -RRB- Y -RRB- 2 ? such that X ae X . A structural cover thus contains the most general rules of the original rule set . Clearly , a structural cover is a cover . Note that a structural cover can be computed completely without looking at the rows of the relation . Example 3 The latter of the following rules is more special than the first one , and has no additional predictive power over it : 1The usage of the term cover is borrowed from database theory , where it is extensively used -LRB- see -LSB- Ull88 -RSB- -RRB- ; there is also a close similarity to the set cover problem . Similar notions have also been used in the machine learning literature -LSB- MMHL86 , CN89 -RSB- . Programming in C , Object Data Bases -RRB- Data Communications -LRB- 0.90 , 0.02 -RRB- Programming in C , Object Data Bases , Computer-Supported Cooperative Work -RRB- Data Communications -LRB- 0.90 , 0.01 -RRB- The second rule is pruned from the structural rule cover as redundant . 2 Structural covers are an efficient way of pruning rule sets . Next we describe an algorithm that finds more optimal rule covers . The algorithm can be used to improve structural covers . 3.2 Rule cover algorithm A close to optimal rule cover can be found with the following greedy algorithm . Algorithm RuleCover Input : Set of rules ? = fXi -RRB- Y j i = 1 ; : : : ; ng : Sets of matched rows m -LRB- XiY -RRB- for all i 2 f1 ; : : : ; ng . Output : Rule cover ? . Method : ? : = ; ; / / rule cover s0 : = Sni = 1 m -LRB- XiY -RRB- ; / / rows unmatched by cover for all i 2 f1 ; : : : ; ng do si = m -LRB- XiY -RRB- ; / / rows of s0 matched by rule i end ; while s0 < > ; do choose i 2 f1 ; : : : ; ng so that -LRB- Xi -RRB- Y -RRB- 2 ? and jsij is largest ; ? : = ? -LSB- fXi -RRB- Y g ; / / add the rule to the cover ? : = ? n -LRB- Xi -RRB- Y -RRB- ; / / remove the rule from the original set for all -LRB- Xj -RRB- Y -RRB- 2 ? do sj = sj n m -LRB- XiY -RRB- ; / / remove matched rows end ; s0 : = s0 n m -LRB- XiY -RRB- ; / / remove matched rows end ; This algorithm gets as input the original rule set ? and the sets of rows matched by each of these rules . Rule cover ? is initialized to an empty set . Variable s0 is used to store those database rows that are not matched by rules in ? ; the sets si contain those rows in s0 that are matched by the rule Xi -RRB- Y . Iteratively , the rule in ? that matches most of the rows in s0 is moved from the rule set ? to the rule cover . The rows matched by this rule are removed from s0 . This is repeated until all the rows matched by the original rule set are matched by the rule cover , i.e. , until the set s0 is empty . This greedy algorithm presumes that the rule cover ? must match exactly the same rows as the original rule set ? . It can be useful to loosen this restriction by allowing that some small portion '' of the rows is not matched by the rule cover ? . The next lemmas follow immediately from the properties of the greedy heuristics for set cover problems -LRB- see , e.g. , -LSB- CLR90 -RSB- -RRB- . Lemma 1 Algorithm RuleCover works correctly . Lemma 2 Denote by opt the size of the smallest cover for the matched database rows m -LRB- XiY -RRB- . Then for the size of the output ? of Algorithm RuleCover we have j?j < = log -LRB- j -LSB- i2f1 ; : : : ; ng m -LRB- XiY -RRB- j -RRB- ? opt ; i.e. , the output ? contains at most a logarithmic factor of extra rules . Lemma 3 The time complexity of Algorithm RuleCover is polynomial with respect to j -LSB- i m -LRB- XiY -RRB- j . The covers computed by the above algorithm seem to produce useful short descriptions of large sets of rules . Example 4 A real telecommunications alarm database induces 1461 rules with the same attribute as a consequent and with a confidence of at least 0.9 -LRB- and with support of at least 2 % -RRB- . The structural cover of the subset consists of 20 rules , which is 98 % improvement . A further rule cover computed by Algorithm RuleCover from the 20 rules only contains 5 rules . 2 4 Grouping rules Given an attribute set Y , the cover for the set of rules of the form X -RRB- Y can still be quite large . The set of rules in the cover can be made more understandable by ordering and grouping the rules . Rules can be ordered based on their interestingness . Obvious measures of interestingness are the confidence and support factors of rules . More complex measures can also be computed to reflect the statistical significance or financial value of rules -LSB- PSM94 -RSB- . Some interestingness measures can be derived from user-specified template information . Such measures are useful for spotting the most interesting rules , but they do not help much in presenting a large collection of rules . A useful method for structuring a set of rules is clustering . In particular , we claim that grouping those rules together that make statements about the same database rows is useful , as then also the way the grouping is done describes the database . For clustering , the distance between two association rules can be defined in numerous ways . We defined the distance between two rules X -RRB- Z and Y -RRB- Z as the amount of rows where the rules differ : d -LRB- X -RRB- Z ; Y -RRB- Z -RRB- = j -LRB- m -LRB- XZ -RRB- -LSB- m -LRB- Y Z -RRB- -RRB- n m -LRB- XYZ -RRB- j = jm -LRB- XZ -RRB- j + jm -LRB- Y Z -RRB- j ? 2jm -LRB- XYZ -RRB- j : Note that the number of matching rows jm -LRB- X -RRB- j has already been computed for all large attribute sets X while forming association rules , and the distances are thus fast to compute . Example 5 We experimented with the structural cover for the course Data Communications . The rule set in the cover consists of 29 rules with confidence of at least 0.9 and with altogether 15 different attributes . We clustered the 29 rules with SAS using a nonparametric density estimation method . A clustering with 3 clusters turned out to break the rule set into clearly separate clusters . For instance , only cluster 1 contains courses VAX/VMS and Distributed Operating Systems . Students matched by rules in the cluster clearly are those specializing in operating systems . Cluster 2 , in turn , contains rules that match to the students specializing in information systems : frequent or distinctive attributes are Object Data Bases , Database Systems II , and Computer Graphics . Cluster 3 is more diverse : the distinctive courses are Artificial Intelligence , Database Systems II , and Programming in C. Studends matched by rules in this cluster are specializing in different areas . This experiment demonstrates that intuitive and \ correct '' rule clusters can be found . An interesting detail is that although the courses VAX/VMS and Distributed Operating Systems are often taken by the same students there actually is no rule in the cover with both of these courses | still rules with either of them ended in the same cluster . 2 5 Concluding remarks Association rules are a simple and natural class of database regularities , useful in various analysis and prediction tasks . The problem with this data mining technique is that the collection of all rules can be very large . We have considered the problem of pruning and grouping rules , in order to improve the understandability of the collection of discovered association rules . A rule cover is a subset of the original set of rules such that the cover matches all the rows that the original set matches . In our experiments , rule covers turned out to produce useful short descriptions of large sets of rules . Particularly useful are structural rule covers which can be computed very efficiently . In order to structure the rule set , we considered clustering of rules . In the experiments we were able to find intuitive rule clusters from rule covers . The distance measures of rules as well as the clustering methods are subject to further investigation . The methods we presented for forming rule covers and rule clusters are based on how the rules match the data set . The results indicate that the data set is itself a very good source for additional information about the discovered rules . The presented methods are efficient . As the techniques are automatic they should be augmented with user-guided methods , such as templates , to take domain knowledge and user 's interests into account . An open problem is pruning within those association rules that are not very strong | the problem is actually to find a satisfactory general definition for redundancy . A closely related problem is how to combine association rules with the same consequent . References -LSB- AIS93 -RSB- Rakesh Agrawal , Tomasz Imielinski , and Arun Swami . Mining association rules between sets of items in large databases . In SIGMOD-93 , 207 -LCB- 216 , May 1993 . -LSB- AS94 -RSB- Rakesh Agrawal and Ramakrishnan Srikant . Fast algorithms for mining association rules in large databases . In VLDB-94 , September 1994 . -LSB- CLR90 -RSB- Thomas H. Cormen , Charles E. Leiserson , and Ronald L. Rivest . Introduction to Algorithms . MIT Press , Cambridge , MA , 1990 . -LSB- CN89 -RSB- Peter Clark and Tim Niblett . The CN2 induction algorithm . Machine Learning , 3:261 -LCB- 283 , 1989 . -LSB- KMR +94 -RSB- Mika Klemettinen , Heikki Mannila , Pirjo Ronkainen , Hannu Toivonen , and A. Inkeri Verkamo . Finding interesting rules from large sets of discovered association rules . In CIKM-94 , 401 -LCB- 407 , November 1994 . -LSB- MMHL86 -RSB- Ryszard S. Michalski , Igor Mozetic , Jiarong Hong , and Nada Lavrac . The multipurpose incremental learning system AQ15 and its testing application to three medical domains . In AAAI-86 , 1041 -LCB- 1045 , 1986 . -LSB- MTV94 -RSB- Heikki Mannila , Hannu Toivonen , and A. Inkeri Verkamo . Efficient algorithms for discovering association rules . In KDD-94 , 181 -LCB- 192 , July 1994 . -LSB- PSM94 -RSB- Gregory Piatetsky-Shapiro and Christopher J. Matheus . The interestingness of deviations . In KDD-94 , 25 -LCB- 36 , July 1994 . -LSB- Ull88 -RSB- Jeffrey D. Ullman . Principles of Database and Knowledge-Base Systems , volume I. Computer Science Press , Rockville , MD , 1988 .

O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

association rules
clustering
covers
data mining