To

Appear

ACM

Transactions

on

Software

Engr

and

Methodology

October

The

Design

and

Implementation

of

Hierarchical

Software

Systems

With

Reusable

Components

Don

Batory

and

Sean

O'Malley

Department

of

Computer

Sciences

The

University

of

Texas

Austin

Texas

Abstract

We

present

a

domain-independent

model

of

hierarchical

software

system

design

and

construction

that

is

based

on

interchangeable

software

components

and

largescale

reuse

The

model

unifies

the

conceptualizations

of

two

independent

projects

Genesis

and

Avoca

that

are

successful

examples

of

software

component/building-block

technologies

and

domain

modeling

Building-block

technologies

exploit

large-scale

reuse

rely

on

open

architecture

software

and

elevate

the

granularity

of

programming

to

the

subsystem

level

Domain

modeling

formalizes

the

similarities

and

differences

among

systems

of

a

domain

We

believe

our

model

is

a

blue-print

for

achieving

software

component

technologies

in

many

domains

This

research

was

supported

in

part

by

a

grant

from

Texas

Instruments

Introduction

Mature

engineering

disciplines

rely

heavily

on

well-understood

technologies

that

have

been

standardized

By

purchasing

off-the-shelf

components

engineers

can

create

customized

systems

economically

by

building

only

the

parts

that

are

application-specific

Unnecessary

reinvention

of

technology

is

thereby

avoided

Contemporary

software

systems

have

been

simple

enough

for

massive

technology

reinvention

to

be

economically

feasible

However

as

software

system

complexity

increases

technology

reinvention

becomes

unaffordable

There

are

many

domains

today

that

are

technologically

stable

and

ripe

for

standardization

Certainly

there

will

be

more

in

the

future

Many

domains

will

concern

hierarchical

systems

where

a

progression

of

increasingly

more

sophisticated

software

technologies

are

layered

upon

each

other

A

classical

but

largely

unrealized

goal

of

software

engineering

is

software

component

-LRB-

building-block

-RRB-

technologies

Such

technologies

are

envisioned

to

exploit

large-scale

reuse

leverage

off

of

open-architecture

designs

and

elevate

the

granularity

of

programming

to

the

sub-system

level

-LSB-

-RSB-

It

is

believed

that

software

component

technologies

can

be

achieved

through

domain

analysis

an

effort

to

formalize

the

similarities

and

differences

among

systems

of

a

mature

and

well-understood

domain

-LSB-

-RSB-

Our

interest

in

component

technologies

and

domain

analysis

has

arisen

from

our

involvement

in

two

independent

projects

Genesis

and

Avoca

Genesis

is

the

first

buildingblocks

technology

for

database

management

systems

-LSB-

-RSB-

Using

a

graphical

layout

editor

a

customized

DBMS

can

be

specified

by

composing

prefabricated

software

components

A

university-quality

-LRB-

e.g.

University

Ingres

-RRB-

DBMS

over

lines

of

C

can

be

produced

and

running

within

twenty

minutes

Avoca

is

a

system

for

constructing

efficient

and

modular

network

software

suites

using

a

combination

of

pre-existing

and

newly

created

communication

protocols

-LSB-

-RSB-

Protocol

suites

are

expressed

as

a

graph

of

prefabricated

protocol

components

The

graph

is

loaded

into

a

communications

kernel

-LRB-

the

xkernel

-LSB-

-RSB-

-RRB-

and

executed

Genesis

and

Avoca

are

successful

examples

of

both

software

component

technologies

and

domain

modeling

When

we

compared

Genesis

and

Avoca

we

were

amazed

at

the

similarities

in

their

conceptual

design

organization

and

implementation

We

concluded

that

the

similarities

were

not

accidental

but

were

intrinsic

to

building-block

technologies

This

paper

reports

our

efforts

to

unify

the

conceptualizations

of

Genesis

and

Avoca

We

present

a

domain-independent

model

of

hierarchical

software

system

design

and

construction

that

is

based

on

interchangeable

software

components

and

large-scale

reuse

A

key

feature

of

this

model

and

our

most

novel

contribution

is

recognition

of

the

fundamental

role

of

symmetric

components

in

large

scale

reuse

these

components

have

the

unusual

property

that

they

can

be

composed

in

virtually

arbitrary

ways

We

demonstrate

the

practicality

of

our

model

by

using

it

to

describe

accurately

the

systems

that

we

have

built

Our

model

is

actually

a

meta-model

of

large

scale

system

construction

which

we

believe

can

be

used

to

define

models

of

open

architectures

for

many

different

domains

We

do

not

present

a

methodology

for

modeling

a

domain

in

terms

of

the

meta-model

that

is

far

beyond

the

scope

of

this

paper

and

is

the

subject

of

ongoing

work

-LSB-

-RSB-

Our

work

affirms

and

extends

basic

insights

of

many

pioneers

in

software

engineering

the

software

families

and

abstract

interface

concepts

of

Parnas

-LSB-

-RSB-

parameterized

types

of

Goguen

-LSB-

-RSB-

hierarchical

system

designs

of

Habermann

-LSB-

-RSB-

object-orientation

of

Goldberg

-LSB-

-RSB-

and

the

frameworks

concept

of

Deutsch

-LSB-

-RSB-

We

begin

by

explaining

the

superstructure

of

large

scale

systems

and

its

relationship

to

the

design

of

open-architecture

software

The

Structure

of

Large

Scale

Software

Systems

The

structure

of

large

scale

software

systems

can

be

modeled

by

an

elementary

notation

that

reflects

the

obvious

fact

that

systems

are

designed

as

assemblies

of

components

and

that

components

fit

together

in

very

specific

ways

The

model

postulates

that

components

are

instances

of

types

and

components

themselves

may

be

parameterized

The

ways

in

which

components

fit

together

to

form

systems

is

captured

elegantly

through

the

use

of

typed

parameters

and

typed

expressions

We

start

with

a

presentation

of

the

model

framework

and

its

notation

We

then

demonstrate

the

model

's

generality

by

reviewing

the

domain

models

of

Genesis

and

Avoca

and

give

some

insights

into

the

problems

of

contemporary

software

systems

The

Model

Framework

and

Notation

Basics

A

type

is

a

set

of

values

An

abstract

data

type

-LRB-

ADT

-RRB-

is

a

type

plus

operations

on

the

values

of

the

type

A

class

is

an

ADT

that

belongs

to

an

inheritance

lattice

-LSB-

-RSB-

A

component

is

a

closely-knit

cluster

of

classes

that

act

as

a

unit

-LSB-

-RSB-

Normally

the

values

of

a

type

are

simple

-LRB-

e.g.

numbers

strings

etc

-RRB-

When

values

become

complex

entities

different

names

-LRB-

other

than

type

-RRB-

are

generally

used

For

example

a

set

of

types

is

called

a

meta-type

A

set

of

ADTs

is

called

a

type

class

-LSB-

-RSB-

and

a

set

of

classes

is

a

theory

-LSB-

-RSB-

We

will

call

a

set

of

components

a

realm

Note

that

meta-types

type

classes

theories

and

realms

are

themselves

types

Our

model

deals

with

components

and

realms

Components

The

fundamental

unit

of

large

scale

software

construction

is

the

component

Every

component

has

an

interface

and

an

implementation

Following

the

lead

of

Parnas

-LSB-

-RSB-

the

interface

of

a

component

is

anything

that

is

visible

externally

to

the

component

Everything

else

belongs

to

its

implementation

Every

component

is

a

member

of

a

realm

T

where

all

members

of

T

realize

exactly

the

same

interface

but

in

different

ways

This

means

that

members

of

a

realm

are

plugcompatible

and

interchangeable

The

interface

of

a

realm

follows

directly

from

an

objectoriented

design

it

is

the

set

of

one

or

more

classes

-LRB-

their

objects

operations

and

interrelationships

-RRB-

that

are

exported

by

each

of

its

members

The

interface

of

a

component

however

has

additional

information

such

as

the

component

's

name

performance

characteristics

source

and

object

files

etc.

Thus

when

we

say

two

components

share

the

same

interface

or

are

plug-compatible

we

are

referring

to

the

realm-specific

portion

of

their

interface

that

they

have

in

common

-LRB-

What

constitutes

a

component

interface

will

be

defined

precisely

in

Section

for

now

an

informal

notion

will

suffice

In

Section

we

consider

ways

components

can

be

members

of

multiple

realms

for

now

we

assume

a

component

is

a

member

of

precisely

one

realm

-RRB-

Libraries

and

Parameterized

Components

As

a

practical

matter

most

members

-LRB-

components

-RRB-

of

a

realm

are

never

implemented

few

ever

get

beyond

the

paper-design

stage

Those

that

are

define

a

library

We

use

the

notation

T

-LCB-

-RCB-

to

mean

that

realm

T

has

and

as

library

members

Realms

and

libraries

are

inherently

extensible

as

it

is

always

possible

to

add

another

component

as

a

member

Components

reference

other

components

via

parameters

Let

the

notation

t

T

mean

that

component

t

is

of

type

T

-LRB-

or

t

belongs

to

realm

T

-RRB-

and

t

-LCB-

T

-RCB-

means

t

is

a

set

of

one

or

more

members

of

T.

Consider

component

c

-LSB-

x

y

-LCB-

-RCB-

-RSB-

c

has

two

parameters

x

and

y

where

x

must

be

component

of

type

and

y

must

be

a

set

of

components

of

type

Component

Semantics

Every

component

implements

an

abstract-to-concrete

mapping

which

is

a

transformation

of

objects

and

operations

visible

at

its

interface

or

abstract

level

to

objects

and

operations

at

its

concrete

level

As

mentioned

above

the

abstract

interface

of

a

component

is

defined

by

its

realm

The

concrete

interface

of

a

component

is

defined

by

the

union

of

the

interfaces

of

the

realms

of

its

parameters

For

example

component

c

-LSB-

x

y

-LCB-

-RCB-

-RSB-

T

exports

T

as

its

abstract

interface

and

imports

as

its

concrete

interface

Thus

c

-LSB-

-RSB-

translates

objects

and

operations

of

T

to

objects

and

operations

of

and

vice

versa

A

critical

concept

here

is

that

components

do

not

know

how

their

concrete

objects

and

operations

are

implemented

A

component

can

be

thought

of

as

a

layer

where

a

software

system

is

a

stacking

of

different

layers

-LRB-

i.e.

a

composition

of

components

-RRB-

It

is

normally

the

case

that

components

can

only

be

composed

-LRB-

stacked

-RRB-

in

a

predefined

order

Figure

a

shows

a

stacking

of

layers

where

is

on

top

and

is

on

the

bottom

translates

its

interface

objects

and

operations

into

the

interface

objects

and

operations

of

in

turn

translates

its

interface

objects

and

operations

into

interface

objects

and

operations

of

Note

that

if

the

realms

of

these

layers

-LRB-

TOP

MIDDLE

and

BOTTOM

-RRB-

are

different

then

only

one

composition

of

these

layers

is

possible

-LRB-

Fig

b

-RRB-

In

Figure

b

the

libraries

for

TOP

MIDDLE

and

BOTTOM

have

a

single

member

calls

calls

-LRB-

b

-RRB-

-LRB-

a

-RRB-

LStack

-LSB-

-LSB-

-RSB-

-RSB-

-RSB-

BOTTOM

-LCB-

-RCB-

MIDDLE

-LCB-

-LSB-

y

BOTTOM

-RSB-

-RCB-

TOP

-LCB-

-LSB-

x

MIDDLE

-RSB-

-RCB-

Figure

Nonpermutable

Stacking

of

Layers

Symmetric

Components

A

distinctive

and

fundamental

concept

of

our

model

is

the

possibility

of

symmetric

components

i.e.

components

that

can

be

composed

-LRB-

stacked

-RRB-

in

arbitrary

orders

More

specifically

a

component

of

realm

T

is

symmetric

iff

it

has

at

least

one

parameter

of

type

T.

Components

d

-LSB-

z

R

-RSB-

and

e

-LSB-

z

R

-RSB-

of

realm

R

are

symmetric

as

both

have

a

parameter

z

of

type

R.

Thus

compositions

d

-LSB-

e

-LSB-

z

R

-RSB-

-RSB-

and

e

-LSB-

d

-LSB-

z

R

-RSB-

-RSB-

are

possible

Unix

file

filters

are

prototypical

examples

of

symmetric

components

Piping

the

output

of

one

filter

into

another

is

component

composition

because

filters

have

the

same

interface

they

can

be

composed

in

different

orders

Usually

the

order

in

which

components

are

composed

makes

a

substantial

difference

in

performance

and

semantics

UFILTERS

-LCB-

dtbl

-LSB-

x

UFILTER

-RSB-

deqn

-LSB-

x

UFILTER

-RSB-

-RCB-

Note

that

Unix

pipe

expressions

like

dtbl

deqn

ditroff

correspond

to

ditroff

-LSB-

deqn

-LSB-

dtbl

-LSB-

-RSB-

-RSB-

-RSB-

in

our

notation

Composition

Systems

and

Domains

Composition

is

the

rules

and

operations

of

component

parameter

instantiation

i.e.

the

guidelines

by

which

components

can

be

glued

together

A

software

system

is

a

type

expression

-LRB-

i.e.

a

composition

of

components

-RRB-

The

system

LStack

of

Figure

b

for

example

is

defined

by

the

expression

-LSB-

-LSB-

-RSB-

-RSB-

The

set

of

all

software

systems

that

present

the

interface

of

realm

T

is

called

the

domain

of

T

denoted

Domain

-LRB-

T

-RRB-

The

concepts

of

domain

and

realm

although

similar

are

actually

quite

different

A

realm

is

a

set

of

components

and

a

domain

is

a

set

of

expressions

Only

when

a

realm

consists

solely

of

parameterless

components

will

it

be

indistinguishable

from

a

domain

It

is

this

special

case

that

allows

us

to

treat

existing

systems

-LRB-

e.g.

commercial

DBMSs

-RRB-

as

primitives

in

defining

higher-level

systems

-LRB-

e.g.

command

and

control

-RRB-

from

components

A

software

tool

that

implements

rules

of

composition

is

a

component

layout

editor

it

provides

a

language

in

which

component

expressions

can

be

written

The

set

of

all

systems

of

Domain

-LRB-

T

-RRB-

that

can

be

specified

by

a

layout

editor

from

compositions

of

library

components

is

called

the

family

of

T

denoted

Family

-LRB-

T

-RRB-

Family

-LRB-

T

-RRB-

is

always

a

subset

of

Domain

-LRB-

T

-RRB-

Figure

summarizes

these

concepts

composition

realm-n

library

editor

layout

system

family

domain

Figure

Realms

Libraries

Composition

Layout

Editor

Family

of

Systems

and

Domain

Grammars

and

Domain

Models

There

is

a

close

correspondence

between

our

concepts

and

grammars

The

left-hand

side

of

the

table

below

shows

two

compound

productions

one

for

S

and

the

other

for

R

the

right-hand

side

shows

the

corresponding

realms

S

and

R

S

fi

a

b

c

S

-LCB-

a

b

c

-RCB-

R

fi

g

S

h

S

i

R

R

-LCB-

g

-LSB-

x

S

-RSB-

h

-LSB-

x

S

-RSB-

i

-LSB-

y

R

-RSB-

-RCB-

Note

the

following

similarities

A

component

corresponds

to

a

production

Parameterized

components

are

productions

whose

right-hand

sides

reference

nonterminals

parameterless

components

are

productions

that

only

reference

terminals

Symmetric

components

correspond

to

recursive

productions

The

left-hand

side

of

a

production

is

the

interface

of

a

component

and

the

right-hand

side

is

its

implementation

A

realm

is

the

set

of

all

productions

with

the

same

head

A

software

system

is

a

sentence

and

a

domain

is

a

language

Semantic

error

checking

is

the

rules

of

composition

A

model

of

a

domain

-LRB-

or

domain

model

-RRB-

is

the

set

of

realms

and

the

rules

of

composition

that

define

the

software

systems

of

that

domain

It

is

also

a

grammar

for

expressing

the

systems

of

a

domain

as

compositions

of

primitive

components

Just

as

recursive

productions

play

a

fundamental

role

in

compactly

expressing

a

language

so

too

do

symmetric

components

play

a

fundamental

role

in

concisely

expressing

fundamental

units

of

large

scale

reuse

of

a

domain

We

will

consider

examples

of

symmetric

components

shortly

Hierarchical

Systems

Complex

software

systems

are

modeled

as

a

sequence

of

named

type

expressions

with

no

forward

or

recursive

references

A

hierarchical

software

system

has

an

acyclic

call

graph

where

nodes

are

components

and

edges

denote

call

relations

The

hierarchical

system

H

shown

below

has

components

B

and

C

both

calling

subsystem

X

X

is

modeled

as

a

common

subexpression

that

is

defined

separately

X

D

-LSB-

E

-RSB-

H

A

-LSB-

B

-LSB-

X

-RSB-

C

-LSB-

X

-RSB-

-RSB-

X

E

D

CB

A

Component

parameter

instantiation

has

call-by-value

semantics

System

H

has

two

instances

of

subsystem

X

a

single

copy

of

the

code

exists

for

X

but

execution

instances

interacting

with

components

B

and

C

are

distinct

In

database

and

network

software

there

may

be

variables

or

data

structures

within

a

component

that

may

be

shared

by

different

component

execution

instances

A

list

of

buffers

is

an

example

In

Avoca

semaphores

are

required

for

shared

variable

access

the

same

holds

for

a

concurrent

version

of

Genesis

Thus

different

execution

instances

appear

externally

to

be

independent

internally

within

a

component

they

may

have

a

controlled

interference

It

is

worth

noting

that

our

model

does

admit

the

possibility

of

systems

with

unbounded

recursion

such

as

Y

m

-LSB-

Y

-RSB-

which

has

a

recursive

definition

We

know

of

no

practical

example

of

such

systems

only

finite

replication

of

components

-LRB-

like

m

-LSB-

m

-LSB-

p

-RSB-

-RSB-

-RRB-

seem

to

occur

Component

Reuse

Recognizing

and

achieving

software

reuse

are

fundamental

problems

in

software

engineering

An

important

form

of

reuse

is

component

reuse

which

occurs

in

our

model

when

two

or

more

expressions

reference

the

same

component

Thus

if

a

-LSB-

b

-LSB-

c

-RSB-

-RSB-

and

d

-LSB-

b

-LSB-

q

-RSB-

-RSB-

are

expressions

-LRB-

software

systems

-RRB-

component

b

is

reused

Common

subexpressions

correspond

to

subsystem

reuse

such

as

subsystem

X

above

We

will

encounter

two

different

types

of

reuse

later

algorithm

reuse

and

class

reuse

Examples

The

domain

models

of

Genesis

and

Avoca

are

presented

in

the

next

sections

to

illustrate

the

above

concepts

Parenthetically

we

note

that

in

order

to

understand

any

domain

model

one

really

needs

to

be

familiar

with

the

domain

itself

The

domain

models

presented

below

are

not

intended

to

be

tutorial

citations

are

given

so

that

interested

readers

can

find

additional

details

Throughout

this

paper

sections

whose

numbers

are

tagged

with

a

dagger

-LRB-

-RRB-

marker

can

be

skipped

by

readers

who

are

not

interested

in

concept

illustrations

Genesis

A

Domain

Model

for

Database

Management

Systems

The

Genesis

-LRB-

-RRB-

prototype

implements

a

portion

of

a

domain

model

for

database

management

systems

enables

centralized

single-client

DBMSs

to

be

synthesized

from

component

libraries

The

complete

domain

model

described

in

-LSB-

-RSB-

covers

multiclient

parallel

and

distributed

DBMS

implementations

The

model

itself

is

nontrivial

alone

supports

twelve

distinct

libraries

It

is

beyond

the

scope

of

this

paper

to

cover

them

all

or

to

explain

the

elaborate

relationships

-LRB-

e.g.

parameter

instantiations

-RRB-

that

can

be

admitted

among

components

In

this

section

we

will

highlight

the

terrain

of

the

model

and

indicate

where

additional

complexities

lie

File

Structures

Realms

File

structures

map

internal

relations

to

blocks

on

secondary

storage

They

are

type

expressions

of

components

from

three

realms

AMETHOD

NODE

and

BLOCK

Some

of

the

current

library

members

are

listed

below

AMETHOD

-LCB-

heap

-LSB-

d

NODE

-RSB-

unord

-LSB-

d

NODE

-RSB-

bplus

-LSB-

d

i

NODE

-RSB-

isam

-LSB-

d

i

NODE

-RSB-

hash

-LSB-

d

NODE

-RSB-

grid

-LSB-

d

NODE

-RSB-

-RCB-

NODE

-LCB-

-LSB-

p

BLOCK

-RSB-

-LSB-

p

BLOCK

-RSB-

-LSB-

p

o

BLOCK

-RSB-

-LSB-

p

o

BLOCK

-RSB-

-RCB-

BLOCK

-LCB-

-RCB-

AMETHOD

is

the

realm

of

access

methods

Among

library

members

are

nonkeyed

access

methods

-LRB-

heap

unord

-RRB-

single-keyed

methods

-LRB-

hash

bplus

isam

-RRB-

and

multi-keyed

methods

-LRB-

grid

-RRB-

Every

AMETHOD

component

maps

an

internal

relation

to

one

or

two

sets

of

logical

blocks

called

nodes

One

set

of

nodes

contains

only

data

records

the

other

set

-LRB-

if

present

-RRB-

contains

index

records

For

example

bplus

trees

and

isam

files

store

data

records

in

their

leaf-level

nodes

and

store

index

records

in

nonleaf

nodes

Hence

the

components

isam

-LSB-

d

i

NODE

-RSB-

and

bplus

-LSB-

d

i

NODE

-RSB-

have

two

parameters

-LRB-

d

-RRB-

to

specify

the

implementation

of

data/leaf

nodes

and

-LRB-

i

-RRB-

to

specify

the

implementation

of

index

nodes

Some

access

methods

such

as

heap

-LSB-

d

NODE

-RSB-

and

hash

-LSB-

d

NODE

-RSB-

have

no

index

nodes

and

have

only

the

single

parameter

-LRB-

d

-RRB-

NODE

is

the

realm

of

node

implementations

A

node

or

logical

block

is

a

sequence

of

records

A

node

component

maps

a

logical

block

to

one

or

more

physical

blocks

where

the

first

block

is

the

primary

block

and

the

remaining

-LRB-

if

any

-RRB-

are

overflow

blocks

How

primary

and

overflow

blocks

are

implemented

is

specified

by

parameters

-LRB-

p

-RRB-

and

-LRB-

o

-RRB-

of

NODE

Readers

may

wonder

how

nonkeyed

single-keyed

and

multikeyed

file

structures

can

have

the

same

interface

Recognizing

that

nonkeyed

and

single-keyed

structures

are

degenerate

cases

of

multikeyed

structures

it

follows

that

a

general-purpose

interface

for

multikeyed

structures

works

for

single-keyed

and

nonkeyed

structures

as

well

See

-LSB-

-RSB-

for

further

details

on

the

implementation

of

Genesis

components

A

node

can

optionally

maintain

records

in

key

order

optionally

share

overflow

blocks

with

other

nodes

and

optionally

have

primary

blocks

and/or

overflow

blocks

Each

combination

of

options

yields

a

distinct

component

-LSB-

p

o

BLOCK

-RSB-

for

example

maintains

records

in

primary

key

order

uses

a

primary

block

and

does

not

share

its

overflow

blocks

with

other

nodes

BLOCK

is

the

realm

of

record

blocking

methods

i.e.

how

records

are

packaged

into

physical

blocks

Records

can

be

optionally

fixed

length

or

variable

length

and

can

be

anchored

-LRB-

i.e

have

physical

addresses

that

do

not

change

with

time

-RRB-

or

unanchored

Four

components

cover

all

options

The

component

assigns

anchored

addresses

to

fixed-length

records

BLOCK

components

do

not

require

external

services

and

hence

are

unparameterized

Type

Expressions

A

file

structure

is

an

expression

of

type

AMETHOD

An

indexed

sequential

file

-LRB-

isf

-RRB-

that

uses

ordered

primary

unshared

implementations

of

data

nodes

and

ordered

primary

block

only

implementations

of

index

nodes

where

all

records

are

fixedlength

and

unanchored

corresponds

to

the

expression

isf

isam

-LSB-

-LSB-

-RSB-

-LSB-

-RSB-

-RSB-

and

an

unordered

file

-LRB-

uf

-RRB-

that

uses

unordered

primary

-LRB-

block

-RRB-

only

implementations

of

data

nodes

where

records

are

variable

length

and

anchored

corresponds

to

the

expression

uf

unord

-LSB-

-LSB-

-RSB-

-RSB-

Clearly

a

large

family

of

different

file

structures

can

be

assembled

from

AMETHOD

NODE

and

BLOCK

components

See

-LSB-

-RSB-

for

further

details

Storage

Systems

Realms

Storage

systems

map

conceptual

relations

to

internal

relations

They

are

type

expressions

of

symmetric

components

from

the

realm

FMAP

Some

of

the

current

library

members

are

listed

below

FMAP

-LCB-

index

-LSB-

d

i

FMAP

-RSB-

-LSB-

d

FMAP

-RSB-

-LSB-

d

FMAP

-RSB-

frag

-LSB-

s

FMAP

-RSB-

-LSB-

s

-LCB-

FMAP

-RCB-

-RSB-

internal

-LSB-

d

AMETHOD

-RSB-

-RCB-

FMAP

is

the

realm

of

file

mapping

components

Each

transforms

a

conceptual

file

to

one

or

more

internal

files

What

is

an

internal

file

to

one

component

may

be

a

conceptual

file

to

another

and

hence

most

FMAP

components

are

symmetric

Consider

the

fragmentation

of

a

long

record

into

short

records

that

are

interlinked

together

Operations

on

long

records

-LRB-

e.g.

insertion

deletion

retrieval

-RRB-

have

an

obvious

translation

to

operations

on

short

records

In

the

component

frag

-LSB-

s

FMAP

-RSB-

encapsulates

this

mapping

It

is

parameterized

because

the

method

-LRB-

s

-RRB-

by

which

short

records

are

stored

is

unknown

Because

this

parameter

-LRB-

s

-RRB-

and

frag

-LSB-

-RSB-

are

of

type

FMAP

frag

-LSB-

-RSB-

is

symmetric

Now

consider

the

mapping

of

uncompressed

records

to

compressed

records

Once

again

operations

on

uncompressed

records

have

an

obvious

translation

to

operations

on

compressed

records

In

the

components

-LSB-

d

FMAP

-RSB-

and

-LSB-

d

FMAP

-RSB-

encapsulate

this

mapping

for

the

run-length

and

ziv-lempel

encoding

algorithms

respectively

Both

are

parameterized

by

the

method

of

storing

encoded

records

As

both

components

and

their

parameters

are

of

type

FMAP

both

are

symmetric

The

composition

frag

-LSB-

-LSB-

d

FMAP

-RSB-

-RSB-

corresponds

to

the

implementation

where

records

are

fragmented

before

being

run-length

encoded

In

contrast

the

composition

-LSB-

frag

-LSB-

d

FMAP

-RSB-

-RSB-

corresponds

to

the

implementation

of

encoding

before

fragmentation

As

mentioned

earlier

the

order

in

which

symmetric

components

are

composed

makes

a

difference

both

in

performance

and

in

results

Consider

two

more

components

index

-LSB-

-RSB-

and

internal

-LSB-

-RSB-

Index

-LSB-

d

i

FMAP

-RSB-

maps

a

conceptual

file

to

an

inverted

file

which

consists

of

a

data

file

and

one

or

more

secondary

index

files

Parameter

-LRB-

d

-RRB-

specifies

the

implementation

of

the

data

file

and

parameter

-LRB-

i

-RRB-

specifies

the

implementation

of

the

index

files

Internal

-LSB-

d

AMETHOD

-RSB-

encapsulates

the

transformation

of

operations

on

conceptual

files

to

low-level

operations

on

file

structures

This

typically

involves

examining

a

retrieval

predicate

to

determine

how

a

file

structure

is

to

be

searched

-LRB-

i.e.

scanned

range

retrievals

point

searches

etc.

-RRB-

Type

Expressions

A

storage

system

is

an

expression

of

type

FMAP

A

storage

system

-LRB-

-RRB-

that

maps

conceptual

relations

to

inverted

files

where

data

files

are

stored

in

unordered

files

-LRB-

defined

earlier

by

expression

uf

-RRB-

and

index

files

stored

in

isam

files

-LRB-

defined

earlier

by

expression

isf

-RRB-

corresponds

to

the

expression

index

-LSB-

internal

-LSB-

uf

-RSB-

internal

-LSB-

isf

-RSB-

-RSB-

Storage

system

is

an

enhancement

of

in

that

it

handles

long

records

Long

records

are

first

indexed

before

being

run-length

compressed

and

then

fragmented

index

-LSB-

-LSB-

frag

-LSB-

internal

-LSB-

uf

-RSB-

-RSB-

-RSB-

internal

-LSB-

isf

-RSB-

-RSB-

With

symmetric

FMAP

components

a

very

large

family

of

storage

systems

can

be

defined

-LSB-

-RSB-

Relational

Database

Systems

Realms

A

database

is

a

collection

of

interconnected

conceptual

relations

A

link

is

an

interconnection

between

the

records

of

two

different

relations

-LRB-

Normally

interconnections

are

specified

by

join

predicates

relating

records

of

one

relation

to

zero

or

more

records

of

another

However

relationships

can

be

defined

by

manually

connecting

one

record

with

another

In

CODASYL

terminology

a

link

is

called

a

set

-LSB-

-RSB-

-RRB-

A

relational

system

maps

a

database

with

a

nonprocedural

data

language

interface

to

a

set

of

conceptual

relations

-LRB-

with

no

links

-RRB-

whose

interface

is

procedural

Relational

systems

are

compositions

of

components

from

the

realms

DLANG

LINK

and

RSTREAM

with

references

to

storage

systems

Among

the

current

library

members

of

these

realms

are

DLANG

-LCB-

sql

-LSB-

l

LINK

o

-LCB-

OPER

-RCB-

-RSB-

quel

-LSB-

l

LINK

o

-LCB-

OPER

-RCB-

-RSB-

-RCB-

LINK

-LCB-

-LSB-

f

FMAP

-RSB-

-LSB-

f

FMAP

-RSB-

-LSB-

f

FMAP

-RSB-

-LSB-

f

FMAP

-RSB-

-RCB-

RSTREAM

-LCB-

-LSB-

s

s

RSTREAM

-RSB-

sort

-LSB-

s

RSTREAM

-RSB-

-RCB-

DLANG

is

the

realm

of

data

models

and

their

nonprocedural

data

languages

Each

DLANG

component

translates

nonprocedural

queries

into

optimized

expressions

that

reference

operations

on

relations

and

links

Furthermore

each

component

is

parameterized

by

-LRB-

l

-RRB-

the

method

by

which

links

between

conceptual

relations

are

implemented

and

-LRB-

o

-RRB-

the

set

of

zero

or

more

record

stream

components

that

may

be

needed

to

process

queries

LINK

is

the

realm

of

link

implementations

Each

component

encapsulates

a

mapping

of

a

database

of

conceptual

relations

and

links

to

a

database

of

conceptual

relations

without

links

Every

LINK

component

is

parameterized

by

-LRB-

f

-RRB-

the

method

by

which

conceptual

relations

are

implemented

LINK

components

are

either

hard

or

soft

Soft

components

implement

traditional

relational

join

algorithms

like

-LSB-

f

FMAP

-RSB-

and

-LSB-

f

FMAP

-RSB-

Hard

components

implement

links

by

physically

connecting

records

together

via

pointers

such

as

-LSB-

f

FMAP

-RSB-

and

-LSB-

f

FMAP

-RSB-

RSTREAM

is

the

realm

of

-LRB-

usually

symmetric

-RRB-

components

that

transform

one

or

more

concrete

input

record

streams

into

an

abstract

output

record

stream

These

components

are

used

by

DLANG

components

to

process

queries

Example

components

include

cross

product

and

sort

Type

Expressions

A

relational

system

is

an

expression

of

type

DLANG

A

relational

system

-LRB-

-RRB-

that

presents

the

QUEL

data

model

and

data

language

implements

links

by

pointer

arrays

and

stores

conceptual

relations

in

the

storage

system

-LRB-

defined

above

-RRB-

and

references

the

cross

product

and

sort

components

corresponds

to

the

expression

quel

-LSB-

-LSB-

-RSB-

-LCB-

-LSB-

-RSB-

sort

-LSB-

-RSB-

-RCB-

-RSB-

Note

that

the

parameters

of

-LSB-

-RSB-

and

sort

-LSB-

-RSB-

components

are

not

instantiated

The

reason

is

that

these

components

are

grafted

onto

operator

trees

that

are

generated

at

run-time

by

query

optimizers

In

other

words

DBMSs

compose

certain

components

dynamically

in

order

to

process

retrieval

requests

With

the

exception

of

the

RSTREAM

realm

all

realms

that

we

have

considered

so

far

have

components

that

are

statically

composed

at

design

time

That

RSTREAM

components

are

composed

dynamically

at

run

time

appears

to

be

an

unusual

feature

of

DBMSs

See

-LSB-

-RSB-

for

details

on

the

reuse

aspects

of

query

The

DLANG

interface

is

character

input

and

output

quel

-LSB-

-RSB-

and

sql

-LSB-

-RSB-

differ

in

the

strings

of

characters

that

they

recognize

even

though

their

procedural

interfaces

are

indistinguishable

optimization

algorithms

Other

Topics

Software

Busses

It

is

common

for

DBMSs

to

offer

alternative

file

and/or

link

implementations

without

exposing

their

difference

at

the

DBMS

interface

By

giving

appropriate

database

schema

directives

specific

file

and

link

implementations

can

be

declared

-LSB-

-RSB-

Genesis

provides

alternative

implementations

via

multiplexing

component

called

a

software

bus

-LRB-

Software

busses

can

be

thought

of

as

the

software-lego

counterpart

to

discriminate

records

-RRB-

As

an

example

-LSB-

s

-LCB-

FMAP

-RCB-

-RSB-

is

a

symmetric

member

of

the

FMAP

realm

It

permits

relations

of

a

database

to

have

any

FMAP

implementation

listed

in

the

-LRB-

s

-RRB-

parameter

of

-LRB-

Each

relation

of

the

database

is

tagged

with

an

identifier

which

specifies

how

the

relation

should

be

implemented

-RRB-

Thus

a

variation

on

relation

system

which

allows

relations

to

be

stored

via

either

storage

system

or

is

quel

-LSB-

-LSB-

-LSB-

-RSB-

-RSB-

-LCB-

-LSB-

-RSB-

sort

-LSB-

-RSB-

-RCB-

-RSB-

When

a

relations

in

a

schema

are

declared

special

statements

are

used

to

specify

its

implementation

to

be

either

or

-LRB-

The

default

is

the

first

storage

system

listed

on

a

bus

-RRB-

Omitted

Parameters

Components

that

handle

transaction

management

recovery

management

buffer

management

primitive

data

types

-LRB-

e.g.

integer

float

etc.

-RRB-

and

other

generic

services

-LRB-

e.g.

predicate

evaluation

-RRB-

are

additional

parameters

to

the

above

components

We

chose

not

to

include

them

just

to

keep

the

model

overview

simple

Further

details

on

these

topics

are

given

in

-LSB-

-RSB-

Layout

Editors

and

Tuning

Because

type

expressions

quickly

become

difficult

to

read

Genesis

has

a

layout

editor

called

DaTE

which

enables

components

of

different

realms

to

be

composed

graphically

DaTE

guarantees

that

design

rules

are

not

violated

which

amounts

to

avoiding

illegal

compositions

of

components

Design

rule

checking

is

briefly

considered

in

Section

and

in

detail

in

-LSB-

-RSB-

Using

DaTE

a

specification

of

a

university-quality

relational

DBMS

takes

less

than

a

half-hour

The

software

that

is

generated

is

untuned

because

tuning

constants

that

are

part

of

every

component

are

assigned

default

values

By

performing

benchmarks

it

is

possible

to

tune

the

generated

software

by

selectively

altering

tuning

constants

and

recompiling

Avoca

A

Domain

Model

for

Network

Software

Systems

Avoca

is

a

network

architecture

and

domain

model

that

supports

the

development

of

encapsulated

reusable

and

efficient

communications

protocols

Protocol

suite

specifications

are

manually

written

as

unlike

Genesis

Avoca

does

not

have

a

component

layout

editor

The

runtime

environment

for

Avoca

is

provided

by

the

xkernel

an

operating

system

kernel

designed

to

run

network

protocols

-LSB-

-RSB-

One

of

the

goals

of

the

xkernel

was

to

support

encapsulated

protocols

efficiently

As

it

turned

out

most

existing

protocols

are

so

unencapsulated

in

design

and

implementation

that

they

could

not

take

advantage

of

many

of

the

xkernel

's

features

Avoca

grew

out

of

an

attempt

to

design

protocols

that

could

The

domain

model

for

Avoca

has

centered

on

the

identification

of

realms

of

protocols

for

remote

procedure

calls

remote

invocation

methods

and

network

file

systems

Work

now

underway

at

the

University

of

Arizona

should

extend

the

model

to

cover

fault

tolerant

protocols

Avoca

systems

or

protocol

suites

are

constructed

out

of

three

types

of

components

Avoca

protocols

virtual

protocols

and

existing

protocols

Avoca

protocols

are

symmetric

components

that

can

be

composed

in

virtually

arbitrary

orders

Virtual

protocols

a

subset

of

Avoca

protocols

dynamically

multiplex

over

several

lower-level

components

and

are

similar

to

Genesis

software

busses

Existing

protocols

are

generally

unencapsulated

and

can

only

be

composed

with

virtual

protocols

and

the

protocols

they

were

originally

designed

to

communicate

with

Avoca

presently

has

three

realms

ASYNC

SYNC

and

STREAM

Each

is

discussed

in

turn

Asynchronous

Protocols

Realm

Protocols

that

send

and

receive

messages

asynchronously

are

members

of

the

ASYNC

realm

Senders

send

messages

without

waiting

for

a

reply

and

all

messages

are

delivered

to

their

destinations

using

upcalls

-LSB-

-RSB-

If

a

destination

replies

to

a

message

the

sender

is

responsible

for

matching

requests

and

replies

using

local

state

or

message

header

information

ASYNC

protocols

are

generally

the

lowest

level

protocols

supported

in

any

network

software

system

Some

of

the

current

ASYNC

library

members

are

listed

below

where

DRIVERS

EXISTING

and

are

library

partitions

DRIVERS

-LCB-

-RCB-

EXISTING

-LCB-

ip

-LSB-

x

ASYNC

-RSB-

udp

-LSB-

x

ASYNC

-RSB-

-RCB-

-LCB-

blast

-LSB-

x

ASYNC

-RSB-

-LSB-

x

ASYNC

-RCB-

-LCB-

vaddr

-LSB-

local

remote

ASYNC

-RSB-

vsize

-LSB-

small

big

ASYNC

-RSB-

-RCB-

ASYNC

DRIVERS

EXISTING

The

ASYNC

realm

has

a

subrealm

of

network

device

drivers

-LRB-

DRIVERS

-RRB-

The

Avoca

library

currently

supports

drivers

for

the

AMD

and

Intel

ethernet

chips

Another

subrealm

of

ASYNC

called

EXISTING

contains

commonly

used

protocols

such

as

ip

-LSB-

-RSB-

and

udp

-LSB-

-RSB-

ip

-LSB-

-RSB-

is

the

Internet

Protocol

and

is

responsible

for

routing

packets

over

the

Internet

and

fragmenting

large

messages

in

to

smaller

ones

-LSB-

-RSB-

udp

-LSB-

-RSB-

is

a

simple

demultiplexing

protocol

which

delivers

messages

to

the

correct

UDP

port

-LRB-

a

bit

integer

address

-RRB-

The

subrealm

contains

symmetric

Avoca

protocols

that

are

not

virtual

blast

-LSB-

-RSB-

breaks

a

large

message

into

fragments

sends

the

fragments

and

reassembles

the

original

message

once

all

fragments

have

arrived

-LSB-

-RSB-

-LSB-

-RSB-

is

a

simple

demultiplexing

protocol

that

is

essentially

a

symmetric

version

of

udp

-LSB-

-RSB-

Virtual

asynchronous

protocols

are

found

in

the

subrealm

vaddr

-LSB-

-RSB-

directs

incoming

packets

to

the

first

of

its

two

ASYNC

arguments

and

determines

if

the

destination

address

can

be

reached

using

that

component

The

idea

behind

vaddr

-LSB-

-RSB-

is

a

simple

optimization

if

the

destination

of

a

packet

is

on

the

local

ethernet

-LRB-

the

first

parameter

-RRB-

then

there

is

no

reason

incur

the

overhead

of

transmitting

it

via

a

remote

transport

protocol

-LRB-

the

second

parameter

-RRB-

vsize

-LSB-

-RSB-

directs

packets

to

the

first

of

its

two

ASYNC

parameters

if

the

length

of

the

packet

is

less

than

a

predeclared

size

Larger

packets

are

directed

to

the

component

specified

by

the

second

parameter

In

general

virtual

protocols

are

multiplexing

components

that

offer

the

possibility

of

choosing

between

several

distinct

implementations

on

the

fly

and

are

like

programming

language

if

statements

where

the

actual

path

chosen

for

a

packet

is

based

on

either

static

or

dynamically

available

information

As

virtual

protocols

are

headerless

they

do

not

affect

the

messages

that

they

multiplex

and

hence

can

be

composed

without

impacting

their

compatibility

with

EXISTING

protocols

-LRB-

Virtual

protocols

are

denoted

here

by

names

beginning

with

the

letter

v

-RRB-

Type

Expressions

An

asynchronous

protocol

suite

is

an

expression

of

type

ASYNC

The

primary

purpose

of

Avoca

was

to

show

that

encapsulated

protocols

can

be

implemented

as

efficiently

as

monolithic

protocols

Hence

the

emphasis

of

Avoca

is

on

performance

Consider

the

following

examples

The

protocol

suite

was

the

first

to

make

use

of

virtual

protocols

Common

practice

before

Avoca

was

to

use

ip

-LSB-

-RSB-

even

if

packets

were

being

sent

to

machines

in

the

next

room

The

system

-LRB-

defined

below

-RRB-

eliminated

this

overhead

by

using

vaddr

-LSB-

-RSB-

to

bypass

ip

-LSB-

-RSB-

for

messages

to

hosts

on

the

local

network

-LRB-

provided

that

these

hosts

also

support

the

optimization

-RRB-

and

by

placing

these

packets

on

the

ethernet

-LRB-

-RRB-

directly

udp

-LSB-

vaddr

-LSB-

ip

-LSB-

-RSB-

-RSB-

-RSB-

Note

that

because

vaddr

-LSB-

-RSB-

does

not

modify

the

messages

that

it

maps

can

exchange

messages

with

systems

-LRB-

e.g.

udp

-LSB-

ip

-LSB-

-RSB-

-RSB-

-RRB-

that

do

not

take

this

short-cut

udp

-LSB-

-RSB-

explicitly

uses

ip

addresses

and

thus

must

sit

atop

ip

-LSB-

-RSB-

-LSB-

-RSB-

removes

this

dependency

The

address

resolution

protocol

is

used

to

make

the

determination

of

locality

As

a

second

example

the

protocol

suite

trans

-LRB-

defined

below

-RRB-

achieves

a

greater

level

of

optimization

by

fragmenting

large

packets

into

smaller

packets

This

is

accomplished

by

using

vsize

-LSB-

-RSB-

to

recognize

long

packets

and

using

blast

-LSB-

-RSB-

for

fragmentation

This

optimization

significantly

improves

the

performance

of

most

communications

systems

trans

vaddr

-LSB-

vsize

-LSB-

blast

-LSB-

-RSB-

-RSB-

ip

-LSB-

-RSB-

-RSB-

-RSB-

Synchronous

Protocols

Realm

Protocols

that

return

a

reply

message

for

each

message

sent

are

members

of

the

SYNC

realm

A

sender

sends

a

message

and

is

blocked

until

a

reply

is

received

Some

of

the

current

SYNC

library

members

are

listed

below

-LCB-

-RCB-

-LCB-

-LSB-

x

SYNC

-RSB-

-LSB-

x

SYNC

-RSB-

ureqrep

-LSB-

x

ASYNC

-RSB-

reqrep

-LSB-

x

ASYNC

-RSB-

-RCB-

-LCB-

vrpc

-LSB-

local

SYNC

-RSB-

-RCB-

SYNC

TERMINAL

The

most

common

synchronous

protocols

is

the

subrealm

of

remote

procedure

call

protocols

-LRB-

-RRB-

It

has

been

experimentally

determined

that

most

remote

procedure

calls

are

in

fact

calls

to

different

address

spaces

on

the

same

machine

-LSB-

-RSB-

is

an

efficient

protocol

that

relies

on

the

local

operating

system

to

process

remote

procedure

calls

is

the

xkernel

implementation

of

Sun

RPC

and

is

fully

compatible

with

existing

implementations

is

the

xkernel

implementation

of

Sprite

RPC

and

is

again

compatible

with

the

original

version

is

the

realm

of

synchronous

Avoca

protocols

ureqrep

-LSB-

-RSB-

and

reqrep

-LSB-

-RSB-

are

protocols

that

perform

SYNC

to

ASYNC

conversions

and

implement

the

request

reply

portion

of

Sun

RPC

and

Sprite

RPC

respectively

ureqrep

-LSB-

-RSB-

provides

an

unreliable

request

reply

service

while

reqrep

-LSB-

-RSB-

is

reliable

To

be

a

reliable

a

request

reply

protocol

must

deliver

only

one

message

to

the

destination

for

each

request

-LSB-

-RSB-

is

the

synchronous

version

of

-LSB-

-RSB-

and

is

used

to

address

remote

procedures

-LSB-

-RSB-

-LSB-

-RSB-

is

the

sun

demultiplexing

protocol

that

takes

three

addresses

as

its

input

and

as

we

'll

see

later

can

be

defined

in

terms

of

-LSB-

-RSB-

Other

protocols

were

created

to

perform

the

other

functions

normally

bundled

into

RPC

protocols

Discussions

of

these

are

beyond

the

scope

of

this

paper

Among

the

virtual

synchronous

protocols

of

subrealm

vrpc

-LSB-

-RSB-

operates

much

like

vaddr

-LSB-

-RSB-

except

that

it

support

three

possible

cases

If

the

target

of

the

RPC

is

local

to

this

machine

the

protocol

for

parameter

-LRB-

local

-RRB-

is

used

If

the

target

is

on

a

remote

machine

that

supports

a

fast

rpc

that

this

machine

understands

the

protocol

for

parameter

-LRB-

-RRB-

is

used

In

all

other

cases

the

protocol

for

parameter

-LRB-

-RRB-

is

used

Type

expressions

Addressing

in

most

network

software

systems

-LRB-

i.e

the

assigning

of

an

address

to

networking

entities

and

the

demultiplexing

of

incoming

messages

to

the

appropriate

entity

-RRB-

is

bundled

within

RPC

protocols

By

elevating

addressing

to

a

distinct

protocol

Avoca

increases

the

flexibility

of

building

and

modifying

network

software

The

protocols

in

the

SYNC

realm

resulted

from

a

decomposition

of

Sprite

RPC

and

are

usually

configured

to

give

Sprite

RPC

semantics

Sprite

RPC

is

known

to

be

much

superior

to

Sun

RPC

and

one

would

like

to

use

Sprite

RPC

wherever

possible

The

easiest

way

to

do

this

is

to

create

a

protocol

-LSB-

-RSB-

which

implements

Sun

RPC

addressing

This

allows

one

to

create

a

protocol

suite

with

Sprite

RPC

semantics

and

the

Sun

RPC

interface

A

virtual

protocol

vrpc

-LSB-

-RSB-

can

then

be

used

which

has

local

RPC

semantics

in

the

local

case

Sprite

RPC

semantics

whenever

the

destination

host

supports

vrpc

-LSB-

-RSB-

and

uses

Sun

RPC

to

communicate

with

those

machines

that

do

not

support

vrpc

-LSB-

-RSB-

The

following

composition

is

a

simplified

version

of

such

a

system

where

trans

denotes

the

ASYNC

system

specified

previously

vrpc

-LSB-

-LSB-

reqrep

-LSB-

trans

-RSB-

-RSB-

-RSB-

Note

that

vrpc

-LSB-

-RSB-

is

responsible

for

hiding

any

semantic

difference

between

the

two

subsystems

The

only

noticeable

effects

are

a

faster

and

more

reliable

version

of

Sun

RPC

Now

consider

another

example

Through

repeated

composition

-LSB-

-RSB-

can

be

used

to

construct

more

complex

addressing

schemes

Recall

that

-LSB-

-RSB-

supports

a

single

bit

address

while

-LSB-

-RSB-

supports

three

bit

address

fields

On

can

use

-LSB-

-RSB-

to

create

-LSB-

-RSB-

in

the

following

fashion

-LSB-

x

SYNC

-RSB-

-LSB-

-LSB-

-LSB-

x

SYNC

-RSB-

-RSB-

-RSB-

The

simplicity

and

flexibility

with

which

addressing

schemes

are

handled

in

Avoca

is

in

stark

contrast

with

traditional

methods

In

particular

the

Internet

community

is

faced

with

the

task

of

modifying

the

internals

of

TCP

because

the

bit

port

number

they

use

for

addresses

is

insufficient

for

many

modern

applications

-LSB-

-RSB-

In

Avoca

this

would

be

a

minute

modification

Network

File

Systems

Realm

The

STREAM

protocol

realm

presents

a

Unix

file

interface

which

offers

file

read

and

write

operations

STREAM

components

offer

different

implementations

of

network

file

systems

Currently

the

only

STREAM

protocol

that

Avoca

supports

is

-LSB-

-RSB-

Sun

's

network

file

system

STREAM

-LCB-

-LSB-

x

SYNC

-RSB-

-RCB-

-LSB-

-RSB-

is

not

really

a

communications

protocol

in

the

traditional

sense

but

more

of

a

Genesis-like

component

It

provides

a

UNIX

file-filter

interface

We

suspect

that

the

most

direct

ways

to

merge

Genesis

and

Avoca

could

be

through

this

component/realm

One

way

would

be

to

revise

-LSB-

-RSB-

to

admit

components

within

database

realms

thereby

enabling

different

file

storage

schemes

to

be

supported

internally

at

a

site

Additionally

one

could

revise

Genesis

IO-realm

components

thereby

enabling

data

to

be

distributed

transparently

across

multiple

hosts

We

are

investigating

both

possibilities

Finally

we

note

that

Avoca-produced

systems

do

not

need

post-production

tuning

unlike

Genesis-produced

DBMSs

From

our

experience

protocols

have

been

simple

enough

that

additional

tuning

has

not

been

required

Contemporary

Software

Systems

Large

software

systems

are

always

implemented

in

terms

of

components

they

are

simply

too

big

to

be

designed

and

developed

in

any

other

way

It

is

not

difficult

to

model

existing

systems

as

a

cast-in-stone

composition

of

hand-crafted

components

The

main

problem

with

today

's

software

are

the

ad

hoc

methods

of

system

design

and

decomposition

Component

reuse

-LRB-

which

we

consider

synonymous

with

large

scale

reuse

-RRB-

is

a

casualty

of

ad

hoc

methods

Each

of

the

problems

below

is

a

consequence

of

this

casualty

No

Families

of

Systems

When

contemporary

software

systems

are

examined

in

the

context

of

realms

it

is

rare

to

find

a

realm

that

has

a

library

with

more

than

one

component

One-of-a-kind

systems

are

prime

examples

their

components

have

unique

interfaces

and

unique

implementations

With

the

possible

exception

of

system

versions

-LRB-

where

outdated

components

are

replaced

with

updated

ones

-RRB-

families

of

different

systems

do

not

exist

In

contrast

families

of

multiple

systems

are

natural

by-products

of

component

reuse

No

Component

Interchangeability

Interchangeability

of

components

from

different

systems

is

impossible

in

contemporary

software

Consider

the

query

optimizer

of

database

systems

-LSB-

x

-RSB-

and

-LSB-

x

-RSB-

are

two

different

DBMSs

each

of

which

has

an

optimizer

-LRB-

x

-RRB-

as

one

of

its

parameters

The

optimizer

used

by

has

a

different

type

than

that

for

i.e.

the

optimizers

of

each

system

have

different

interfaces

and

are

not

plug

compatible

Both

DBMSs

would

need

to

use

optimizers

of

the

same

type

for

interchangeability

to

be

possible

Commercial

DBMSs

like

most

software

have

never

been

designed

with

interchangeability

of

components

in

mind

No

Symmetric

Components

The

true

building

blocks

for

some

realms

are

symmetric

components

As

we

have

shown

a

small

number

of

symmetric

components

can

be

composed

in

a

vast

number

of

ways

Not

recognizing

such

components

is

a

lost

opportunity

for

achieving

reuse

on

a

large

scale

Furthermore

by

building

only

their

compositions

one

reduces

the

likelihood

that

a

particular

composition

will

ever

be

reused

The

problem

here

may

have

a

cultural

aspect

It

is

more

important

today

to

promote

a

system

on

the

basis

of

its

uniqueness

rather

than

stressing

its

the

similarity

with

other

systems

Consequently

opportunities

for

potential

reuse

are

often

dismissed

or

go

unrecognized

Recap

We

have

shown

that

an

elementary

and

domain

independent

notation

captures

a

fundamental

aspect

of

software

system

design

namely

systems

are

assemblies

of

components

and

that

components

fit

together

in

very

specific

ways

Our

model

postulates

that

components

are

instances

of

types

and

components

themselves

may

be

parameterized

Software

systems

are

modeled

elegantly

as

type

expressions

The

type

expression

notation

and

component-realm

metamodel

provides

a

scaffolding

or

superstructure

on

which

other

aspects

of

software

design

can

be

organized

This

occasionally

requires

concepts

to

be

force-fitted

into

this

framework

sometimes

causing

a

nontrivial

shift

in

thinking

patterns

in

order

to

do

so

As

an

example

our

model

suggests

that

the

design

of

complex

software

systems

is

guided

primarily

by

few

decisions

-LRB-

-RRB-

what

components

to

use

and

-LRB-

-RRB-

what

is

the

order

in

which

components

are

to

be

composed

The

actual

implementation

of

the

components

although

important

is

a

lower-level

detail

It

is

worth

noting

that

over

fifteen

years

ago

Haberman

observed

that

software

system

designs

are

hierarchical

but

system

implementations

need

not

be

-LSB-

-RSB-

Our

notation

and

metamodel

extends

this

insight

by

making

the

conceptual

layering

that

exists

in

systems

explicit

Although

explicit

component

boundaries

are

maintained

in

Genesis

and

Avocaproduced

systems

this

is

not

mandatory

-LRB-

e.g.

components

could

be

macro-expanded

together

-RRB-

We

amplify

this

perspective

in

the

next

and

subsequent

sections

by

examining

features

of

component

implementations

General

Features

of

Component

Implementations

When

we

compared

components

of

Genesis

and

Avoca

we

were

astonished

at

the

similarity

of

their

organization

On

closer

inspection

the

commonalities

that

we

observed

would

need

to

be

present

in

any

concurrent

layered

system

In

this

section

we

examine

general

features

of

component

implementations

A

general

mechanism

by

which

concurrently

executing

components

communicate

is

presented

in

Section

In

Section

we

examine

how

individual

components

can

be

customized

how

algorithms

within

components

fit

together

and

how

algorithms

can

be

reused

A

Model

of

Component

Exteriors

Component

Operations

Every

component

C

presents

operations

-LCB-

Tn

Bm

-RCB-

as

its

interface

Calls

within

C

to

operations

external

to

C

-LRB-

whose

services

are

provided

by

other

components

-RRB-

are

-LCB-

dn

um

-RCB-

The

Ti

operations

are

top

operations

meaning

that

components

above

C

-LRB-

i.e.

components

whose

parameters

are

instantiated

by

C

-RRB-

can

call

Ti

for

lower-level

services

The

Bi

are

bottom

operations

meaning

components

below

C

-LRB-

i.e.

the

components

that

instantiate

C

's

parameters

-RRB-

can

call

Bi

for

higher-level

services

C

itself

may

call

external

operations

di

and

ui

The

di

are

down

calls

requesting

services

from

lower-level

components

and

the

ui

are

upcalls

requests

for

services

from

higher-level

components

-LSB-

-RSB-

Upcalls

arise

in

systems

that

receive

asynchronous

inputs

through

lower

level

components

and

that

usher

the

processing

of

these

inputs

up

through

the

system

Upcalls

are

common

in

networks

and

operating

systems

and

are

initiated

via

hardware

interrupts

external

calls

interface

operations

C

um

dn

Bm

Tn

As

a

general

rule

however

most

layered

systems

process

inputs

strictly

in

a

top-down

manner

so

that

bottom

operations

are

not

present

For

these

systems

m

m

Bottom

operations

are

also

absent

in

unparameterized

components

which

must

be

the

terminal

components

of

a

system

Component

Communication

Components

communicate

directly

by

calling

each

other

's

top

and

bottom

operations

For

nonconcurrent

or

nonreentrant

executions

component

communication

is

straightforward

In

the

case

of

concurrent

and

reentrant

executions

which

is

standard

for

networks

and

DBMSs

a

more

interesting

form

of

communication

is

used

In

order

for

several

distinct

executions

of

a

component

to

exist

simultaneously

a

component

must

be

reentrant

and

the

state

variables

of

each

execution

must

be

kept

in

a

separate

control

block

Control

blocks

not

only

provide

storage

for

component-specific

state

information

they

often

provide

the

means

-LRB-

e.g.

storage

-RRB-

by

which

data

and

results

of

computations

are

transmitted

from

one

component

to

another

Control

blocks

are

the

primary

data

conduits

or

pipelines

through

which

components

communicate

We

illustrate

these

ideas

with

examples

from

Genesis

and

Avoca

shortly

Component

communications

follow

a

standard

sequence

of

three

steps

-LRB-

-RRB-

Component

A

initiates

communication

with

component

B

by

creating

a

control

block

SB

for

B

This

is

done

by

calling

the

allocate-control-block

operation

of

B.

-LRB-

-RRB-

A

then

transmits

its

requests

or

data

via

calls

to

B

using

SB

as

a

parameter

of

every

call

B

either

writes

data

in

SB

for

A

's

consumption

or

vice

versa

depending

on

the

operation

-LRB-

-RRB-

A

terminates

communication

by

deallocating

SB

which

is

accomplished

by

calling

the

deallocate-control-block

operation

of

B.

A

process

can

create

and

reference

any

number

of

control

blocks

It

may

be

the

case

that

a

control

block

corresponds

to

the

state

variables

of

a

local

process

that

exists

only

within

a

single

component

-LSB-

-RSB-

How

processes

generally

fit

into

our

model

is

a

problem

that

we

are

now

investigating

An

Example

from

Genesis

Genesis

components

communicate

through

control

blocks

called

cursors

Let

F

be

a

FMAP

component

and

C

be

a

file

-LRB-

FMAP

-RRB-

cursor

The

interface

to

an

FMAP

component

includes

both

component

operations

and

cursor

operations

some

of

which

are

listed

below

Object

Operation

Semantics

___________________________________________________________________

FMAP

Component

C

-LRB-

F

-RRB-

component

F

allocates

a

file

cursor

C

-LRB-

F

R

-RRB-

F

partially

maps

abstract

relation

definition

R

to

its

concrete

counterpart

-LRB-

F

R

-RRB-

F

completes

the

mapping

of

R

by

filling

in

identifiers

of

foreign

relations

FMAP

Cursor

-LRB-

C

Q

L

-RRB-

initialize

cursor

C

for

the

retrieval

of

records

that

satisfy

predicate

Q

and

returning

fields

specified

in

list

L

adv

-LRB-

C

-RRB-

advance

cursor

C

to

next

qualified

record

acc

-LRB-

C

A

-RRB-

position

cursor

C

over

record

with

address

A

upd

-LRB-

C

-RRB-

update

record

referenced

by

cursor

C

del

-LRB-

C

-RRB-

delete

record

referenced

by

cursor

C

-LRB-

C

-RRB-

terminate

cursor

C

for

retrieval

-LRB-

C

-RRB-

deallocate

file

cursor

C

A

component

wishing

to

communicate

with

FMAP

component

F

-LRB-

-RRB-

allocates

a

file

cursor

C

by

calling

-LRB-

F

-RRB-

-LRB-

-RRB-

initializes

C

for

retrieval

via

-LRB-

C

Q

L

-RRB-

-LRB-

-RRB-

retrieves

records

one

at

a

time

by

calling

adv

-LRB-

C

-RRB-

-LRB-

-RRB-

terminates

the

retrieval

via

-LRB-

C

-RRB-

and

finally

-LRB-

-RRB-

deallocates

C

using

-LRB-

C

-RRB-

Allocation

and

initialization

are

separate

operations

as

a

cursor

can

be

initialized

any

number

of

times

once

it

is

allocated

FMAP

components

sit

atop

of

AMETHOD

-LRB-

access

method

-RRB-

components

and

communicate

via

file

cursors

AMETHOD

components

in

turn

sit

atop

NODE

-LRB-

logical

block

-RRB-

components

and

communicate

via

node

cursors

NODE

components

in

turn

sit

atop

BLOCK

components

and

communicate

via

block

cursors

and

so

on

-LRB-

see

Fig.

a

-RRB-

As

a

general

rule

when

the

top-most

component

of

a

system

creates

a

control

block

to

a

lower-level

component

the

lower-level

component

will

in

turn

create

a

control

block

to

components

beneath

it

and

those

components

beneath

them

and

so

on

causing

literally

a

top-to-bottom

wave

of

control

block

creations

The

overhead

for

multiple

control

block

creations

can

be

reduced

when

the

composition

order

of

components

is

fixed

-LRB-

i.e.

there

is

a

nonpermutable

stacking

of

layers

as

in

Figure

-RRB-

In

such

cases

it

is

possible

to

define

a

composite

control

block

that

is

the

union

of

all

the

control

blocks

that

would

have

been

allocated

in

a

top-to-bottom

creation

wave

Thus

in

a

single

and

efficient

operation

all

needed

control

blocks

can

be

allocated

-LRB-

or

deallocated

-RRB-

simultaneously

This

optimization

is

used

in

Genesis

Every

cursor

has

a

file

-LRB-

sub

-RRB-

cursor

a

node

-LRB-

sub

-RRB-

cursor

and

a

block

-LRB-

sub

-RRB-

cursor

inside

it

When

a

cursor

is

allocated/deallocated

all

of

its

lower-level

cursors

are

simultaneously

allocated/deallocated

-LRB-

Fig.

b

-RRB-

A

similar

BLOCK

BLOCK

NODE

NODE

AMETHOD

AMETHOD

FMAP

FMAP

-LRB-

b

-RRB-

cursor

-LRB-

a

-RRB-

Component

Component

Component

Component

Component

Component

Component

block

cursor

node

cursor

file

cursor

Component

file

cursor

node

cursor

block

cursor

Figure

Control

Block

Allocation

in

Genesis

optimization

is

used

for

components

that

are

symmetric

-LSB-

-RSB-

An

Example

from

Avoca

Avoca

components

communicate

through

control

blocks

called

session

objects

Components

are

called

protocol

objects

The

interface

to

an

ASYNC

component

for

example

presents

a

small

set

of

operations

on

protocol

and

session

objects

some

of

which

are

listed

below

Note

that

and

denote

protocol

objects

S

is

a

control

block

M

is

a

message

and

A

is

a

host

address

Object

Operation

Semantics

ASYNC

Component

S

open

-LRB-

A

-RRB-

creates

an

initialized

session

S

that

allows

to

send

messages

to

host

address

-LRB-

s

-RRB-

A

openenable

-LRB-

A

-RRB-

notifies

that

is

willing

to

accept

messages

addressed

to

A

demux

-LRB-

S

M

-RRB-

session

S

performs

upcall

to

to

deliver

message

M

to

ASYNC

Session

push

-LRB-

S

M

-RRB-

send

message

M

using

connection

S

pop

-LRB-

S

M

-RRB-

protocol

performs

upcall

on

session

S

to

deliver

message

M

to

S

close

-LRB-

S

-RRB-

protocol

closes

connection

and

destroys

its

reference

to

session

S

Suppose

protocol

wants

to

establish

a

connection

with

some

other

host

via

protocol

first

performs

an

open

-LRB-

A

-RRB-

on

with

the

address

A

of

the

host

to

create

a

session

S.

To

send

a

message

M

to

the

host

invokes

the

push

-LRB-

S

M

-RRB-

operation

When

the

remote

host

sends

a

reply

to

the

message

M

will

first

arrive

at

protocol

via

the

upcall

demux

-LRB-

S

M

-RRB-

where

S

is

a

session

object

that

is

delivering

the

incoming

message

to

As

there

is

almost

always

more

than

one

session

object

per

protocol

object

messages

are

demultiplexed

by

protocol

objects

to

the

correct

session

object

based

on

the

information

in

the

message

's

header

Thus

determines

that

S

is

the

appropriate

session

object

for

M

and

makes

the

upcall

pop

-LRB-

S

M

-RRB-

to

pass

the

message

to

S.

S

in

turn

passes

the

message

to

the

protocol

object

via

demux

-LRB-

S

M

-RRB-

and

so

on

Protocol

closes

the

connection

by

performing

a

close

-LRB-

S

-RRB-

operation

-LRB-

As

it

is

possible

for

multiple

higher-level

session

objects

to

share

a

common

lower-level

session

object

session

objects

are

reference

counted

and

are

not

always

destroyed

by

a

close

-LRB-

-RRB-

operation

-RRB-

As

a

general

rule

when

the

top-most

protocol

of

a

system

creates

a

session

to

a

lowerlevel

protocol

the

lower-level

protocol

will

in

turn

create

sessions

to

protocols

beneath

it

and

those

components

beneath

them

and

so

on

causing

literally

a

top-to-bottom

wave

of

session

creations

identical

to

cursor

creations

in

Genesis

In

Avoca

the

overhead

for

It

is

worth

noting

that

the

interface

to

SYNC

components

is

almost

identical

to

ASYNC

The

only

difference

is

that

push

-LRB-

S

M

M

-RRB-

returns

message

M

as

its

result

multiple

session

creations

is

not

a

critical

concern

Most

protocols

create

a

session

and

use

that

session

to

send

many

messages

so

the

overhead

for

session

creation

is

rarely

an

issue

Figure

shows

the

protocol

composition

tcp

-LSB-

ip

-LSB-

ethernet

-RSB-

-RSB-

along

with

created

session

objects

Some

user

-LRB-

or

other

protocol

-RRB-

performed

an

open

operation

on

the

TCP

component

and

received

a

TCP

session

with

which

it

will

communicate

with

the

party

whose

address

was

passed

to

TCP

as

a

parameter

to

open

TCP

in

turn

would

have

opened

the

IP

component

with

the

IP

portion

of

the

TCP

address

passed

to

it

The

IP

component

would

return

a

new

or

existing

session

which

the

TCP

session

uses

to

communicate

with

the

remote

machine

If

the

IP

session

was

not

reused

the

IP

component

would

have

to

invoke

the

open

operation

of

the

ethernet

component

and

an

ethernet

session

would

be

returned

ethernet

ip

session

session

Protocol

Protocol

Protocol

Ethernet

IP

TCP

session

tcp

Figure

Control

Block

Allocation

in

Avoca

A

Model

of

Component

Interiors

Every

operation

of

a

component

's

interface

is

implemented

by

one

or

perhaps

several

algorithms

Cataloging

these

algorithms

exposes

the

potentially

complex

internal

structure

of

components

Algorithm

catalogs

explain

how

variations

of

components

arise

in

practice

how

individual

components

may

be

customized

to

suit

a

particular

task

and

how

as

is

algorithm

reuse

can

be

realized

Algorithm

cataloging

is

not

appropriate

for

all

realms

Sometimes

algorithms

are

rarely

published

for

security

or

proprietary

reasons

Such

is

the

case

for

command-and-control

systems

Other

times

algorithms

within

components

are

so

intertwined

that

researchers

and

practitioners

make

no

attempt

to

separate

them

Such

is

the

case

with

network

protocols

-LSB-

-RSB-

The

database

domain

is

replete

with

realms

in

which

algorithm

cataloging

retrieval

algorithms

in

particular

is

possible

In

this

section

we

explain

algorithm

catalogs

and

algorithm

reuse

and

later

illustrate

their

utility

in

the

database

domain

Algorithm

Catalogs

There

can

be

any

number

of

algorithms

that

implement

an

operation

of

a

component

Let

OC

be

a

top

or

bottom

operation

of

component

C

and

let

S

be

a

control

block

The

j

algorithms

that

implement

OC

can

be

cataloged

as

rewrite

rules

of

the

form

OC

-LRB-

S

argk

-RRB-

-LRB-

S

argk

-LRB-

-RRB-

dn

-LRB-

-RRB-

-LRB-

-RRB-

um

-LRB-

-RRB-

-RRB-

-LRB-

S

argk

-LRB-

-RRB-

dn

-LRB-

-RRB-

-LRB-

-RRB-

um

-LRB-

-RRB-

-RRB-

algorithmj

-LRB-

S

argk

-LRB-

-RRB-

dn

-LRB-

-RRB-

-LRB-

-RRB-

um

-LRB-

-RRB-

-RRB-

where

is

read

is

realized

by

Every

algorithm

takes

the

arguments

-LRB-

S

argk

-RRB-

of

its

operation

OC

and

makes

any

number

of

down

calls

dn

and

upcalls

um

to

process

its

input

-LRB-

Note

that

control

block

S

is

present

as

a

parameter

if

OC

is

a

control

block

operation

-RRB-

When

components

are

composed

operation

calls

are

replaced

with

their

algorithms

in

the

obvious

way

-LSB-

-RSB-

Rewrite

rules

can

be

adorned

with

additional

information

Preconditions

are

an

example

Algorithms

without

preconditions

are

robust

as

they

will

always

work

no

matter

what

their

input

might

be

However

robust

algorithms

tend

to

be

inefficient

Faster

algorithms

often

exist

but

work

only

for

restricted

inputs

These

algorithms

are

nonrobust

In

general

robust

algorithms

are

interchangeable

while

nonrobust

algorithms

are

not

A

typical

software

design

strategy

is

to

use

a

nonrobust

algorithm

whenever

possible

-LRB-

because

of

its

performance

advantages

-RRB-

and

to

use

robust

algorithms

as

a

default

This

means

that

nonrobust

algorithms

typically

exist

only

in

the

presence

of

robust

algorithms

We

'll

consider

an

example

from

Genesis

shortly

Component

Customization

and

Algorithm

Reuse

There

are

many

possible

implementations

of

a

component

Stated

another

way

a

component

can

be

customized

for

a

particular

task

through

an

appropriate

selection

from

catalogs

of

one

or

more

algorithms

for

each

of

the

component

's

operations

Normally

only

one

-LRB-

robust

-RRB-

algorithm

is

selected

per

operation

However

it

is

possible

for

multiple

algorithms

to

be

chosen

Algorithm

reuse

occurs

when

the

same

algorithm

is

used

in

two

different

components

We

'll

see

an

example

of

algorithm

reuse

in

the

next

section

To

illustrate

the

issues

of

multiple

selections

query

optimization

in

databases

involves

the

evaluation

of

alternative

algorithms

for

processing

a

retrieval

operation

The

algorithm

The

elimination

of

explicit

layering

via

inline

expansion

which

replaces

an

operation

with

its

algorithmic

body

was

proposed

in

-LSB-

-RSB-

Inline

expansion

is

possible

in

systems

that

have

no

bottom

operations

because

unbounded

recursion

may

arise

-LRB-

e.g.

a

bottom

call

B

calls

a

top

call

T

which

in

turn

may

call

B

again

-RRB-

Although

recursion

is

definitely

bounded

at

execution

time

the

amount

of

recursion

can

not

be

determined

at

compile

time

Thus

the

simple

idea

of

eliminating

explicit

layering

via

inline

substitution

does

not

always

work

Most

operations

of

a

component

share

the

same

state

data

By

designing

all

operations

to

work

off

of

a

standard

representation

of

state

information

the

compatibility

of

different

implementations

of

operations

is

ensured

whose

preconditions

are

satisfied

and

that

is

the

cheapest

-LRB-

as

estimated

by

cost

functions

-RRB-

for

a

given

situation

is

the

one

chosen

for

execution

At

first

glance

one

might

want

to

include

all

algorithms

on

the

presumption

that

the

best

algorithm

will

surely

be

used

in

processing

a

retrieval

Generally

this

is

a

bad

idea

as

the

overhead

for

query

optimization

increases

with

the

addition

of

more

algorithms

From

our

experience

selecting

few

-LRB-

perhaps

two

or

three

-RRB-

algorithms

per

operation

is

sufficient

An

Example

From

Genesis

Recall

that

DLANG

is

the

realm

of

nonprocedural

data

languages

Every

DLANG

component

offers

a

nonprocedural

retrieval

statement

-LRB-

For

example

SQL

uses

the

SELECT-FROM-WHERE

statement

QUEL

and

GEM

-LSB-

-RSB-

use

RETRIEVE-WHERE

-RRB-

These

statements

are

translated

into

optimized

expressions

called

access

plans

that

are

compositions

of

retrieval

and

join

operations

over

base

relations

In

this

section

we

catalog

implementations

of

DLANG

retrieval

statements

Expanded

discussions

on

this

subject

are

given

in

-LSB-

-RSB-

and

a

tutorial

on

query

optimization

is

presented

in

-LSB-

-RSB-

Let

R

be

a

relational

query

-LRB-

As

readers

will

soon

see

the

particular

language

used

to

express

R

does

n't

matter

-RRB-

The

most

abstract

description

of

query

processing

in

a

relational

DBMS

is

captured

by

the

following

rewrite

which

maps

R

to

its

result

R

Eval

-LRB-

-LRB-

R

-RRB-

-RRB-

-LRB-

-RRB-

is

the

query

optimization

operation

that

transforms

R

into

an

executable

expression

E.

Eval

-LRB-

E

-RRB-

executes

E.

Different

relational

DBMSs

realize

-LRB-

-RRB-

in

different

ways

-LRB-

-RRB-

can

be

decomposed

into

a

composition

of

three

suboperations

-LRB-

R

-RRB-

-LRB-

-LRB-

-LRB-

R

-RRB-

-RRB-

-RRB-

-LRB-

-RRB-

RfiG

maps

a

relational

query

R

to

a

query

graph

G

-LSB-

-RSB-

GfiG

maps

a

query

graph

without

semijoin

operations

to

one

that

does

-LRB-

which

is

the

classical

means

of

optimizing

distributed

queries

-LSB-

-RSB-

-RRB-

and

-LRB-

-RRB-

GfiE

maps

query

graphs

to

executable

expressions

In

the

following

paragraphs

we

catalog

algorithms

for

each

of

these

suboperations

has

many

implementations

one

for

every

data

language

-LRB-

R

-RRB-

-LRB-

R

-RRB-

SQL

language

-LSB-

-RSB-

-LRB-

R

-RRB-

QUEL

language

-LSB-

-RSB-

-LRB-

R

-RRB-

GEM

language

-LSB-

-RSB-

The

actual

choice

of

algorithm

is

component-specific

-LRB-

e.g.

-LRB-

R

-RRB-

is

used

for

components

that

present

an

SQL

interface

-LRB-

R

-RRB-

is

used

for

components

that

present

a

QUEL

interface

and

so

on

-RRB-

and

algorithms

are

data

language

independent

A

partial

catalog

of

algorithms

for

each

operation

is

shown

below

-LRB-

G

-RRB-

G

identity

no

optimization

-LRB-

G

-RRB-

algorithm

-LSB-

-RSB-

Bc

-LRB-

G

-RRB-

Bernstein

Chiu

algorithm

-LSB-

-RSB-

Yol

-LRB-

G

-RRB-

Yu

et

al.

algorithm

-LSB-

-RSB-

-LRB-

G

-RRB-

-LRB-

G

-RRB-

System

R

algorithm

-LSB-

-RSB-

-LRB-

G

-RRB-

University

Ingres

algorithm

-LSB-

-RSB-

Exodus

-LRB-

G

RS

-RRB-

Exodus

rule

optimizer

-LSB-

-RSB-

RS

is

the

rule

set

From

the

above

catalogs

it

is

easy

to

see

how

different

implementations

of

components

can

arise

A

set

of

potential

implementations

is

defined

by

equation

-LRB-

-RRB-

to

be

the

cross

product

of

the

catalogs

of

algorithms

that

implement

the

and

operations

Just

from

the

few

that

are

listed

distinct

implementations

of

can

be

written

Before

we

consider

specific

combinations

we

again

remind

readers

why

composition

is

possible

We

have

imposed

interface

standards

-LRB-

which

includes

standard

data

representations

-RRB-

on

the

class

of

relational

query

optimization

algorithms

in

order

to

make

them

plugcompatible

and

interchangeable

This

requires

algorithms

to

be

rewritten

to

this

standard

one

can

not

simply

copy

algorithms

from

an

existing

system

with

ad

hoc

interfaces

and

nonstandard

data

representations

and

expect

them

to

work

Specific

instances

of

-LRB-

-RRB-

correspond

to

query

processing

algorithms

of

commercial

DBMSs

The

data

model/data

language

component

of

-LSB-

-RSB-

uses

the

following

implementation

of

-LRB-

-RRB-

-LRB-

R

-RRB-

-LRB-

-LRB-

R

-RRB-

-RRB-

That

is

uses

-RRB-

to

map

SELECT-FROM-WHERE

queries

to

query

graphs

-RRB-

the

identity

function

G

for

its

reducing

phase

algorithm

and

-RRB-

the

algorithm

to

map

query

graphs

to

executable

expressions

Many

other

combinations

are

possible

A

DBMS

with

a

SQL

front-end

that

uses

the

University

INGRES

joining

phase

algorithm

and

the

reducing

phase

algorithm

is

Parameters

that

are

not

shown

in

these

and

other

catalogs

of

this

section

is

the

list

of

cost

functions

-LRB-

provided

by

lower-level

components

-RRB-

that

direct

the

optimization

of

G

by

estimating

the

cost

of

performing

join

semijoin

and

retrieval

operations

on

base

relations

We

chose

not

to

include

them

here

simply

because

it

clutters

our

discussion

unnecessarily

-LRB-

R

-RRB-

-LRB-

-LRB-

-LRB-

R

-RRB-

-RRB-

-RRB-

Another

possibility

is

a

DBMS

with

an

SQL

front-end

that

uses

the

rule-based

EXODUS

joining

phase

algorithm

with

the

reducing

phase

algorithm

-LRB-

R

-RRB-

Exodus

-LRB-

-LRB-

-LRB-

R

-RRB-

-RRB-

RS

-RRB-

Two

final

points

First

recall

that

algorithm

reuse

occurs

when

the

same

algorithm

is

used

in

two

different

expressions

The

above

three

examples

show

the

reuse

of

the

algorithm

Second

it

is

worth

noting

that

almost

all

reducing

phase

algorithms

are

nonrobust

typically

reducing

phase

algorithms

can

only

process

tree

graphs

-LSB-

-RSB-

As

mentioned

earlier

nonrobust

algorithms

can

be

paired

with

robust

algorithms

to

form

new

robust

algorithms

A

new

reducing

phase

algorithm

-LRB-

G

-RRB-

is

a

composition

of

the

Bernstein

and

Chiu

algorithm

-LRB-

which

works

only

on

tree

graphs

-RRB-

and

the

identity

algorithm

-LRB-

which

works

on

any

graph

-RRB-

-LRB-

G

-RRB-

G

otherwise

Bc

-LRB-

G

-RRB-

if

G

is

a

tree

As

the

above

example

shows

the

set

of

implementations

is

actually

much

larger

than

the

cross

product

of

the

and

catalogs

In

analyzing

existing

DBMSs

one

finds

many

implementations

of

which

are

cleanly

modeled

by

algorithm

catalogs

This

is

how

we

have

explained

variations

in

component

implementations

Recap

Components

are

not

monolithic

but

are

suites

of

algorithms

that

translate

data

and

operations

a

component

's

abstract

interface

to

data

and

operations

of

its

concrete

interface

When

component

executions

are

not

concurrent

or

reentrant

components

communicate

by

directly

calling

each

others

operations

In

the

case

of

concurrent

executions

the

primary

parameter

of

an

operation

is

a

control

block

which

is

a

run-time

object

that

contains

the

state

variables

of

an

execution

within

a

component

Distinct

control

blocks

are

used

maintain

the

state

variables

of

different

executions

of

the

same

component

There

are

many

ways

to

implement

operations

of

a

component

We

have

presented

a

simple

and

informal

model

for

cataloging

the

algorithms

of

an

operation

Catalogs

explain

how

rich

variations

in

component

implementations

arise

through

algorithm

compositions

and

how

as

is

algorithm

reuse

is

achieved

In

the

next

section

we

explain

how

these

ideas

can

be

integrated

with

recognized

concepts

of

object-oriented

design

Integration

with

Object-Oriented

Design

Concepts

Object-oriented

design

concepts

are

necessary

but

not

sufficient

to

realize

large-scale

reuse

-LRB-

i.e.

component

reuse

-RRB-

Nor

are

they

sufficient

to

explain

the

design

technology

and

framework

that

we

have

outlined

in

the

previous

sections

In

this

section

we

use

the

ER

model

to

review

the

basic

ideas

of

object-oriented

software

design

-LRB-

We

do

not

depend

on

a

specific

variant

of

the

ER

model

but

only

its

core

concepts

-RRB-

We

then

explain

how

our

work

transcends

these

ideas

by

presenting

specific

extensions

that

are

needed

to

explain

the

design

concepts

of

Genesis

and

Avoca

We

also

show

how

frameworks

an

important

OO

design

concept

play

a

role

in

our

model

We

reinforce

our

discussions

with

specific

examples

taken

from

the

Genesis

and

Avoca

prototypes

Layered

Software

Designs

and

Transformations

Object-Oriented

Designs

A

primary

result

of

an

object-oriented

design

is

an

object

model

or

schema

that

defines

the

classes

of

objects

of

an

application

and

their

interrelationships

-LSB-

-RSB-

Associated

with

each

class

is

the

set

of

operations

that

can

be

performed

on

its

objects

plus

the

set

of

object

attributes

Object

models

are

depicted

by

class

diagrams

where

classes

are

nodes

and

edges

are

relationships

We

draw

class

diagrams

as

ER

diagrams

although

any

comparable

notation

will

do

Figure

depicts

three

different

object

models

TA

TB

TC

as

ER

diagrams

-RCB-

-LCB-

class

F

B

-LRB-

c

-RRB-

Model

TC

-LRB-

b

-RRB-

Model

TB

-LRB-

a

-RRB-

Model

TA

G

Legend

A

C

D

K

J

I

H

E

relationship

Figure

Class

Diagrams

in

ER

Notation

Layered

or

Hierarchical

Designs

A

layered

or

hierarchical

software

system

is

designed

level

by

level

At

each

level

there

is

an

object

model

that

defines

the

classes

objects

and

operations

that

the

next

higher

level

may

reference

-LRB-

In

systems

with

upcalls

the

next

lower

level

may

reference

these

operations

as

well

-RRB-

The

object

model

at

the

topmost

level

of

a

system

is

the

external

model

and

the

object

model

at

the

inner-most

level

is

the

internal

model

Every

level

provides

a

virtual

machine

in

which

objects

and

operations

of

the

next

higher

level

-LRB-

or

next

lower

level

-RRB-

are

For

simplicity

we

assume

communication

is

always

between

adjacent

levels

Figure

shows

-LRB-

a

portion

of

-RRB-

a

hierarchical

design

where

object

model

TA

resides

on

a

level

immediately

above

that

of

object

model

TB

GF

lower

levels

higher

levels

transformation

TAB

TA

B

C

A

TB

D

E

Figure

Two

levels

of

a

Hierarchical

System

Design

It

is

worth

noting

that

the

ideas

of

hierarchical/layered

system

design

were

well-understood

over

two

decades

ago

-LSB-

-RSB-

Astonishingly

the

idea

of

stratified

designs

and

layered

systems

is

virtually

absent

in

contemporary

object-oriented

literature

A

primary

contribution

of

object-orientation

to

the

design

of

hierarchical

systems

is

that

an

object

model

precisely

defines

both

the

operations

and

objects

of

a

level

-LRB-

virtual

machine

-RRB-

Earlier

attempts

did

not

fully

recognize

the

role

of

objects

that

were

visible

at

each

level

thus

making

design

encapsulations

difficult

-LRB-

if

not

impossible

-RRB-

to

achieve

Our

model

places

no

restrictions

on

the

use

of

static

externally

preceptible

variables

The

software

design

problem

to

be

solved

will

dictate

whether

or

not

such

variables

are

needed

and

can

be

used

Although

limiting

communication

between

adjacent

levels

seems

restrictive

in

fact

it

is

not

If

object

O

on

level

i

needs

to

directly

address

an

object

K

on

level

i-j

it

is

easy

enough

to

have

object

models

for

levels

through

i-j

to

include

an

object

K

which

simply

transmits

calls

to

its

counterpart

on

the

level

below

eventually

terminating

in

calls

on

object

K

on

level

i-j

As

noted

in

-LSB-

-RSB-

defining

transmit

calls

as

macros

and

expanding

inline

through

a

sequence

of

layers

could

possibly

result

in

a

single

instruction

It

is

the

system

design

that

is

hierarchical

not

its

implementation

Macro-expanding

calls

through

levels

is

in

fact

an

important

implementation

and

modeling

technique

used

in

Genesis

Transformations

A

transformation

is

a

mapping

between

adjacent

levels

of

a

system

it

is

the

correspondence

of

an

abstract

or

higher-level

object

model

with

a

concrete

or

lower-level

object

model

A

transformation

defines

all

objects

and

all

operations

-LRB-

i.e.

everything

-RRB-

of

an

abstract

model

in

terms

of

objects

and

their

operations

of

a

concrete

model

Figure

depicts

transformation

TAB

which

maps

abstract

model

TA

to

concrete

model

TB

As

mentioned

above

hierarchical

systems

can

be

defined

as

a

progression

of

increasingly

more

concrete

object

models

starting

from

the

external

model

and

ending

with

the

internal

model

Equivalently

a

hierarchical

system

can

be

defined

by

-LRB-

-RRB-

the

external

object

model

and

-LRB-

-RRB-

the

progression

of

transformations

that

convert

the

external

model

into

the

internal

model

The

advantage

of

using

transformations

becomes

apparent

when

one

recognizes

that

transformations

are

implemented

by

components

-LRB-

or

layers

-RRB-

The

upper

interface

of

a

component

is

the

transformation

's

abstract

object

model

and

the

lower

interface

is

the

transformation

's

concrete

object

model

A

progression

of

transformations

is

a

composition

of

components

-LRB-

a

stacking

of

layers

-RRB-

Realms

and

Parameterized

Components

All

components

that

implement

the

same

object

model

constitute

a

realm

Suppose

c

is

a

component

of

the

realm

whose

object

model

is

TA

Suppose

further

that

c

transforms

TA

into

concrete

model

TB

as

in

Figure

Then

c

has

a

parameter

b

TB

-LRB-

i.e.

c

-LSB-

b

TB

-RSB-

-RRB-

to

specify

how

model

TB

is

implemented

More

generally

the

concrete

object

model

to

which

a

component

c

maps

may

actually

be

the

union

of

several

disjoint

concrete

models

each

of

which

has

its

own

realm

of

implementing

components

Every

concrete

model

T

requires

c

to

have

a

parameter

of

type

T.

Figure

shows

a

component

e

-LSB-

x

TB

y

TC

-RSB-

of

type

TA

that

has

two

parameters

one

of

type

TB

and

another

of

type

TC

-LRB-

Equivalently

component

e

-LSB-

x

y

-RSB-

maps

its

abstract

object

model

of

type

TA

to

concrete

object

models

of

types

TB

and

TC

-RRB-

e

transformation

K

J

I

TA

B

C

A

TB

D

E

H

F

G

TC

Figure

Parameterized

Component

e

-LSB-

x

TB

y

TC

-RSB-

TA

Note

that

Figure

only

shows

classes

and

relationships

that

belong

to

component

interfaces

There

can

be

additional

classes

and

relationships

hidden

within

component

implementations

that

connect

abstract

classes

to

concrete

classes

-LRB-

e.g.

a

relationship

that

connects

abstract

class

C

to

concrete

class

H

-RRB-

Such

details

often

exist

but

are

not

shown

in

Figure

Control

Blocks

Recall

from

Section

that

components

communicate

directly

with

each

other

or

when

there

are

concurrent

executions

indirectly

through

control

blocks

In

a

concurrent

setting

a

component

class

and

at

least

one

control

block

class

must

appear

in

every

object

model

that

defines

a

component

interface

In

Figures

we

have

diagrammatically

distinguished

component

classes

control

block

classes

and

other

classes

as

other

classclass

component

class

control

block

Thus

in

object

model

TA

class

B

is

the

component

class

and

class

C

is

a

control

block

class

Other

classes

of

an

object

model

in

this

case

A

and

D

define

additional

information

that

is

to

be

exchanged

between

components

As

a

general

rule

operations

on

components

-LRB-

either

direct

or

indirect

-RRB-

involve

a

translation

of

a

web

of

abstract

objects

into

a

web

of

concrete

objects

We

'll

illustrate

examples

from

Genesis

and

Avoca

shortly

Frameworks

In

object-oriented

design

a

class

can

have

multiple

implementations

This

is

accomplished

by

defining

an

abstract

class

which

only

specifies

the

class

interface

with

multiple

concrete

subclasses

each

providing

a

different

implementation

of

the

abstract

class

A

set

of

abstract

classes

with

their

concrete

classes

defines

a

framework

which

is

an

important

organizational

concept

in

the

design

of

families

of

software

systems

-LSB-

-RSB-

Object

models

that

define

realm

interfaces

are

implemented

by

frameworks

In

the

most

general

case

every

class

of

an

object

model

is

an

abstract

class

Each

component

is

implemented

by

a

special

concrete

class

for

each

abstract

class

in

the

model

The

number

of

concrete

classes

per

abstract

class

equals

the

number

of

components

in

a

library

Thus

if

there

are

A

abstract

classes

in

a

realm

R

's

object

model

and

there

are

N

components

in

R

's

library

then

each

of

the

A

abstract

classes

will

have

N

distinct

concrete

subclasses

one

for

each

component

of

realm

R

More

typically

not

all

classes

of

an

object

model

are

abstract

Some

classes

have

a

single

implementation

that

is

shared

-LRB-

reused

-RRB-

across

many

different

components

-LRB-

This

is

an

example

of

as

is

class

reuse

-RRB-

Frameworks

in

Genesis

and

Avoca

fit

this

more

common

situation

An

Example

From

Genesis

The

FMAP

Interface

Recall

from

Section

that

FMAP

is

the

realm

of

file

mapping

components

Each

transforms

an

abstract

file

into

one

or

more

concrete

files

Consequently

FMAP

components

are

symmetric

Object

Model

Figure

shows

the

object

model

interface

of

FMAP

components

Fmap

Attribute

Relation

Buffer

Trace

Clause

Query

Cursor

Figure

The

FMAP

Object

Model

Interface

Instances

of

the

Fmap

class

correspond

to

FMAP

components

Among

Fmap

operations

are

component

initialization

schema

mapping

and

the

creation

and

linking

of

cursors

-LRB-

via

link

-RRB-

Cursors

are

the

control

blocks

of

FMAP

components

When

a

database

schema

is

compiled

there

is

an

object

in

the

Relation

class

for

every

relation

Further

each

relation

type

is

defined

by

a

set

of

attributes

-LRB-

i.e.

data

fields

-RRB-

where

each

attribute

is

an

object

of

the

Attribute

class

The

link

between

relation

objects

and

their

attribute

objects

is

Every

cursor

is

an

object

of

the

Cursor

class

and

is

bound

to

single

relation

-LRB-

-RRB-

Every

cursor

is

also

bound

to

a

retrieval

predicate

called

a

Query

object

-LRB-

-RRB-

and

an

attribute

projection

list

called

an

object

-LRB-

-RRB-

A

query

specifies

the

set

of

tuples

of

the

designated

relation

that

the

cursor

may

reference

and

the

into-list

specifies

the

attribute

values

of

these

tuples

that

are

to

be

retrieved

and/or

modified

A

Query

object

is

a

tree

of

Clause

objects

where

a

clause

is

a

boolean

connective

or

a

terminal

of

the

form

-LRB-

attribute

rel-op

value

-RRB-

A

tree

of

clauses

is

specified

by

links

and

Since

a

tuple

may

be

variable-length

a

special

object

is

needed

at

run-time

to

reference

the

offset

of

an

attribute

in

a

particular

tuple

Such

objects

are

called

traces

Every

terminal

clause

references

a

Trace

object

-LRB-

-RRB-

A

Trace

object

in

turn

references

its

corresponding

attribute

definition

-LRB-

-RRB-

and

a

buffer

in

which

a

value

of

that

attribute

can

be

placed

-LRB-

-RRB-

An

into-list

object

is

simply

a

list

of

into-list

entries

-LRB-

IL-entrys

-RRB-

each

of

which

identifies

an

attribute

that

is

to

be

retrieved

and/or

modified

A

list

of

is

achieved

by

links

and

Traces

are

used

by

into-list

entries

to

reference

attributes

-LRB-

-RRB-

-LRB-

Note

that

the

object

model

suggests

that

traces

-LRB-

and

their

buffers

-RRB-

can

be

shared

by

and

Clause

objects

In

fact

this

is

commonly

done

-RRB-

Transformation

The

object

model

of

Figure

serves

both

as

the

abstract

and

concrete

interface

of

symmetric

FMAP

components

Consider

the

-LSB-

d

FMAP

-RSB-

component

which

maps

abstract

relations

-LRB-

files

-RRB-

to

compressed

concrete

relations

Let

C

be

a

cursor

Q

be

a

query

and

IL

be

an

into-list

-LRB-

Remember

that

Q

is

connected

to

a

web

of

Clause

Trace

and

Buffer

objects

that

define

a

selection

predicate

and

IL

is

connected

to

a

web

of

Trace

and

Buffer

objects

that

define

an

into

list

-RRB-

When

receives

the

operation

-LRB-

C

Q

IL

-RRB-

it

initializes

C

to

range

over

all

tuples

that

satisfy

predicate

Q

and

that

IL

is

the

list

of

attributes

for

qualified

records

whose

values

are

to

be

retrieved

and/or

modified

The

initialization

is

accomplished

by

mapping

C

Q

and

IL

to

their

concrete

counterparts

C

Q

and

IL

The

translation

is

simple

C

is

bound

to

the

corresponding

compressed

relation

Q

is

and

IL

references

the

compressed

byte

string

that

contains

the

compressed

image

of

the

tuple

Both

Q

and

IL

are

webs

of

objects

Once

initialization

has

taken

place

abstract

tuples

can

be

retrieved

via

the

operation

adv

-LRB-

C

-RRB-

A

retrieval

of

an

abstract

tuple

translates

into

the

retrieval

of

compressed

concrete

tuple

via

adv

-LRB-

C

-RRB-

a

decoding

application

of

predicate

Q

to

the

decoded

tuple

and

if

Q

is

satisfied

the

unencoded

attributes

of

IL

are

returned

Framework

Notes

All

classes

in

Figure

with

the

exception

of

Buffer

Trace

Query

and

are

abstract

Thus

these

mentioned

classes

have

a

single

implementation

that

is

shared

-LRB-

reused

-RRB-

by

all

FMAP

components

An

Example

from

Avoca

The

ASYNC

Interface

Recall

from

Section

that

ASYNC

is

the

realm

of

asynchronous

protocols

that

transform

abstract

messages

into

one

or

more

concrete

messages

Generally

ASYNC

components

are

symmetric

Object

Model

Figure

shows

the

object

model

interface

to

ASYNC

components

Message

SessionProtocol

Figure

The

ASYNC

Object

Model

Interface

Protocol

objects

are

instances

of

the

Protocol

class

Protocol

objects

create

Session

objects

which

are

the

control

blocks

of

ASYNC

protocols

and

interconnect

them

via

link

This

connection

is

needed

to

process

the

bottom

operation/upcall

demux

-LRB-

-RRB-

which

routes

incoming

messages

to

the

appropriate

session

-LRB-

Routing

is

accomplished

by

looking

at

the

message

header

and

extracting

a

protocol-specified

key

that

identifies

the

recipient

session

-RRB-

A

Session

object

may

be

linked

to

one

or

more

packets

-LRB-

via

-RRB-

which

are

instances

of

the

Message

class

A

Message

object

consists

of

headers

and

data

-LRB-

and

-RRB-

Most

traditional

protocol

implementations

support

a

notion

of

header

encapsulation

in

which

the

header

of

a

lowerlevel

protocol

is

considered

to

be

indistinguishable

from

message

data

A

key

point

of

Avoca

is

that

for

both

efficiency

and

software

engineering

reasons

segregating

header

and

data

information

in

any

given

message

is

a

good

idea

The

addition

and

removal

of

message

headers

obeys

a

stack

discipline

and

therefore

message

headers

can

be

stored

efficiently

in

a

object

using

traditional

stack

operations

Message

trailers

information

added

to

the

tail

of

a

message

are

rare

and

can

be

safely

ignored

The

most

common

manipulation

of

message

data

is

to

fragment

large

messages

into

smaller

messages

and

to

reassemble

the

fragments

into

a

single

message

Therefore

message

data

is

stored

in

one

or

more

Buffer

objects

that

are

linked

together

as

a

directed-acyclic

graph

-LRB-

-RRB-

The

operations

required

to

support

typical

message

data

manipulations

are

essentially

identical

to

those

of

standard

string

manipulation

Note

that

the

object

model

suggests

that

different

message

objects

could

share

the

same

buffers

and

header

stacks

Such

sharing

is

actually

critical

to

the

efficiency

of

protocol

code

as

each

copy

made

of

a

long

message

can

easily

halve

the

performance

of

a

protocol

suite

The

Message

class

in

effect

is

a

message

abstraction

that

supports

both

stack-based

header

manipulation

and

string-based

data

manipulation

These

abstractions

sometimes

interfere

with

one

another

For

example

when

a

large

message

is

fragmented

the

first

fragment

uses

the

original

header

while

subsequent

fragments

must

have

an

empty

header

stack

created

for

them

Also

in

traditional

protocols

such

as

IP

data

is

sometimes

found

in

the

header

and

headers

are

sometimes

found

in

the

buffer

DAG

Therefore

it

is

important

that

one

be

able

to

move

information

between

header

stack

and

buffer

representations

efficiently

Transformation

The

object

model

of

Figure

is

both

the

abstract

and

concrete

interface

of

symmetric

ASYNC

components

Consider

the

blast

-LSB-

x

ASYNC

-RSB-

component

which

fragments

large

messages

Let

S

be

a

session

object

and

let

M

be

a

Message

object

which

is

connected

to

a

object

and

a

web

of

Buffer

objects

When

the

blast

protocol

receives

the

operation

push

-LRB-

S

M

-RRB-

which

means

send

message

M

to

the

destination

of

S

concrete

message

objects

are

created

that

represent

fragments

of

M.

For

efficiency

buffers

of

M

are

shared

with

their

message

fragments

As

stated

above

new

header

stacks

are

created

for

each

fragment

message

except

for

the

first

A

blast

header

is

added

to

each

message

and

a

copy

of

the

resulting

message

is

saved

for

possible

retransmission

Each

concrete

message

is

sent

via

session

S

that

is

the

concrete

counterpart

of

S

The

peer

blast

component

receives

each

concrete

message

via

a

demux

-LRB-

-RRB-

upcall

and

examines

its

header

to

determine

the

correct

session

for

reassembly

and

then

invokes

that

session

's

pop

-LRB-

-RRB-

operation

The

selected

session

assembles

the

transmitted

concrete

messages

by

stripping

the

blast

header

from

each

message

fragment

and

concatenating

fragments

together

Once

the

entire

message

has

been

formed

the

session

passes

the

message

to

the

next

protocol

layer

-LRB-

a

pointer

to

which

is

stored

in

every

sessions

local

state

-RRB-

by

invoking

that

protocol

's

demux

-LRB-

-RRB-

operation

These

steps

are

repeated

for

each

layer

in

the

protocol

hierarchy

Framework

Notes

Only

the

Protocol

and

Session

classes

in

Figure

are

abstract

The

Message

Map

Buffer

and

classes

have

single

implementations

that

are

used

by

all

ASYNC

components

Recap

Hierarchical

or

layered

systems

are

designed

level-by-level

Each

level

is

designed

independently

of

adjacent

levels

and

is

defined

by

an

object

model

whose

classes

objects

and

operations

comprise

that

level

's

virtual

machine

Components

are

mappings

-LRB-

transformations

-RRB-

between

adjacent

levels

Components

that

implement

the

same

object

model

belong

to

the

same

realm

Because

their

interfaces

are

the

same

components

within

a

realm

are

plug-compatible

and

interchangeable

Contemporary

object-oriented

design

techniques

are

basically

two-dimensional

they

concentrate

on

developing

a

single

object

model

which

can

be

drawn

on

a

flat

-LRB-

twodimensional

-RRB-

surface

that

explains

an

application

Our

work

recognizes

and

extends

concepts

of

early

software

engineering

pioneers

-LSB-

-RSB-

that

software

design

techniques

must

be

extended

by

another

dimension

in

order

to

account

for

layering

in

hierarchical

systems

Where

our

work

departs

from

known

results

are

the

concepts

of

realms

typeexpressions

and

the

role

of

interface

standardization

Other

Issues

and

Related

Work

The

scope

of

this

paper

precludes

us

from

addressing

many

of

the

issues

that

are

of

concern

to

software

engineers

In

this

section

we

briefly

discuss

a

variety

of

topics

that

were

not

considered

previously

but

are

important

for

understanding

our

results

Model

Limitations

Ad

Hoc

Layering

We

believe

our

model

is

generally

applicable

to

a

broad

class

of

hierarchical

software

systems

However

there

may

be

many

systems

with

ad

hoc

layering

that

are

not

expressible

In

such

systems

components

result

from

mixtures

of

functional

and

object-oriented

decompositions

and

do

not

conform

to

the

definition

of

a

component

in

our

model

We

are

not

interested

in

capturing

ad

hoc

layering

nor

do

we

believe

our

model

can

be

used

to

do

so

Our

research

concentrates

on

a

subset

of

layered

systems

for

which

layer/component

composability

is

possible

We

have

outlined

properties

of

these

systems

and

without

these

properties

we

fail

to

see

how

layer

composability

will

work

Much

of

our

work

in

domain

modeling

incidentally

has

been

to

redraw

layer

boundaries

in

ad

hoc

system

designs

so

that

our

resulting

layers/components

are

in

fact

composable

Thus

we

feel

that

our

model

can

capture

system

functionality

but

not

ad

hoc

layering

Inheritance

Our

model

imposes

the

constraint

that

all

components

of

a

realm

must

have

exactly

the

same

interface

and

that

a

component

belongs

to

precisely

one

realm

This

may

be

too

restrictive

and

that

an

inheritance

lattice

which

relates

OOVM

interfaces

-LRB-

and

their

realms

-RRB-

may

be

needed

For

example

suppose

the

OOVM

of

realm

is

a

specialization

of

the

OOVM

for

realm

-LRB-

That

is

presents

additional

functions

possibly

even

additional

classes

as

part

of

its

OOVM

interface

-RRB-

Then

certainly

a

component

of

could

be

used

where

ever

a

component

of

is

used

Such

an

addition

we

believe

would

allow

us

to

express

the

class

of

hierarchical

systems

described

by

Habermann

Flon

and

Cooprider

-LSB-

-RSB-

Multiple

Standards

Consensus

is

needed

for

interface

standards

As

both

of

our

projects

relied

on

a

small

group

of

designers

consensus

was

n't

difficult

In

general

however

we

would

expect

independent

design

teams

would

arrive

at

different

standards

Consensus

and

component

interoperability

would

then

become

problematic

Even

if

there

is

general

agreement

on

the

operations

that

need

to

be

supported

there

can

be

a

choice

among

internal

data

representations

to

use

It

is

certainly

possible

for

different

representations

to

impact

performance

and

choosing

among

alternative

representations

should

be

one

of

the

design

decisions

in

specifying

a

target

system

Other

Features

Our

model

does

not

explicitly

show

how

processes

fit

into

a

building-block

framework

Nor

does

our

model

explain

how

exceptions

are

handled

how

distributed

systems

impact

reusable

software

designs

or

how

parallelism

can

be

exploited

Avoca

does

not

compose

components

dynamically

but

Genesis

does

-LRB-

during

query

processing

-RRB-

Our

model

is

geared

to

the

static

composition

of

components

and

this

may

be

unnecessarily

restrictive

Furthermore

all

components

in

Genesis

and

Avoca

are

hand-crafted

It

is

clear

from

our

experience

that

there

is

an

enormous

similarity

among

components

within

a

realm

This

raises

the

possibility

that

software

generators

could

be

created

to

simplify

or

automate

the

development

of

large

numbers

of

components

Thus

the

distinction

we

have

made

between

realms

and

their

libraries

may

have

limited

applicability

All

of

these

topics

are

subject

for

further

research

Related

Research

Modular

DBMS

Software

The

first

layered

architecture

to

gain

popularity

in

databases

was

the

three-level

ANSI/SPARC

model

-LSB-

-RSB-

Mappings

between

levels

-LRB-

the

external

conceptual

and

internal

-RRB-

were

layers

whose

implementation

it

was

claimed

could

be

altered

independently

To

our

knowledge

no

DBMS

ever

implemented

the

ANSI/SPARC

architecture

it

was

a

paper

design

that

gave

few

clues

about

how

to

build

actual

database

systems

Furthermore

ANSI/SPARC

layers

were

effectively

monolithic

leaving

little

opportunity

for

exploring

and

achieving

component

reuse

More

recently

research

on

extensible

database

systems

has

lead

to

more

modular

DBMS

architectures

However

only

Genesis

and

Starburst

-LSB-

-RSB-

were

based

on

formal

models

of

DBMS

software

The

Starburst

model

deals

with

rule-based

query

optimization

which

essentially

is

a

model

of

what

we

have

called

algorithm

reuse

The

optimizer

glues

algorithms

together

to

form

access

plans

However

there

is

no

notion

of

components

layers

or

classes

in

the

model

itself

How

algorithms

are

defined

how

they

are

related

-LRB-

if

at

all

-RRB-

to

components

is

unclear

Starburst

supports

attachments

that

are

similar

to

but

less

powerful

than

Genesis

components

-LSB-

-RSB-

Attachments

are

not

part

of

the

optimizer

model

Modular

Network

Software

Outside

of

the

networking

community

the

ISO

layer

model

-LSB-

-RSB-

is

often

cited

as

an

example

of

the

application

of

modular

design

to

production

systems

Inside

the

networking

community

the

ISO

model

is

the

root

cause

of

the

rejection

of

layering

as

an

efficient

way

of

implementing

network

software

by

a

variety

of

network

architectures

-LRB-

VMPT

-LSB-

-RSB-

Sprite

-LSB-

-RSB-

-RRB-

ISO

architecture

sacrificed

efficiency

for

modularity

at

a

time

when

the

introduction

of

high

speed

networks

required

an

increase

in

efficiency

While

the

networking

continued

to

use

layered

models

to

describe

network

software

layered

implementations

were

actually

considered

harmful

-LSB-

-RSB-

In

-LSB-

-RSB-

it

was

shown

that

it

is

possible

to

increase

performance

without

sacrificing

modularity

Some

of

the

methods

used

by

Avoca

to

organize

heavily

layered

protocol

suites

were

also

used

in

the

higher

layers

of

the

ISO

architecture

However

Avoca

's

approach

is

much

more

integrated

with

the

protocol

design

and

also

more

efficient

The

only

modular

networking

architecture

that

is

actually

in

use

is

Unix

System

V

streams

Streams

are

an

extension

of

the

Unix

file

filter

model

into

the

kernel

designed

to

support

serial

line

drivers

It

has

been

extended

to

support

some

the

implementation

of

communications

protocols

However

how

one

does

this

is

not

at

all

obvious

In

addition

the

central

purpose

of

Avoca

was

to

design

protocols

that

compose

No

attempt

has

ever

been

made

to

design

composable

stream-oriented

protocols

Module

Interconnection

Languages

MILs

were

proposed

as

a

means

by

which

previously

existing

modules

written

in

potentially

different

languages

and

running

on

different

machines

could

be

part

of

a

reuse

repository

Our

model

can

be

recognized

as

a

very

simple

MIL

However

there

are

three

important

differences

First

a

problem

with

existing

MIL

's

is

that

they

are

not

sufficient

to

guarantee

module

composability

and

in

some

cases

do

n't

even

seem

to

encourage

the

creation

of

composable

modules

Avoca

is

most

closely

related

to

the

MIL

IDL

-LSB-

-RSB-

and

the

multi-language

programming

environments

MLP

-LSB-

-RSB-

and

Polylith

-LSB-

-RSB-

-LRB-

In

fact

Polylith

notation

was

used

initially

for

Avoca

-RRB-

However

the

critical

feature

of

Avoca

is

the

methodology

used

to

design

composable

protocols

-LSB-

-RSB-

The

notation

used

to

describe

such

protocols

is

not

nearly

as

important

as

properties

such

as

symmetric

components

Second

by

making

it

simple

to

define

arbitrarily

complex

interfaces

the

probability

of

any

module

being

composable

is

reduced

It

is

not

uncommon

in

networking

for

the

implementor

to

unnecessarily

expose

all

internal

details

of

a

protocol

in

the

interface

thus

ensuring

non-composability

Third

current

MILs

are

slow

for

a

variety

of

reasons

but

primarily

because

they

have

sophisticated

type

systems

which

are

necessary

for

packing

and

unpacking

complex

data

types

Performance

and

Other

Myths

of

Layered

Software

Performance

In

many

domains

users

are

willing

to

trade

a

certain

amount

of

performance

for

the

benefits

of

a

software

engineering

environment

In

the

networking

domain

and

others

this

is

emphatically

not

the

case

Network

protocol

design

and

implementation

is

performance-driven

to

the

extreme

The

design

of

Avoca

reflects

this

fact

Avoca

far

from

being

a

top

down

exercise

in

software

design

is

the

side-effect

of

a

performance

experiment

It

was

a

widely-held

belief

in

the

network

community

that

encapsulated

and

layered

protocol

design

inflicts

a

prohibitive

performance

penalty

-LSB-

-RSB-

The

Avoca

domain

model

was

a

side

effect

of

an

experiment

designed

to

show

that

this

belief

was

untrue

or

overstated

It

involved

the

decomposition

of

the

monolithic

Sprite

RPC

protocol

into

thirty

or

so

micro-protocols

a

model

was

required

simply

to

manage

the

large

number

of

protocols

The

majority

of

the

work

on

Avoca

was

focused

on

making

highly

layered

protocol

suites

efficient

While

much

of

this

work

is

protocol-specific

-LRB-

and

in

any

case

there

is

too

much

of

it

to

be

presented

here

see

papers

on

the

xkernel

-RRB-

some

of

it

is

applicable

to

the

general

problem

of

fast

layered

systems

The

first

lesson

is

that

the

temptation

to

make

each

layer

a

process

has

to

be

avoided

at

all

costs

When

organized

in

this

fashion

traversing

each

additional

layer

costs

at

least

one

context

switch

Context

switches

even

for

lightweight

processes

are

enormously

expensive

when

one

considers

the

effect

on

a

memory

cache

-LSB-

-RSB-

Most

interprocess

communication

primitives

have

additional

built-in

overheads

that

only

make

matters

worse

The

xkernel

uses

a

process-per-message

as

it

enters

the

kernel

This

process

shepherds

the

message

through

a

graph

of

protocol

objects

until

it

leaves

the

kernel

In

general

only

one

context

switch

is

required

for

each

message

regardless

of

the

number

of

layers

The

second

lesson

is

the

cost

of

copying

data

and

the

power

of

shared

abstractions

Memory-to-memory

copies

of

large

data

blocks

-LRB-

bytes

-RRB-

are

expensive

Any

layered

protocol

design

that

requires

data

to

be

copied

from

layer

to

layer

would

be

prohibitively

slow

The

xkernel

message

abstraction

-LRB-

i.e.

class/ADT

-RRB-

manages

message

data

buffers

in

a

lazy

fashion

by

avoiding

copies

whenever

possible

and

sometimes

by

eliminating

copying

altogether

All

Avoca

protocols

use

a

common

message

class/ADT

to

manipulate

messages

Finally

while

shared

classes/ADTs

are

often

looked

upon

as

time-savers

that

avoid

duplicate

work

we

found

that

sharing

classes

is

good

not

because

it

reduces

the

total

amount

of

work

but

because

when

a

class

is

constantly

being

reused

any

performance

or

correctness

problems

will

be

found

and

fixed

Consequently

such

fixes

improve

the

reliability

and

performance

of

all

components

that

use

the

class

thereby

greatly

increasing

the

potential

performance

gain

over

component

specific

improvements

The

approach

to

developing

Avoca

was

to

produce

the

fastest

possible

protocol

suites

and

against

all

preconceived

notions

-LRB-

including

the

designers

-RRB-

the

decomposed

version

of

Sprite

RPC

turned

out

to

be

faster

than

the

original

and

monolithic

version

in

both

latency

and

bandwidth

-LRB-

see

-LSB-

-RSB-

for

actual

performance

figures

-RRB-

This

is

not

to

say

that

protocol

encapsulation

improves

performance

in

most

cases

but

that

the

per-protocol

overhead

was

low

enough

that

it

did

not

dominate

the

performance

outcome

Avoca

demonstrates

that

careful

attention

to

performance

can

lead

to

component-based

systems

whose

performance

is

equal

or

even

superior

to

monolithic

ad-hoc

systems

Decomposition

A

classical

fallacy

that

has

been

ingrained

in

the

computer

science

culture

is

the

belief

that

hierarchical

decomposition

of

software

is

impossible

if

performance-competitive

algorithms

are

used

First

decomposition

of

software

into

conceptual

or

virtual

layers

is

always

possible

Decomposition

is

a

correctness

issue

not

a

performance

issue

The

means

by

which

one

decomposes

an

inefficient

software

system

is

identical

to

those

used

for

decomposing

an

efficient

system

hierarchical

decomposition

has

nothing

to

do

with

performance

Second

it

happens

all

the

time

that

components

will

have

various

degrees

of

composability

Some

components

have

little

or

no

constraints

on

their

use

Other

components

can

only

work

in

the

presence

-LRB-

or

can

not

work

in

the

presence

-RRB-

of

a

specific

class

of

components

All

of

these

possibilities

arise

in

practice

This

is

the

reason

why

design

rule

checking

is

present

in

the

Genesis

layout

editor

these

rules

preclude

incorrect

combinations

of

components

from

being

specified

We

will

say

more

about

design

rule

checking

shortly

But

again

the

fact

that

dependencies

between

components

exist

does

not

preclude

a

hierarchical

system

decomposition

Other

Issues

Encapsulation

Our

work

distinguishes

two

types

of

encapsulation

ADT

and

design

We

are

all

familiar

with

ADT

encapsulation

programming

languages

-LRB-

ADA

C++

-RRB-

enforce

ADT

encapsulation

through

the

hiding

of

local

variables

and

local

functions

-LSB-

-RSB-

The

idea

is

to

permit

changes

to

an

ADT

's

implementation

without

triggering

changes

to

other

ADTs

Design

encapsulation

on

the

other

hand

is

the

form

of

encapsulation

that

is

required

for

software

component

technologies

It

is

the

hiding

of

design

details

within

components

and

as

we

will

see

below

is

the

result

of

a

deliberate

design

Unlike

ADT

encapsulation

there

are

aspects

of

design

encapsulation

that

are

not

enforceable

by

compilers

Consider

the

following

examples

Two

commonly

identified

ADTs

-LRB-

modules

-RRB-

in

a

DBMS

are

the

storage

system

-LRB-

which

stores

and

retrieves

tuples

of

relations

-RRB-

and

the

query

optimizer

-LRB-

which

identifies

the

fastest

search

strategy

to

be

used

by

the

storage

system

in

processing

a

query

-RRB-

-LSB-

-RSB-

Both

the

storage

system

and

query

optimizer

can

be

implemented

by

distinct

ADTs

where

each

other

's

local

variables

and

local

functions

are

hidden

from

view

Although

ADT

encapsulation

is

preserved

design

encapsulation

is

not

In

general

the

query

optimizer

needs

to

know

exactly

what

structures

are

maintained

by

the

storage

system

in

order

to

estimate

retrieval

costs

Adding

or

deleting

a

storage

structure

must

always

be

accompanied

by

a

corresponding

change

to

the

query

optimizer

because

design

information

is

shared

by

both

Thus

design

encapsulation

transcends

the

encapsulation

of

individual

ADTs/classes

and

normally

deals

with

sets

of

closely

related

ADTs

We

can

not

imagine

how

design

encapsulation

in

this

example

could

be

enforced

by

compilers

As

a

second

example

the

data

definition

language

-LRB-

DDL

-RRB-

of

a

DBMS

is

used

to

declare

schemas

and

hints

to

the

DBMS

on

how

to

store

and

retrieve

data

efficiently

For

example

if

an

indexing

component

is

present

in

a

DBMS

there

will

be

a

special

statement

in

the

DDL

allowing

schema

designers

to

declare

that

specific

fields

are

to

be

indexed

If

the

indexing

component

is

absent

the

indexing

statement

is

not

present

in

the

DDL

A

standard

way

to

implement

DDLs

is

via

Lex

and

Yacc

-LSB-

-RSB-

An

individual

DDL

statement

is

represented

by

one

or

more

Lex

tokens

and

one

or

more

Yacc

rules

The

presence

or

absence

of

a

component

will

trigger

the

inclusion

or

exclusion

of

tokens

and

rules

The

inclusion/exclusion

process

must

be

done

at

precompile

time

a

task

that

has

nothing

to

do

with

to

ADT

encapsulation

Thus

design

encapsulation

can

not

always

be

realized

solely

by

ADT

encapsulation

We

believe

that

Parnas

was

the

first

to

recognize

the

concept

of

design

encapsulation

-LSB-

-RSB-

and

Guttag

was

among

the

first

to

recognize

the

concept

of

ADT

encapsulation

-LSB-

-RSB-

However

the

distinction

between

design

and

ADT

encapsulation

was

not

evident

to

us

before

we

began

the

Genesis

and

Avoca

projects

Domain

Modeling

Creating

a

domain

model

is

definitely

not

an

ad

hoc

process

It

involves

a

careful

study

of

existing

systems

within

a

domain

to

discern

the

fundamental

layering

present

in

all

such

systems

and

a

disentanglement

of

abstruse

designs

of

ad

hoc

systems

into

a

standard

language

-LRB-

i.e.

type

expressions

-RRB-

Using

existing

systems

as

a

guide

to

standardize

the

decomposition

of

systems

and

designing

generic

interfaces

for

components/realms

is

the

essence

of

domain

modeling

There

is

no

standard

notation

or

metamodel

for

expressing

a

domain

model

For

the

few

domain

models

that

exist

each

is

expressed

in

terms

of

its

own

special-purpose

concepts

which

not

only

complicates

any

modeling

effort

-LRB-

i.e.

identification

of

an

appropriate

notation

is

nontrivial

-RRB-

but

it

also

makes

model

comprehension

difficult

-LRB-

because

ad

hoc

notations

and

metamodels

are

rarely

similar

-RRB-

Our

type-expression

notation

and

our

component-realm

metamodel

are

domain

independent

We

have

shown

that

a

common

set

of

ideas

apply

to

network

software

and

database

software

even

though

both

domains

are

vastly

different

We

conjecture

that

our

notation

and

metamodel

are

applicable

to

a

large

class

of

domains

-LRB-

i.e.

those

with

hierarchical

systems

-RRB-

and

they

present

a

concrete

language

for

domain

analysts

to

express

the

results

of

their

domain

models

Design

Rule

Checking

Not

all

combinations

of

components

are

meaningful

Indiscriminate

but

syntactically

correct

type

expressions

may

correspond

to

systems

that

can

not

possibly

work

An

integral

part

of

software

component

technologies

are

layout

editors

that

enforce

design

rules

that

preclude

illegal

combinations

of

components

The

basic

idea

is

simple

enough

every

component

c

may

insist

that

lower-level

components

-LRB-

which

instantiate

c

's

parameters

-RRB-

satisfy

constraints

in

addition

to

that

of

type

membership

That

is

if

c

-LSB-

x

R

-RSB-

has

parameter

x

of

type

R

it

may

be

that

not

all

components

of

R

can

legally

instantiate

x

only

those

with

certain

properties

are

permitted

Design

rules

express

these

constraints

and

confirm

common

intuition

as

one

composes

components

constraints

on

legal

instantiations

are

inherited

in

a

top-down

manner

thus

further

restricting

the

set

of

components

that

can

be

used

at

any

given

position

in

a

system

We

explain

in

-LSB-

-RSB-

how

design

rules

are

defined

and

enforced

in

the

Genesis

component

layout

editor

-LRB-

DaTE

-RRB-

DaTE

presents

a

graphical

interface

by

which

system

designs

can

compose

components

quickly

while

ensuring

that

incorrect

compositions

are

impossible

to

specify

As

is

evident

from

Section

type

expressions

for

typical

systems

are

difficult

to

read

Graphically

visualizing

expressions

significantly

simplifies

understanding

and

interpretation

of

type

expressions

and

reinforces

the

lego

paradigm

that

underlies

software

components

Since

our

type-expression

formalism

is

generic

it

is

possible

that

domain-independent

layout

editors

can

be

built

Such

editors

be

will

table

driven

where

tables

encode

domainspecific

knowledge

As

more

domains

are

reduced

to

a

type-expression

formalism

we

expect

to

better

understand

the

features

that

these

editors

will

need

to

provide

Component

Interconnection

Component

interconnections

can

be

realized

in

a

variety

of

efficient

ways

Both

Avoca

and

Genesis

use

dispatch

tables

that

encode

the

static

topology

of

stacked

components

Thus

connections

between

components

are

determined

at

runtime

This

implementation

strategy

easily

supports

systems

where

a

component

is

referenced

several

times

-LRB-

e.g.

m

-LSB-

a

a

-RSB-

-RRB-

Only

one

copy

of

the

component

's

source

code

will

be

present

but

multiple

execution

instances

will

exist

Control

blocks

are

used

to

differentiate

multiple

execution

instances

Whenever

there

is

a

choice

of

different

downward

paths

to

follow

as

occurs

in

Avoca

virtual

protocols

and

Genesis

software

busses

static

dispatch

tables

are

not

sufficient

and

additional

connectivity

information

must

be

stored

Such

information

is

recorded

in

component-internal

tables

and

pointers

in

control

blocks

Avoca

goes

further

by

using

additional

dynamically-defined

tables

-LRB-

called

Maps

-RRB-

to

support

upward

paths

arising

from

upcalls

When

possible

Genesis

also

fuses

components

together

at

compile-time

by

a

very

simple

technique

In

the

composition

x

-LSB-

y

-RSB-

component

x

calls

y

This

means

that

each

downcall

di

of

x

is

a

call

to

the

top

operation

Ti

of

y.

Creating

a

set

of

C

precompiler

statements

one

for

each

downcall

of

x

of

the

form

#define

di

Ti

will

literally

rename

each

di

in

x

with

Ti

thereby

making

x

call

y

directly

Figure

illustrates

component

fusion

-LRB-

b

-RRB-

after

fusion

-LRB-

a

-RRB-

before

fusion

y

x

#define

#define

y

x

Figure

Component

Fusion

via

Call

Renaming

Conclusions

The

success

of

mature

engineering

disciplines

lies

in

the

standardization

of

wellunderstood

technologies

By

purchasing

off-the-shelf

components

engineers

can

create

customized

systems

economically

by

building

only

the

parts

that

are

application-specific

Unnecessary

reinvention

of

technology

is

avoided

Software

engineering

-LRB-

and

computer

science

itself

-RRB-

is

a

relatively

young

and

immature

discipline

Contemporary

software

systems

have

been

simple

enough

that

massive

technology

reinvention

was

economically

feasible

This

will

not

be

true

of

the

future

There

are

many

domains

today

that

are

technologically

mature

and

ripe

for

standardization

and

there

will

certainly

be

many

more

a

decade

from

now

Many

of

these

domains

will

center

on

hierarchical

systems

where

a

progression

of

increasingly

more

sophisticated

software

technologies

are

layered

upon

each

other

We

have

presented

a

validated

model

of

hierarchical

system

design

and

implementation

that

explains

how

families

of

such

systems

can

be

assembled

quickly

and

economically

using

prefabricated

components

-LRB-

i.e.

software

ICs

software

legos

-RRB-

Our

model

is

aimed

specifically

at

mature

software

technologies

where

standardization

makes

sense

Standardization

is

the

key

to

large

scale

reuse

We

have

shown

that

complex

domains

can

be

expressed

by

an

elementary

metamodel

of

realms

of

interchangeable

and

plug-compatible

components

Software

systems

of

enormous

complexity

are

elegantly

represented

as

type

expressions

We

have

demonstrated

that

symmetric

components

are

the

key

to

large-scale

reuse

in

important

domains

-LRB-

e.g.

unix

file

filters

database

FMAP

components

communication

protocols

-RRB-

These

components

have

the

unusual

property

that

they

can

be

composed

in

virtually

arbitrary

orders

We

have

noted

common

features

that

all

software

component

technologies

should

exhibit

and

have

identified

three

distinct

granularities

of

as-is

reuse

component

class

and

algorithm

We

explained

how

our

concepts

reinforce

and

extend

the

pioneering

ideas

of

Parnas

Habermann

et

al.

in

the

design

of

hierarchical

systems

and

how

these

ideas

transcend

current

object-oriented

design

techniques

Specifically

we

used

the

ER

model

and

notation

as

a

leverage

to

present

additional

concepts

that

are

needed

to

elevate

the

fundamental

unit

of

large

scale

software

construction

from

individual

classes

to

parametric

components

of

highly

interrelated

classes

We

explained

that

highly

layered

systems

can

be

as

efficient

as

their

monolithic

counterparts

and

that

software

components

require

design

encapsulation

a

form

of

encapsulation

that

is

different

than

conventional

ADT

encapsulation

Because

our

type-expression

notation

and

component-realm

metamodel

are

domain

independent

we

also

feel

that

our

work

can

serve

as

the

basis

for

a

common

language

in

which

other

domain

models

can

be

expressed

We

conjecture

that

our

model

is

a

blue-print

for

a

practical

software

components

technology

The

first

major

test

of

this

conjecture

will

be

the

DARPA-funded

upgrade

of

the

Mach

operating

system

that

will

replace

Mach

's

current

communication

kernel

with

that

of

Avoca/xkernel

This

upgrade

which

is

planned

to

be

the

standard

release

for

a

future

version

of

Mach

OS

is

now

underway

at

the

University

of

Arizona

Acknowledgements

We

are

grateful

to

Bob

Schwanke

Don

Fussell

Ira

Baxter

Paul

Clements

and

the

referees

for

their

suggestions

on

improving

earlier

drafts

of

this

paper

We

also

thank

Jay

Roy

Rob

Read

Ham

Richards

and

Kelsey

Shepherd

for

their

helpful

comments

References

-LSB-

-RSB-

M.J.

Bach

The

Design

of

the

Unix

Operating

System

Prentice-Hall

-LSB-

-RSB-

D.S.

Batory

Modeling

the

Storage

Architectures

of

Commercial

Database

Systems

ACM

TODS

-LRB-

December

-RRB-

-LSB-

-RSB-

D.S.

Batory

Extensible

Cost

Models

and

Query

Optimization

in

GENESIS

in

IEEE

Database

Engineering

-LSB-

-RSB-

D.S.

Batory

Concepts

for

a

Database

System

Compiler

ACM

PODS

-LSB-

-RSB-

D.S.

Batory

J.R.

Barnett

J.F.

Garza

K.P.

Smith

K.

Tsukuda

B.C.

Twichell

T.E.

Wise

GENESIS

An

Extensible

Database

Management

System

IEEE

Trans

Software

Engr.

-LSB-

-RSB-

D.S.

Batory

J.R.

Barnett

J.

Roy

B.C.

Twichell

and

J.

Garza

Construction

of

File

Management

Systems

from

Software

Components

COMPSAC

-LSB-

-RSB-

D.S.

Batory

On

the

Reusability

of

Query

Optimization

Algorithms

Information

Systems

-LSB-

-RSB-

D.S.

Batory

The

Genesis

Database

System

Compiler

User

Manual

U.

of

Texas

Tech

Rep.

-LSB-

-RSB-

D.S.

Batory

and

J.R.

Barnett

DaTE

The

Genesis

DBMS

Software

Layout

Editor

in

Conceptual

Modelling

Databases

and

CASE

R.

Zicari

ed.

McGrawHill

-LSB-

-RSB-

D.S.

Batory

A

Domain

Modeling

Methodology

in

preparation

-LSB-

-RSB-

D.S.

Batory

et

al.

The

Implementation

of

Genesis

in

preparation

-LSB-

-RSB-

I.

Baxter

personal

communication

-LSB-

-RSB-

P.A.

Bernstein

and

D.M.

Chiu

Using

Semi-Joins

to

Solve

Relational

Queries

Journal

of

the

ACM

-LRB-

January

-RRB-

-LSB-

-RSB-

P.A.

Bernstein

et

al.

Query

Processing

in

a

System

for

Distributed

Databases

-LRB-

-RRB-

ACM

Transactions

on

Database

Systems

-LRB-

December

-RRB-

-LSB-

-RSB-

B.

Bershad

T.

Anderson

E.

Lazowska

and

H.

Levy

Lightweight

Remote

Procedure

Call

ACM

Symposium

on

Operating

Systems

Principles

-LSB-

-RSB-

T.J.

Biggerstaff

and

A.J.

Perlis

Software

Reusability

I

Concepts

and

Models

ACM

Press

-LSB-

-RSB-

T.J.

Biggerstaff

and

A.J.

Perlis

Software

Reusability

II

Applications

and

Experience

ACM

Press

-LSB-

-RSB-

G.

Booch

Software

Components

with

Ada

Benjamin

Cummings

-LSB-

-RSB-

G.

Booch

Object-Oriented

Design

With

Applications

Benjamin

Cummings

-LSB-

-RSB-

K.H.

Britton

R.A.

Parker

and

D.L.

Parnas

A

Procedure

for

Designing

Abstract

Interfaces

for

Device

Interface

Modules

International

Conference

on

Software

Engineering

-LSB-

-RSB-

L.

Cardelli

and

P.

Wegner

On

Understanding

Types

Data

Abstraction

and

Polymorphism

ACM

Computing

Surveys

-LRB-

December

-RRB-

-LSB-

-RSB-

D.D.

Chamberlin

et

al.

SEQUEL

A

Unified

Approach

to

Data

Definition

Manipulation

and

Control

IBM

Journal

of

Research

and

Development

-LRB-

November

-RRB-

-LSB-

-RSB-

D.

R.

Cheriton

VMTP

A

transport

protocol

for

the

next

generation

of

communications

systems

ACM

SIGCOM

Symposium

-LRB-

August

-RRB-

-LSB-

-RSB-

D.D.

Clark

The

Structuring

of

Systems

Using

Upcalls

ACM

Symposium

on

Operating

Systems

Principles

-LSB-

-RSB-

P.L.

Deutsch

Design

Reuse

and

Frameworks

in

the

Programming

System

in

-LSB-

-RSB-

-LSB-

-RSB-

E.W.

Dijkstra

The

Structure

of

THE

Multiprogramming

System

Communications

of

the

ACM

-LRB-

May

-RRB-

-LSB-

-RSB-

J.

Goguen

Parameterized

Programming

IEEE

Trans

Software

Engr.

-LRB-

September

-RRB-

-LSB-

-RSB-

A.

Goldberg

The

Interactive

Programming

Environment

Addison-Wesley

-LSB-

-RSB-

G.

Graefe

and

D.J.

DeWitt

The

EXODUS

Optimizer

Generator

ACM

SIG

MOD

-LSB-

-RSB-

H.

Graves

and

W.

Polak

Common

Intermediate

Design

Language

Overview

Lockheed

Palo

Alto

Research

Laboratory

Spring

-LSB-

-RSB-

J.

Guttag

Abstract

Data

Types

and

the

Development

of

Data

Structures

Comm

ACM

-LRB-

June

-RRB-

-LSB-

-RSB-

J.

Guttag

E.

Horowitz

and

D.R.

Musser

Abstract

Data

Types

and

Software

Validation

Communications

of

the

ACM

-LRB-

December

-RRB-

-LSB-

-RSB-

A.N.

Habermann

L.

Flon

and

L.

Cooprider

Modularization

and

Hierarchy

in

a

Family

of

Operating

Systems

Communications

of

the

ACM

-LRB-

May

-RRB-

-LSB-

-RSB-

A.N.

Habermann

L.

Flon

and

L.

Cooprider

Modularization

and

Hierarchy

in

a

Family

of

Operating

Systems

Communications

of

the

ACM

-LRB-

May

-RRB-

-LSB-

-RSB-

R.

Hayes

S.

Manweiler

and

R.

Schlichting

A

simple

system

for

constructing

distributed

mixed

language

programs

Software

Practice

and

Experience

July

-LSB-

-RSB-

N.C.

Hutchinson

and

L.L.

Peterson

The

x-Kernel

An

Architecture

for

Implementing

Network

Protocols

IEEE

Transactions

on

Software

Engineering

-LRB-

January

-RRB-

-LSB-

-RSB-

M.

Jarke

and

J.

Koch

Query

Optimization

in

Database

Systems

ACM

Computing

Surveys

-LRB-

June

-RRB-

-LSB-

-RSB-

R.E.

Johnson

and

B.

Foote

Designing

Reusable

Classes

Journal

of

ObjectOriented

Programming

June/July

-LSB-

-RSB-

H.F.

Korth

and

A.

Silberschatz

Database

System

Concepts

McGraw-Hill

-LSB-

-RSB-

B.

Lindsay

private

communication

-LSB-

-RSB-

B.

Liskov

A.

Snyder

R.

Atkinson

and

C.

Schaffert

Abstraction

Mechanisms

in

CLU

Communications

of

the

ACM

-LRB-

August

-RRB-

-LSB-

-RSB-

G.

Lohman

Grammar-like

Functional

Rules

for

Representing

Query

Optimization

Alternatives

Proc

ACM

SIGMOD

-LSB-

-RSB-

T.

Mason

and

D.

Brown

Unix

Programming

Tools

Lex

Yacc

O'Reilly

Associates

-LSB-

-RSB-

J.C.

Mogul

and

A.

Borg

The

Effect

of

Context

Switches

on

Cache

Performance

Conference

on

Architectural

Support

for

Programming

Languages

and

Operating

Systems

April

-LSB-

-RSB-

S.W.

O'Malley

Ph.D.

Dissertation

University

of

Arizona

-LSB-

-RSB-

S.

O'Malley

Avoca

An

Environment

For

Programming

With

Protocols

Ph.D.

Dissertation

University

of

Arizona

-LRB-

Aug.

-RRB-

-LSB-

-RSB-

S.W.

O'Malley

and

L.L.

Peterson

A

New

Methodology

for

Designing

Network

Software

submitted

for

publication

-LSB-

-RSB-

S.W.

O'Malley

M.B.

Abbott

N.C.

Hutchinson

and

L.L.

Peterson

A

Transparent

Blast

Facility

Journal

of

Internetworking

-LRB-

December

-RRB-

-LSB-

-RSB-

J.K.

Ousterhout

A.R.

Cherenson

F.

Douglis

M.N.

Nelson

and

B.B.

Welch

The

Sprite

Network

Operating

System

IEEE

Computer

-LRB-

February

-RRB-

-LSB-

-RSB-

D.L.

Parnas

On

the

Criteria

to

be

used

in

Decomposing

Systems

into

Modules

Communications

of

the

ACM

-LSB-

-RSB-

D.L.

Parnas

Designing

Software

for

Ease

of

Extension

and

Contraction

IEEE

TOSE

March

-LSB-

-RSB-

R.

Prieto-Diaz

and

G.

Arango

ed.

Domain

Analysis

and

Software

Systems

Modeling

IEEE

Computer

Society

Press

-LSB-

-RSB-

J.

Purtilo

Polylith

An

environment

to

support

management

of

tool

interfaces

In

Proceedings

of

the

ACM

SIGPLAN

Symposium

on

Language

Issues

in

Programming

Environments

July

-LSB-

-RSB-

J.

Roy

Design

and

Use

of

the

Jupiter

File

Management

System

M.Sc

Thesis

Dept.

of

Computer

Science

University

of

Texas

-LSB-

-RSB-

J.

Rumbaugh

M.

Blaha

W.

Premerlani

F.

Eddy

and

W.

Lorensen

ObjectOriented

Modeling

and

Design

Prentice-Hall

-LSB-

-RSB-

Software

Engineering

Institute

Proceedings

of

the

Workshop

on

DomainSpecific

Software

Architectures

July

-LSB-

-RSB-

P.G.

Selinger

et

al.

Access

Path

Selection

in

a

Relational

Database

Management

System

ACM

SIGMOD

-LSB-

-RSB-

R.

Snodgrass

The

Interface

Description

Language

Definition

and

Use

Computer

Science

Press

Rockville

MD.

-LSB-

-RSB-

M.

Stonebraker

E.

Wong

P.

Kreps

and

G.

Held

The

Design

and

Implementation

of

INGRES

ACM

Transactions

on

Database

Systems

-LRB-

Sept.

-RRB-

-LSB-

-RSB-

A.

S.

Tanenbaum

Computer

Networks

Prentice-Hall

-LSB-

-RSB-

D.

L.

Tennenhouse

Layered

Multiplexing

Considered

Harmful

International

Workshop

on

High-Speed

Networks

-LRB-

November

-RRB-

-LSB-

-RSB-

T.J.

Teorey

D.

Yang

and

J.P.

Fry

A

Logical

Design

Methodology

for

Relational

Databases

Using

the

Extended

Entity-Relationship

Model

Computing

Surveys

-LRB-

June

-RRB-

-LSB-

-RSB-

T.J.

Teorey

Database

Modeling

and

Design

The

Entity-Relationship

Approach

Morgan-Kaufmann

-LSB-

-RSB-

D.

Tsichritzis

and

A.

Klug

-LRB-

editors

-RRB-

The

DBMS

Framework

AFIPS

Press

-LSB-

-RSB-

P.

Wadler

and

S.

Blott

How

to

make

ad-hoc

polymorphism

less

ad

hoc

ACM

POPL

-LSB-

-RSB-

E.

Wong

and

K.

Youseffi

Decomposition

A

Strategy

for

Query

Processing

ACM

Transactions

on

Database

Systems

-LRB-

Sept.

-RRB-

-LSB-

-RSB-

C.T.

Yu

Z.M.

Ozsoyoglu

and

K.

Lam

Optimization

of

Tree

Queries

Journal

of

Computer

and

System

Science

-LRB-

December

-RRB-

-LSB-

-RSB-

C.

Zaniolo

The

Database

Language

GEM

ACM

SIGMOD

-LSB-

-RSB-

H.

Zimmermann

OSI

Reference

Model

The

ISO

Model

of

Architecture

for

Open

Systems

Interconnection

IEEE

Transactions

on

Communications

-LRB-

April

-RRB-

open system architectures
software building-blocks