Enhancing

Concurrency

in

Reusable

Data-Structure

Components

L.

Gunaseelan

and

Richard

J.

LeBlanc

Jr.

College

of

Computing

Georgia

Institute

of

Technology

Atlanta

GA

Abstract

Most

object-oriented

language

implementations

include

a

reusable

data-structure

library

that

provides

classes

such

as

lists

trees

stacks

sets

etc.

Most

of

these

libraries

assume

a

single

threaded

application

and

do

not

permit

concurrent

activity

by

multiple

threads

In

this

paper

we

discuss

the

issues

involved

in

threads

traversing

concurrently

through

such

reusable

data

structure

objects

and

provide

possible

approaches

for

enhancing

concurrency

Specifically

we

introduce

two

new

forms

of

concurrency

namely

single-lane

concurrency

and

non-intersecting

concurrency

in

addition

to

the

traditional

form

of

reader-writer

concurrency

We

provide

mental

tools

for

an

implementor

of

a

data

structure

library

for

analyzing

the

operations

of

a

data

structure

to

identify

the

potential

for

various

forms

of

concurrency

We

provide

a

specification

scheme

which

an

implementor

can

use

to

precisely

specify

the

concurrent

interactions

allowed

among

the

operations

and

subsequently

derive

the

implementation

We

find

that

the

matrix

view

of

looking

at

the

concurrency

specification

simplifies

studying

of

concurrency

behavior

under

inheritance

The

ideas

described

here

are

applicable

in

programming

passive

persistent

sharable

objects

supported

by

operating

systems

like

Clouds

This

work

is

funded

by

NSF

grant

Introduction

Most

object-oriented

language

implementations

include

a

reusable

data-structure

library

that

provides

classes

such

as

lists

trees

stacks

sets

etc.

Most

of

these

libraries

assume

a

singlethreaded

application

and

in

general

do

not

permit

concurrent

activity

by

multiple

threads

With

the

developments

in

the

area

of

concurrent

object-oriented

programming

there

is

a

critical

need

to

address

the

issue

of

supporting

concurrent

activity

in

such

reusable

data

structure

components

Meyer

-LSB-

-RSB-

advocates

the

use

of

cursors

for

designing

data

structure

libraries

Meyer

subscribes

to

the

notion

of

treating

the

data

structures

as

state-machines

with

a

built-in

cursor

that

points

to

the

current

element

To

access

the

next

element

in

the

data-structure

object

the

programmer

typically

writes

l.forth

advances

the

cursor

l'

i

l.item

returns

the

item

at

cursor

position

However

the

idea

of

treating

data-structures

as

state

machines

with

cursors

does

not

extend

well

to

a

multi-threaded

environment

For

example

if

two

threads

and

are

to

traverse

a

data

structure

concurrently

they

need

individual

cursors

to

point

to

their

respective

current

elements

As

each

thread

needs

to

maintain

its

own

perception

of

the

data

structure

state

we

can

not

treat

the

data

structure

as

a

machine

with

a

single

state

Any

recourse

to

locking

the

whole

data

structure

for

the

entire

duration

of

access

strictly

serializes

all

accesses

though

exclusive

locking

provides

safe

access

with

consistent

state

such

an

approach

does

not

allow

any

concurrency

at

all

The

idea

of

using

iterators

-LSB-

-RSB-

to

access

elements

of

a

data

structure

has

been

around

for

some

time

Though

an

iterator

separates

the

cursor

from

the

data

structure

and

maintains

it

externally

it

continues

to

make

assumptions

about

the

state

of

the

data

structure

As

an

example

two

iterators

and

of

threads

and

may

be

pointing

to

the

same

element

e

if

decides

to

remove

the

element

e

from

the

data

structure

then

's

perception

of

e

as

a

member

of

the

data

structure

is

no

longer

correct

It

is

important

that

the

state

of

the

data

structure

as

perceived

by

different

iterators/cursors

be

consistent

Also

if

and

concurrently

try

to

add

new

elements

and

next

to

a

particular

element

e

synchronization

is

required

between

threads

and

In

this

paper

we

discuss

the

issues

involved

in

traversing

concurrently

through

reusable

data

structure

components

and

provide

possible

approaches

for

enhancing

concurrency

Specifically

we

introduce

two

new

forms

of

concurrency

namely

single-lane

concurrency

and

non-intersecting

concurrency

in

addition

to

the

traditional

form

of

reader-writer

concurrency

We

provide

methodology

for

how

an

implementor

of

a

data

structure

component

can

identify

the

potential

for

these

concurrency

forms

and

combine

them

effectively

in

a

single

implementation

Section

of

this

paper

defines

the

notion

of

an

iteration

path

this

concept

is

used

in

Section

to

derive

the

two

new

forms

of

concurrency

Section

shows

how

the

operations

of

a

data

structure

can

be

analyzed

in

three

different

dimensions

to

identify

the

potential

for

these

concurrency

forms

Moreover

it

is

important

that

the

programmer

be

able

to

combine

these

concurrency

forms

in

a

single

implementation

Section

describes

how

this

can

be

done

In

Section

we

study

the

implications

of

inheriting

from

a

data

structure

component

that

implements

these

concurrency

forms

Section

is

the

concluding

section

Concurrent

traversal

and

Iteration

path

Our

notion

of

consistent

state

is

as

follows

with

respect

to

an

element

-LRB-

or

item

-RRB-

of

a

data

structure

we

are

interested

only

in

its

characteristics

as

a

member

of

the

data

structure

Specifically

we

are

interested

in

operations

that

add

delete

or

access

elements

as

members

of

a

data

structure

-LRB-

like

push

pop

and

top

in

a

stack

data

structure

-RRB-

rather

than

the

operations

provided

by

a

member

element

itself

Any

synchronization

needed

among

the

operations

of

an

element

we

assume

is

done

in

the

element

objects

Figure

shows

multiple

threads

accessing

a

data

structure

object

with

individual

iterators

and

external

cursors

Client

-LRB-

traverse

and

update

all

elements

-RRB-

-LRB-

traverse

and

print

all

elements

-RRB-

Client

-LRB-

with

local

Cursor

-RRB-

Client

-LRB-

with

local

Cursor

-RRB-

-LRB-

traverse

and

insert

a

new

element

-RRB-

-LRB-

traverse

and

delete

an

existing

element

-RRB-

Figure

Concurrent

access

by

multiple

threads

We

define

iteration

path

as

the

locus

of

traversal

by

a

thread

Iteration

path

defines

the

order

and

the

extent

of

traversal

in

a

data

structure

Order

denotes

the

order

in

which

the

elements

are

accessed

and

the

extent

denotes

the

region

of

thread

activity

For

example

a

search

operation

in

linked-list

may

traverse

in

linear

forward

direction

-LRB-

order

-RRB-

up

to

a

certain

element

-LRB-

extent

-RRB-

Similarly

a

doubly-linked

list

can

be

traversed

either

linear

forward

or

linear

backward

a

tree

can

be

traversed

in

pre-order

or

post-order

and

a

graph

can

be

traversed

either

depth-first

or

breadth

first

The

extent

of

traversal

may

be

to

access

all

the

elements

of

the

data

structure

or

up

to

a

particular

element

the

extent

could

also

be

a

particular

zone

of

the

data

structure

For

example

in

a

queue

data

structure

the

extent

of

the

append

operations

is

the

tail

zone

of

the

queue

while

the

extent

of

remove

operations

is

the

head

zone

of

the

queue

TraversalIn-Order

Traversal

Figure

Different

iteration

paths

of

a

Tree

Figure

shows

multiple

threads

traversing

a

tree

data

structure

in

different

iteration

paths

Successive

elements

in

a

iteration

path

need

not

be

adjacent

in

the

data

structure

they

are

adjacent

only

in

that

iteration

path

Thus

iteration

path

provides

a

linear

abstraction

of

the

traversal

As

shown

in

the

next

section

we

identify

two

forms

of

concurrency

directly

from

the

order

and

extent

notions

of

iteration

path

Even

though

Figure

explicitly

shows

the

iterators

used

we

are

only

concerned

about

the

way

various

operations

iterate

-LRB-

-RRB-

through

the

data

structure

Specifically

an

implementor

should

identify

an

operation

's

order

and

extent

of

traversal

based

on

how

the

operation

is

actually

implemented

For

example

inserting

an

item

in

a

list

will

typically

involve

traversing

the

list

up

to

a

particular

element

and

performing

the

insertion

The

insert

operation

should

be

classified

as

traversing

linear

forward

up

to

the

required

element

A

do

all

operation

may

involve

accessing

all

the

elements

in

some

order

Another

classification

required

is

whether

the

operation

is

a

reader

or

a

writer

These

three

attributes

of

an

operation

-LRB-

order

extent

read/write

attribute

-RRB-

form

part

of

the

operation-specification

repetitive

access

characterizes

the

nature

of

traversing

we

use

the

terms

traverse

and

iterate

interchangeably

Enhancing

Concurrency

Single-lane

Concurrency

Consider

the

following

where

three

threads

in

an

iteration

path

try

to

access

elements

of

a

data

structure

In

the

figure

let

thread

be

moving

through

all

the

elements

of

the

data

structure

summing

up

a

particular

value

-LRB-

say

a

stock

quantity

-RRB-

Let

threads

and

be

accessing

respectively

to

add

and

delete

an

item

If

adds

the

new

item

somewhere

after

's

current

cursor

when

completes

's

sum

will

reflect

the

quantity

of

the

new

item

added

On

the

other

hand

if

the

adds

the

item

at

a

point

where

has

already

traversed

then

when

completes

's

sum

will

not

reflect

the

value

of

the

new

item

added

Similarly

the

value

of

the

item

deleted

by

will

or

will

not

be

reflected

depending

on

whether

the

item

was

deleted

after

or

before

traversed

through

the

point

of

deletion

More

over

and

should

not

simultaneously

try

to

add

or

delete

at

the

same

point

in

the

path

Resorting

to

serializability

semantics

if

the

results

of

the

individual

operations

reflect

some

serial

execution

order

we

can

consider

the

state

of

the

data

structure

as

well

as

the

result

returned

by

operations

to

be

consistent

-LRB-

for

example

the

sum

value

as

seen

by

-RRB-

We

introduce

the

following

constraint

Multiple

threads

should

traverse

through

an

iteration

path

strictly

according

to

some

serial

order

In

other

words

concurrent

threads

in

an

iteration

path

should

not

overtake

each

other

The

form

of

concurrency

we

call

single-lane

concurrency

can

be

directly

derived

from

the

above

constraint

Using

single-lane

concurrency

multiple

threads

can

traverse

through

the

items

of

a

data

structure

in

sequence

As

soon

as

a

thread

leaves

a

particular

element

the

next

thread

can

access

the

element

-LRB-

imagine

cars

moving

along

a

single-lane

road

concurrently

-RRB-

Each

thread

as

it

moves

to

the

next

element

in

the

iteration

path

strictly

yields

access

to

the

next

thread

in

line

Single-lane

concurrency

is

a

useful

mechanism

for

providing

concurrency

among

multiple

threads

in

the

same

iteration

path

For

example

multiple

threads

can

traverse

a

linked-list

in

linear-forward

as

long

as

they

all

traverse

linear-forward

However

two

threads

can

not

traverse

concurrently

if

one

is

accessing

linear-forward

and

the

other

is

accessing

linear-backward

This

leads

to

the

rule

At

any

point

of

time

multiple

threads

can

be

active

in

a

data

structure

along

only

one

particular

iteration

path

Similarly

two

threads

can

traverse

a

tree

in

pre-order

-LRB-

or

post-order

-RRB-

concurrently

The

threads

are

free

to

add

and

delete

elements

as

they

traverse

rather

than

just

access

the

elements

Non-intersecting

Concurrency

The

second

form

of

concurrency

we

discuss

is

based

on

the

extent

notion

of

the

iteration

path

This

form

of

concurrency

is

based

on

the

following

observation

Two

threads

can

concurrently

access

a

data

structure

as

long

as

they

are

accessing

different

parts

of

the

data

structure

We

call

this

form

of

concurrency

non-intersecting

concurrency

For

example

in

a

queue

data

structure

operations

enqueue

and

dequeue

can

go

on

concurrently

since

enqueue

operations

act

on

the

tail

zone

of

the

queue

while

dequeue

operations

act

on

the

head

zone

of

the

queue

But

two

enqueue

-LRB-

or

dequeue

-RRB-

operations

need

to

synchronize

with

each

other

In

the

following

we

discuss

how

such

a

zone

or

extent

of

activity

can

be

used

to

enhance

concurrency

in

accessing

a

tree

data

structure

In

the

case

of

a

tree

two

threads

can

be

active

one

in

the

left

subtree

of

the

root

and

the

other

in

right

subtree

of

the

root

it

is

important

that

there

are

no

active

threads

in

the

root

node

itself

Consider

searching

a

tree-structured

dictionary

object

for

a

word

-LRB-

and

inserting

the

word

if

it

is

not

found

-RRB-

The

code

for

the

root

node

would

look

Boolean

left

subtree

active

right

subtree

active

check

with

current

value

return

if

there

is

a

match

if

traversing

along

left

subtree

is

required

then

if

not

left

subtree

active

then

left

subtree

active

TRUE

synchronization

needed

in

accessing

this

variable

left.operation

-LRB-

-RRB-

else

block

and

retry

later

endif

else

if

traversing

along

right

subtree

is

required

then

similar

code

end

We

need

not

restrict

this

idea

only

to

the

left

and

right

subtrees

of

the

root

node

In

fact

any

node

can

pass

along

another

thread

to

its

left

or

right

child

if

its

sure

there

are

no

active

threads

in

its

immediate

left

or

right

child

Figure

shows

an

implementation

for

the

BINARY

SEARCH

TREE

in

Eiffel

-LSB-

-RSB-

The

synchronization

implemented

is

that

of

single-lane

concurrency

Threads

along

a

particular

subtree

of

a

parent

resort

to

single-lane

concurrency

while

threads

along

the

left

and

right

subtrees

exploit

non-intersecting

concurrency

In

the

all

procedures

carry

a

previous

parameter

a

thread

moving

to

the

ith

item

in

the

iteration

path

carries

a

reference

to

the

i

item

on

which

it

is

currently

holding

a

Only

after

it

successfully

acquires

a

lock

on

the

ith

item

does

it

release

the

lock

on

the

i

item

-LRB-

if

any

-RRB-

to

enable

other

threads

access

the

i

item

In

the

implementation

it

is

interesting

to

note

that

an

element

can

be

a

member

of

more

than

one

data

structure

at

the

same

time

The

element

itself

does

not

get

locked

but

it

is

the

node

-LRB-

or

cell

-RRB-

that

maintains

the

chaining

in

the

data

structure

that

gets

locked

The

only

other

work

that

we

are

familiar

with

that

deals

with

providing

concurrency

in

reusable

data

structures

is

the

work

in

Booch

Components

-LSB-

-RSB-

The

approach

taken

in

the

design

of

concurrent

Booch

components

is

that

of

reader/writer

concurrency

Strictly

classifying

the

operations

as

readers

or

writers

is

not

enough

as

two

add

operations

can

be

considered

as

writers

and

strict

mutual

exclusion

will

be

enforced

The

approach

proposed

here

takes

advantage

of

the

structural

properties

of

the

data

structure

as

well

as

the

way

it

is

accessed

that

this

is

searching

rather

than

doing

a

complete

traversal

Eiffel

the

language

defined

variable

Current

denotes

the

current

instance

class

-LSB-

T

COMPARABLE

-RSB-

export

search

feature

left

right

like

Current

item

T

sem

SEMAPHORE

-LRB-

element

T

prev

like

Current

-RRB-

do

sem.P

acquire

lock

on

the

current

node

if

not

prev.void

then

prev.sem.V

release

lock

on

previous

node

end

if

element

item

then

io.putstring

-LRB-

Item

exists

-RRB-

sem.V

release

the

lock

else

if

element

item

then

if

not

left.void

then

left.search

-LRB-

element

Current

-RRB-

else

left

create

node

for

key

sem.V

release

the

lock

end

else

if

element

item

then

if

not

right.void

then

right.search

-LRB-

element

Current

-RRB-

else

right

create

node

for

key

sem.V

release

the

lock

end

end

end

other

routines

end

Usage

bt

-LSB-

STRING

-RSB-

bt.search

-LRB-

-RRB-

Figure

Implementation

of

BINARY

SEARCH

TREE

Dimensions

of

Concurrency

The

three

characteristics

of

operations

-LRB-

namely

the

order

of

iteration

extent

of

iteration

and

reader/writer

access

-RRB-

provide

three

dimensions

along

which

the

operations

of

the

data

structure

can

be

evaluated

for

exploiting

concurrency

The

time-complexity

of

an

operation

provides

clues

as

to

how

the

implementor

can

go

about

making

a

particular

choice

For

example

consider

a

list

data

structure

The

table

in

Figure

shows

the

various

operations

of

a

LINKED

LIST

data

structure

and

the

order

extent

and

read/write

attributes

for

various

operations

OperationAdd_headRemove_headAdd_tailRemove_tailcopyis_equalsearch_and

insertsearch_and_deletelength_ofhead_oftail_ofis_emptyTimeComplexityO

-LRB-

-RRB-

O

-LRB-

-RRB-

O

-LRB-

-RRB-

O

-LRB-

-RRB-

O

-LRB-

-RRB-

O

-LRB-

-RRB-

O

-LRB-

-RRB-

O

-LRB-

-RRB-

O

-LRB-

N

-RRB-

O

-LRB-

N

-RRB-

O

-LRB-

N

-RRB-

O

-LRB-

N

-RRB-

Order

inear_forwardlinear_forwardlinear_forward

Reader/WriterWWWWRRWWRRRRConcurrencyPotential

Single_laneSingle_laneSingle_laneSingle_laneReader

Writer,,,,,,Non-int

ersect.Non-intersect.

Figure

Operations

of

a

LINKED

LIST

class

The

operations

length

of

head

of

tail

of

and

is

empty

are

expected

to

return

precomputed

values

and

hence

take

only

constant

time

Operations

that

operate

on

the

head

or

tail

zone

of

the

list

-LRB-

add

head

remove

head

etc.

-RRB-

also

take

only

constant

time

Other

operations

-LRB-

copy

is

equal

search

and

insert

etc.

-RRB-

potentially

traverse

the

entire

list

and

take

O

-LRB-

N

-RRB-

time

As

the

operations

that

take

a

constant

time

do

not

lock

the

data

structure

for

long

they

can

be

given

exclusive

access

-LRB-

or

multiple-reader/single-writer

access

-RRB-

Operations

that

take

O

-LRB-

N

-RRB-

time

-LRB-

copy

search

and

insert

etc.

-RRB-

could

lock

the

data

structure

for

a

significant

time

if

given

exclusive

access

In

such

a

case

the

implementor

can

resort

to

single-lane

concurrency

Even

though

is

equal

and

copy

are

readers

and

search

and

insert

and

search

and

delete

are

writers

all

these

operations

can

go

on

concurrently

if

they

follow

single-lane

concurrency

The

implementor

may

decide

to

provide

add

head

remove

head

-LRB-

and

add

tail

remove

tail

-RRB-

BucketsLinked

lst

of

items

in

a

Concurrency

Non-intersecting

Concurrency

across

buckets

Single-lane

Concurrency

within

a

bucket

Index

TableNon-intersecting

Concurrency

Single-lane

Concurrency

Single-lane

Concurrency

Single-lane

Concurrency

-LRB-

a

-RRB-

Hash

buckets

-LRB-

b

-RRB-

Multi-level

index

tables

Figure

Single-lane

and

Non-intersecting

concurrency

operations

either

non-intersecting

access

or

treat

them

as

simple

writers

and

provide

them

exclusive

access

The

implementor

can

make

a

choice

that

he

deems

appropriate

say

based

on

which

class

of

operations

are

invoked

often

It

is

not

difficult

to

think

of

a

data

structure

library

that

offers

a

variety

of

LINKED

LISTs

suited

to

different

application

needs

As

another

example

consider

the

hash

bucket

data

structure

for

storing

items

-LRB-

Figure

-RRB-

Multiple

threads

can

concurrently

hash

and

search

the

different

buckets

Once

in

a

bucket

the

threads

should

follow

single-lane

concurrency

-LRB-

Remember

the

threads

are

free

to

add

or

delete

items

rather

than

just

look

up

-RRB-

If

two

threads

are

accessing

different

buckets

virtually

no

synchronization

is

required

at

all

-LRB-

non-intersecting

concurrency

-RRB-

Another

example

is

use

of

multi-level

index

tables

-LRB-

Figure

-RRB-

Threads

can

search

an

index

table

at

a

particular

level

using

single-lane

concurrency

As

a

result

of

indexing

if

two

threads

at

level

i

go

to

different

index

tables

at

level

i

then

no

synchronization

is

required

between

them

they

can

execute

freely

If

they

both

enter

the

same

index

table

then

they

should

follow

singlelane

concurrency

Thus

within

the

index

table

at

a

particular

level

threads

follow

single-lane

concurrency

threads

in

different

index

tables

exploit

non-intersecting

concurrency

Though

an

implementor

of

a

software

component

can

analyze

the

operations

along

each

of

the

three

dimensions

independently

and

implement

one

particular

form

of

concurrency

there

is

often

a

need

to

combine

more

than

one

form

of

concurrency

in

a

single

implementation

Also

it

may

be

possible

to

automate

the

creation

of

such

a

combination

We

address

these

questions

in

the

next

section

Combining

more

than

one

form

of

Concurrency

When

an

implementor

has

identified

the

potential

for

various

forms

of

concurrency

the

next

step

is

to

classify

the

operations

into

various

concurrency

groups

We

show

how

the

implementor

can

specify

such

groups

in

a

concurrency

specification

and

use

the

specification

to

arrive

at

the

implementation

Before

we

discuss

how

to

combine

and

implement

different

forms

of

concurrency

in

a

single

implementation

we

describe

a

specific

type

of

lock

called

a

mode-lock

that

will

form

the

basis

of

our

implementation

ideas

A

mode-lock

allows

locking

under

various

user-defined

locking

modes

The

programmer

is

required

to

specify

the

compatibility

among

various

modes

with

a

compatibility

matrix

Typically

the

Create

operation

for

the

mode-lock

takes

as

parameters

the

various

locking

modes

and

the

compatibility

matrix

as

shown

below

l

lock

l.create

-LRB-

<list of modes>

<compatibility-matrix>

-RRB-

When

multiple

threads

request

locks

in

different

modes

the

implementation

uses

the

compatibility

matrix

to

decide

whether

to

grant

the

lock

It

is

usually

quite

simple

to

implement

such

locks

As

an

example

we

show

below

how

a

traditional

reader/writer

lock

can

be

created

The

language

Aeolus

-LSB-

-RSB-

provides

mode-locks

as

a

language

level

construct

rw

lock

rw.create

-LRB-

fread

writeg

fread

-LSB-

read

-RSB-

write

-RSB-

g

-RRB-

We

associate

a

mode-lock

with

every

data-structure

object

for

synchronization

among

different

groups

The

concurrency

specification

lists

the

various

concurrency

groups

and

describes

how

they

are

compatible

with

each

other

As

discussed

in

the

next

section

the

specification

also

provides

key

interface

information

for

the

inheriting-clients

of

a

class

We

propose

the

following

construct

for

specifying

the

concurrency

groups

and

the

allowed

degree

of

concurrency

among

operations

in

a

group

Concurrency

Specification

<group-name>

-LSB-

CONCURRENT

-RSB-

-LCB-

<set of operations>

-RCB-

-LSB-

list

of?compatible

groups

-RSB-

EXCLUSIVE

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

Class

copy

has

Concurrency

Specification

-LCB-

copy

has

-RCB-

-LSB-

-RSB-

-LCB-

-RCB-

-LSB-

-RSB-

CONCURRENT

-LCB-

-RCB-

-LSB-

-RSB-

EXCLUSIVE

-LCB-

-RCB-

-LSB-

-RSB-

EXCLUSIVE

-LCB-

-RCB-

-LSB-

-RSB-

Figure

Concurrency

Specification

for

LINKED

LIST

Figure

shows

the

concurrency

specification

for

the

operations

of

the

LINKED

LIST

class

discussed

in

Figure

The

first

component

group-name

is

a

user

defined

label

for

the

concurrency

group

The

second

component

specifies

the

intra-group

concurrency

properties

It

could

be

either

CONCURRENT

or

SINGLE

LANE

or

EXCLUSIVE

meaning

that

operations

belonging

to

the

group

can

all

be

simultaneously

active

or

they

follow

single-lane

rule

or

they

mutually

exclude

each

other

within

the

group

The

next

component

is

the

list

of

operations

that

belong

to

the

group

The

fourth

component

specifies

the

list

of

other

compatible

groups

and

in

effect

provides

the

compatibility

matrix

among

various

groups

A

CONCURRENT

group

is

compatible

with

itself

and

the

members

of

the

group

can

be

concurrently

active

A

SINGLE-LANE

group

is

also

compatible

with

itself

and

its

member

routines

traverse

the

data

structure

along

a

particular

iteration

path

An

EXCLUSIVE

group

is

not

compatible

with

itself

and

its

members

get

exclusive

access

Implementors

can

also

add

optional

comments

to

the

specification

The

non-intersecting

form

of

concurrency

does

not

require

any

active

synchronization

and

such

information

can

be

explicitly

specified

as

a

comment

In

Figure

all

operations

that

traverse

linear-forward

are

grouped

into

two

SINGLE

LANE

groups

linear

forward

readers

and

linear

forward

writers

and

both

groups

are

compatible

with

each

other

The

non

traversing

operations

are

also

classified

into

readers

and

writers

with

the

group

attributes

CONCURRENT

and

EXCLUSIVE

In

the

specification

the

readers

are

always

compatible

with

every

other

reader

group

writers

are

compatible

with

other

readers

only

if

they

both

follow

the

same

single-lane

rule

The

non

traversing

writers

group

is

further

split

into

two

exclusive

groups

and

this

allows

operations

at

the

head

and

tail

zone

of

the

list

to

go

on

concurrently

The

implementation

of

concurrency

among

various

groups

is

now

obvious

the

scheme

described

by

a

specification

can

be

implemented

by

using

a

mode-lock

in

the

data-structure

object

and

by

associating

a

locking

mode

for

each

of

the

concurrency

groups

The

mode-lock

automatically

implements

the

given

compatibility

matrix

however

single-lane

concurrency

must

still

be

explicitly

coded

by

the

implementor

A

note

about

the

working

of

mode-lock

is

appropriate

here

suppose

a

lock

is

currently

held

in

mode

A

if

there

is

a

request

for

the

lock

in

mode

B

the

lock

is

granted

if

A

and

B

are

compatible

If

there

is

then

another

request

for

the

lock

in

mode

C

the

lock

will

be

granted

only

if

C

is

compatible

with

both

A

and

B.

Thus

at

any

point

in

time

the

lock

is

held

not

in

a

single

mode

but

in

a

group

of

compatible

modes

-LRB-

like

A

and

B

in

this

case

-RRB-

Any

new

request

will

be

granted

if

it

is

compatible

with

every

one

of

the

existing

modes

From

the

concurrency

specification

a

compiler

can

automatically

generate

the

suitable

synchronization

code

for

various

operations

Figures

and

show

the

concurrency

specification

for

the

QUEUE

and

the

BINARY

SEARCH

TREE

data

structures

In

the

QUEUE

specification

note

that

the

enqueuers

are

compatible

with

the

dequeuers

but

not

with

themselves

and

vice-versa

for

the

dequeuers

In

the

BINARY

SEARCH

TREE

concurrency

specification

all

pre-order

post-order

and

in-order

operations

are

grouped

into

one

CONCURRENT

group

as

these

operations

do

not

modify

the

structure

of

the

tree

but

perform

some

operation

on

every

element

On

the

other

hand

the

balance

operation

modifies

the

structure

of

the

tree

and

is

in

a

separate

EXCLUSIVE

group

This

means

when

one

thread

is

balancing

the

tree

all

other

threads

wait

In

fact

it

does

not

make

sense

for

two

threads

to

be

Class

Name

QUEUEExport

enqueue

dequeueConcurrency

Specification

enqueuers

EXCLUSIVE

-LCB-

enqueue

-RCB-

-LSB-

dequeuers

-RSB-

Non

intersecting

concurrency

with

dequeuers

dequeuers

EXCLUSIVE

-LCB-

dequeue

-RCB-

-LSB-

enqueuers

-RSB-

Non

intersecting

concurrency

with

enqueuers

Figure

Concurrency

Specification

for

QUEUE

Class

insert

balanced

balance

Concurrency

Specification

CONCURRENT

-LCB-

height

balanced

-RCB-

-LSB-

-RSB-

-LCB-

-RCB-

-LSB-

-RSB-

Non-intersecting

conc

between

left

and

right

subtree

-LCB-

insert

-RCB-

-LSB-

-RSB-

Non-intersecting

conc

between

left

and

right

subtree

balancing

EXCLUSIVE

-LCB-

balance

-RCB-

-LSB-

-RSB-

Figure

Concurrency

Specification

for

BINARY

SEARCH

TREE

balancing

the

tree

simultaneously

In

a

clever

implementation

when

one

thread

finishes

balancing

the

tree

the

other

calls

to

balance

will

notice

that

the

tree

has

just

been

balanced

-LRB-

by

observing

the

balanced

boolean

variable

-RRB-

and

return

without

doing

any

work

The

operations

insert

and

look

up

perform

a

binary

traversal

of

the

tree

and

implement

a

combination

of

single

lane

and

non-intersecting

concurrency

as

discussed

before

They

belong

to

separate

concurrency

groups

that

are

compatible

with

each

other

Again

it

should

be

noted

that

every

reader

is

compatible

with

every

other

reader

The

run-time

execution

behavior

in

the

data

structure

very

much

depends

on

the

order

of

lock

requests

by

various

operations

The

specification

precisely

describes

the

complex

interaction

patterns

and

hence

the

scheme

is

very

powerful

In

general

the

routines

of

a

class

can

be

classified

as

traversing

and

non-traversing

The

traversing

routines

of

a

class

can

be

grouped

into

different

single-lane

groups

depending

on

their

iteration

paths

The

non-traversing

routines

can

be

classified

into

readers

and

writers

While

the

non-traversing

readers

can

be

grouped

into

one

CONCURRENT

group

the

non-traversing

writers

can

be

analyzed

for

non-intersecting

access

and

can

be

grouped

into

one

or

more

EXCLUSIVE

groups

Each

one

of

single-lane

groups

can

be

further

split

into

two

SINGLE

LANE

groups

readers

and

writers

the

readers

are

always

compatible

with

every

other

reader

group

the

readers

and

writers

are

compatible

only

if

they

belong

to

same

single-lane

group

The

compatibility

matrix

between

the

groups

is

sufficient

for

specifying

as

well

as

for

implementing

all

of

these

forms

of

concurrency

Concurrency

under

Inheritance

Any

approach

to

providing

reusability

can

not

ignore

the

importance

of

inheritance

Inheritance

is

a

code

reuse

mechanism

Using

inheritance

a

subclass

may

refine

a

parent

class

by

either

adding

a

new

method

or

redefining

an

existing

method

Alternatively

a

subclass

may

combine

the

behavior

of

multiple

parents

to

get

new

functionality

-LRB-

multiple

inheritance

-RRB-

For

the

purpose

of

our

discussion

we

will

use

the

Eiffel

-LSB-

-RSB-

inheritance

scheme

which

allows

subclasses

to

rename/redefine

the

features

of

a

parent

class

and

also

provides

multiple

inheritance

We

only

consider

the

effects

of

ffl

addition

of

new

features

in

subclass

ffl

renaming

and

redefinition

of

existing

features

and

ffl

multiple

inheritance

We

do

not

consider

the

deletion

of

a

method

by

a

child

class

as

a

child

class

can

choose

not

to

export

a

method

provided

by

its

parent

and

thus

exclude

it

from

the

visibility

of

its

clients

In

the

previous

section

we

showed

how

the

concurrency

specification

can

be

used

to

arrive

at

the

implementation

Here

for

the

treatment

of

concurrency

under

inheritance

we

require

that

the

concurrency

specification

be

part

of

the

class

interface

The

specification

clearly

outlines

the

concurrency

assumptions

and

the

details

of

the

implementation

to

its

inheriting

clients

For

each

inheriting

class

the

implementor

of

the

subclass

must

provide

a

new

concurrency

specification

that

includes

all

the

exported

routines

of

the

subclass

including

even

the

routines

inherited

from

the

parent

class

Figure

shows

the

concurrency

specification

of

a

QUEUE

inheriting

from

a

LINKED

LIST

While

concurrency

without

inheritance

can

be

coded

directly

by

the

programmer

-LRB-

from

the

specification

-RRB-

using

mode

locks

it

turns

out

that

compiler

support

is

needed

for

implementing

concurrency

under

inheritance

If

the

allowable

modes

and

the

code

to

acquire

and

release

locks

are

hard-coded

in

a

parent

class

it

will

not

be

possible

to

add

new

groups

-LRB-

i.e.

add

new

modes

-RRB-

or

move

routines

from

one

group

to

another

with

redefinition

-LRB-

e.g.

reimplementing

a

writer

routine

with

single-lane

concurrency

in

a

subclass

-RRB-

In

fact

it

is

not

inconceivable

that

may

find

it

odd

that

we

require

that

implementation

details

be

known

at

the

class

interface

Our

reasoning

is

as

follows

A

class

has

two

types

of

clients

direct

clients

and

inheriting

clients

While

the

class

interface

provides

an

abstraction

of

the

class

functionality

for

the

direct

clients

and

protects

them

from

the

need

to

know

the

implementation

details

the

same

is

not

the

case

with

inheriting

clients

When

a

subclass

provides

alternate

implementation

of

a

routine

defined

in

the

parent

class

-LRB-

with

redefinition

-RRB-

or

adds

new

routines

to

extend

the

functionality

it

anyway

needs

to

know

the

details

of

the

implementation

Elsewhere

it

has

been

argued

that

inheritance

is

a

code

-LRB-

implementation

-RRB-

sharing

mechanism

rather

than

a

sub-typing

mechanism

In

fact

a

class

effectively

provides

two

types

of

interface

one

meant

for

its

direct

clients

and

other

meant

for

its

inheriting

clients

detailing

the

implementation

assumptions

Class

Name

copy

has

Concurrency

Specification

-LCB-

copy

has

-RCB-

-LSB-

-RSB-

EXCLUSIVE

-LCB-

-RCB-

-LSB-

-RSB-

CONCURRENT

-LCB-

-RCB-

-LSB-

-RSB-

Class

Name

Queueinherit

rename

as

enqueue

rename

as

dequeue

Export

enqueue

dequeue

Specification

enqueuers

EXCLUSIVE

-LCB-

enqueue

-RCB-

-LSB-

dequeuers

-RSB-

Non

intersecting

concurrency

with

dequeuers

dequeuers

EXCLUSIVE

-LCB-

dequeue

-RCB-

-LSB-

enqueuers

-RSB-

Non

intersecting

concurrency

with

enqueuers

CONCURRENT

-LCB-

-RCB-

-LSB-

-RSB-

Figure

QUEUE

inheriting

from

LINKED

LIST

a

scheme

can

be

found

where

subclasses

can

specify

changes

in

concurrency

specification

as

set

operations

manipulating

the

concurrency

groups

of

the

superclass

Addition

of

new

routines

When

a

subclass

adds

a

new

routine

the

new

routine

may

form

a

concurrency

group

of

its

own

or

it

may

be

added

to

any

of

the

existing

concurrency

groups

Assuming

that

a

parent

class

has

m

concurrency

groups

-LRB-

with

a

m?m

compatibility

matrix

-RRB-

the

addition

of

a

new

group

involves

arriving

at

a

new

m

m

compatibility

matrix

Of

m

m

elements

the

m

m

portion

of

the

compatibility

matrix

-LRB-

shaded

region

in

the

Figure

-RRB-

has

to

be

the

same

as

in

the

parent

class

Hence

all

that

the

implementor

-LRB-

m

-RRB-

X

-LRB-

m

-RRB-

X

mm

X

mmm

Figure

Addition

of

a

new

group

of

the

new

group

has

to

provide

is

compatibility

of

the

new

group

with

the

existing

groups

-LRB-

or

that

of

the

existing

groups

with

the

new

group

-RRB-

thereby

filling

up

the

m

row

-LRB-

and

the

m

column

-RRB-

The

new

group

may

have

one

of

the

labels

CONCURRENT

SINGLE

LANE

or

EXCLUSIVE

-LRB-

which

determines

the

-LRB-

m

m

-RRB-

th

element

of

the

new

matrix

-RRB-

When

a

routine

is

added

to

any

of

the

existing

groups

it

requires

no

change

in

the

compatibility

matrix

It

can

simply

be

added

to

any

of

the

existing

groups

as

long

as

the

following

guidelines

are

adhered

to

ffl

A

CONCURRENT

group

includes

only

reader

routines

and

is

compatible

with

itself

ffl

An

EXCLUSIVE

group

includes

only

writers

and

is

not

compatible

with

itself

But

still

it

may

be

compatible

with

other

groups

-LRB-

e.g.

specification

for

the

QUEUE

example

-RRB-

ffl

A

SINGLE

LANE

group

includes

both

readers

and

writers

in

which

all

the

operations

traverse

along

some

particular

iteration

path

The

group

is

compatible

with

itself

X

X

Figure

Splitting

a

group

While

in

the

context

of

discussing

addition

of

new

groups

-LRB-

and

hence

new

modes

-RRB-

it

should

be

noted

that

a

subclass

may

decide

to

split

one

of

the

groups

of

its

parent

class

and

thereby

introduce

new

groups

without

actually

adding

any

new

routines

This

is

especially

possible

when

a

subclass

implementation

identifies

the

potential

for

non-intersecting

concurrency

and

decides

to

split

one

of

the

exclusive

groups

of

its

parent

One

such

example

is

the

QUEUE

inheriting

from

LINKED

LIST

as

shown

in

Figure

To

understand

how

concurrency

specification

as

a

result

of

splitting

an

existing

group

follows

the

rules

of

addition

of

a

new

group

discussed

above

consider

a

specification

with

a

m?m

matrix

With

out

loss

of

generality

assume

that

the

mth

group

is

split

into

two

resulting

in

a

new

m

group

As

seen

from

the

Figure

the

portion

corresponding

to

the

m

m

matrix

is

the

same

in

both

the

specifications

-LRB-

region

-RRB-

The

compatibility

of

the

first

m

groups

with

the

mth

group

that

is

being

split

up

is

predetermined

in

the

parent

class

specification

and

hence

the

row

entries

in

region

should

be

the

same

for

both

the

rows

m

and

m

Similarly

the

column

entries

in

region

should

be

the

same

for

both

columns

m

and

m

The

diagonal

entry

-LRB-

m

m

-RRB-

is

determined

from

the

type

of

the

group

-LRB-

CONCURRENT

SINGLE

LANE

or

EXCLUSIVE

-RRB-

and

remains

the

same

even

after

the

split

up

This

means

for

all

the

split-up

groups

the

diagonal

entries

-LRB-

which

determines

the

compatibility

of

a

group

with

itself

-RRB-

must

be

same

This

only

leaves

the

shaded

region

to

be

filled

when

an

implementor

of

a

subclass

splits

an

existing

group

as

the

entries

in

the

shaded

area

are

not

constrained

by

the

entries

in

the

parent

class

In

other

words

when

a

class

is

split

the

implementor

is

free

to

specify

the

compatibility

between

the

split

groups

in

the

way

he

sees

fit

and

that

is

the

only

thing

he

is

free

to

specify

Figure

shows

the

case

when

a

group

is

split

into

three

groups

resulting

in

the

addition

of

two

new

groups

As

mentioned

before

the

programmer

is

free

to

fill-in

the

entries

in

the

shaded

region

m

m

X

X

Figure

Splitting

a

group

into

three

Multiple

Inheritance

When

a

class

inherits

from

more

than

one

parent

it

has

to

deal

with

more

than

one

compatibility

matrix

provided

by

different

parents

Suppose

if

two

parents

respectively

provide

a

m?m

and

a

n

n

compatibility

matrices

the

subclass

need

to

arrive

at

a

new

-LRB-

m

n

-RRB-

-LRB-

m

n

-RRB-

matrix

for

the

m

n

groups

inherited

from

both

the

parents

This

means

the

implementor

of

the

subclass

has

to

provide

the

compatibility

entries

for

the

shaded

region

in

Figure

In

other

words

the

implementor

has

to

specify

how

the

concurrency

groups

of

one

parent

commute

with

the

concurrency

groups

of

another

parent

In

fact

this

is

the

main

reason

why

we

require

that

the

concurrency

specification

of

a

subclass

be

provided

in

full

even

for

the

groups

of

m

X

mn

X

nm

X

mn

X

n

Figure

Multiple

Inheritance

the

parent

class

rather

than

as

a

addition

or

correction

of

the

parent

class

specification

Renaming

existing

routines

When

existing

routines

are

renamed

it

is

obvious

that

no

change

is

required

in

the

compatibility

matrix

provided

by

the

parent

Figure

shows

the

concurrency

specification

of

a

PRIORITY

QUEUE

that

inherits

from

both

SORTED

LIST

and

QUEUE

It

shows

how

the

routine

names

have

been

renamed

The

group

names

linear

forward

non

traversing

writers

and

non

traversing

readers

have

been

renamed

as

enqueuers

dequeuers

and

others

respectively

However

the

compatibility

matrix

remains

unchanged

Redefining

existing

routines

The

redefinition

of

an

existing

routine

-LRB-

i.e.

providing

an

alternate

implementation

-RRB-

may

or

may

not

change

its

concurrency

behavior

namely

how

and

whether

it

traverses

the

data

structure

and

whether

it

is

a

reader

or

a

writer

If

the

reimplementation

of

the

routine

only

enhances

its

functionality

but

does

not

affect

the

way

it

interacts

with

other

routines

it

will

continue

to

belong

to

the

same

concurrency

group

and

hence

no

change

in

the

concurrency

specification

will

be

required

In

fact

most

routine

redefinitions

that

slightly

enhance

the

functionality

of

parent

routine

directly

call

the

parent

implementation

as

well

in

their

body

and

do

not

exhibit

any

drastic

change

in

their

concurrency

behavior

However

if

the

reimplementation

does

affect

the

concurrency

behavior

of

the

routine

then

it

not

only

affects

the

compatibility

of

the

the

routine

with

other

routines

in

the

group

but

its

Export

same

as

QUEUEClass

Specification

enqueuers

-LCB-

put

-RCB-

-LSB-

enqueuers

-RSB-

dequeuers

EXCLUSIVE

-LCB-

remove

-RCB-

-LSB-

-RSB-

others

CONCURRENT

-LCB-

-RCB-

-LSB-

others

-RSB-

rename

as

put

as

as

remove

Class

Specification

-LCB-

has

-RCB-

-LSB-

-RSB-

CONCURRENT

-LCB-

-RCB-

-LSB-

-RSB-

has

EXCLUSIVE

-LCB-

-RCB-

-RSB-

Class

QUEUE

abstract

classExport

put

remove

provides

NO

concurrency

specification

Left

to

the

sub

classes

that

provide

the

actual

implementation

Figure

PRIORITY

QUEUE

inheriting

from

SORTED

LIST

compatibility

with

routines

belonging

to

other

groups

as

well

In

this

case

the

implementor

will

have

to

look

at

the

implementation

in

a

new

light

and

analyze

the

routines

for

potential

forms

of

concurrency

as

described

in

section

and

rewrite

a

new

concurrency

specification

Conclusion

In

this

paper

we

provide

mental

tools

for

the

implementor

of

a

reusable

data-structure

library

for

analyzing

the

operations

of

a

software

component

and

identifying

the

potential

for

various

forms

of

concurrency

We

provide

a

specification

scheme

which

the

implementor

can

use

to

specify

precisely

the

concurrent

interactions

allowed

among

the

operations

and

which

can

be

used

to

directly

derive

the

implementation

The

matrix

view

of

looking

at

the

concurrency

specification

simplifies

studying

of

concurrency

behavior

under

inheritance

The

ideas

described

here

are

applicable

in

programming

passive

persistent

sharable

objects

-LSB-

-RSB-

supported

by

operating

systems

like

Clouds

-LSB-

-RSB-

where

an

object

programmer

has

to

deal

with

concurrency

among

multiple

threads

One

may

be

concerned

about

the

cost

of

acquiring

and

releasing

locks

at

the

individual

element

level

When

multiple

threads

concurrently

access

a

data-structure

with

the

single-lane

scheme

the

overall

delay

-LRB-

the

time

a

thread

is

blocked

waiting

to

acquire

access

-RRB-

is

reduced

and

hence

results

in

improved

performance

The

performance

penalty

is

for

the

lone

threads

a

lone

thread

has

to

anyway

acquire

and

release

locks

on

individual

elements

even-though

there

are

no

other

active

threads

It

should

be

noted

however

that

the

increase

in

execution

time

is

by

a

constant

factor

if

a

thread

spends

twice

the

amount

of

time

now

with

every

element

-LRB-

spent

in

acquiring

and

releasing

locks

-RRB-

one

can

get

the

same

performance

by

going

to

a

faster

processor

with

double

the

speed

The

single-lane

scheme

does

not

modify

the

algorithmic

performance

an

operation

originally

taking

time

now

takes

time

with

a

different

constant

With

the

trend

of

increasing

hardware

speeds

we

believe

that

any

performance

penalty

that

increases

the

execution

time

by

a

constant

factor

for

lone

threads

can

be

paid

to

get

improved

performance

during

concurrent

access

References

-LSB-

-RSB-

Grady

Booch

Software

Components

with

Ada

Benjamin/Cummings

Reading

MA

-LSB-

-RSB-

P.

Dasgupta

R.

C.

Chen

S.

Menon

M.

P.

Pearson

R.

Ananthanarayanan

U.

Ramachandran

M.

Ahamad

R.

J.

LeBlanc

W.

F.

Appelbe

J.

M.

Bernab?eu

Aub?an

P.

W.

Hutto

M.

Y.

A.

Khalidi

and

C.

J.

Wilkenloh

The

Design

and

Implementation

of

the

Clouds

Distributed

Operating

System

Usenix

Computing

Systems

-LRB-

-RRB-

-LSB-

-RSB-

L.

Gunaseelan

and

Richard

J.

LeBlanc

Distributed

Eiffel

A

language

for

programming

multi-granular

distributed

objects

In

Proceedings

of

the

International

Conference

on

Computer

Languages

April

-LSB-

-RSB-

D.A.

Lamb

Specification

of

iterators

IEEE

Transactions

on

Software

Engineering

-LRB-

-RRB-

-LCB-

December

-LSB-

-RSB-

Bertrand

Meyer

Object

Oriented

Software

Construction

Prentice

Hall

-LSB-

-RSB-

C.

Thomas

Wilkes

and

Richard

J.

LeBlanc

Rationale

for

the

design

of

aeolus

A

systems

programming

language

for

an

action/object

system

In

Proceedings

of

the

International

Conference

on

Computer

Languages

October

O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O B O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O B O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B I O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O B O O O O O O O B O B O B O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

concurrency
data structures
reuse
software components