Runtime

Metrics

Collection

for

Middleware

Supported

Adaptation

of

Mobile

Applications

Hendrik

Gani

School

of

Computer

Science

and

Information

Technology,

RMIT

University,

Melbourne,

Australia

hgani@cs.rmit.edu.au

Caspar

Ryan

School

of

Computer

Science

and

Information

Technology,

RMIT

University,

Melbourne,

Australia

caspar@cs.rmit.edu.au

Pablo

Rossi

School

of

Computer

Science

and

Information

Technology,

RMIT

University,

Melbourne,

Australia

pablo@cs.rmit.edu.au

ABSTRACT

This

paper

proposes,

implements,

and

evaluates

in

terms

of

worst

case

performance,

an

online

metrics

collection

strategy

to

facilitate

application

adaptation

via

object

mobility

using

a

mobile

object

framework

and

supporting

middleware.

The

solution

is

based

upon

an

abstract

representation

of

the

mobile

object

system,

which

holds

containers

aggregating

metrics

for

each

specific

component

including

host

managers,

runtimes

and

mobile

objects.

A

key

feature

of

the

solution

is

the

specification

of

multiple

configurable

criteria

to

control

the

measurement

and

propagation

of

metrics

through

the

system.

The

MobJeX

platform

was

used

as

the

basis

for

implementation

and

testing

with

a

number

of

laboratory

tests

conducted

to

measure

scalability,

efficiency

and

the

application

of

simple

measurement

and

propagation

criteria

to

reduce

collection

overhead.

Categories

and

Subject

Descriptors

Distributed

Systems;

Metrics

General

Terms

Measurement,

Performance.

INTRODUCTION

The

different

capabilities

of

mobile

devices,

plus

the

varying

speed,

error

rate

and

disconnection

characteristics

of

mobile

networks

make

it

difficult

to

predict

in

advance

the

exact

execution

environment

of

mobile

applications.

One

solution

which

is

receiving

increasing

attention

in

the

research

community

is

application

adaptation

in

which

applications

adjust

their

behaviour

in

response

to

factors

such

as

network,

processor,

or

memory

usage.

Effective

adaptation

requires

detailed

and

up

to

date

information

about

both

the

system

and

the

software

itself.

Metrics

related

to

system

wide

information

(e.g.

processor,

memory

and

network

load)

are

referred

to

as

environmental

metrics

while

metrics

representing

application

behaviour

are

referred

as

software

metrics

Furthermore,

the

type

of

metrics

required

for

performing

adaptation

is

dependent

upon

the

type

of

adaptation

required.

For

example,

service-based

adaptation,

in

which

service

quality

or

service

behaviour

is

modified

in

response

to

changes

in

the

runtime

environment,

generally

requires

detailed

environmental

metrics

but

only

simple

software

metrics

On

the

other

hand,

adaptation

via

object

mobility

also

requires

detailed

software

metrics

since

object

placement

is

dependent

on

the

execution

characteristics

of

the

mobile

objects

themselves.

With

the

exception

of

MobJeX

existing

mobile

object

systems

such

as

Voyager

FarGo

and

JavaParty

do

not

provide

automated

adaptation,

and

therefore

lack

the

metrics

collection

process

required

to

support

this

process.

In

the

case

of

MobJeX,

although

an

adaptation

engine

has

been

implemented

preliminary

testing

was

done

using

synthetic

pre-scripted

metrics

since

there

is

little

prior

work

on

the

dynamic

collection

of

software

metrics

in

mobile

object

frameworks,

and

no

existing

means

of

automatically

collecting

them.

Consequently,

the

main

contribution

of

this

paper

is

a

solution

for

dynamic

metrics

collection

to

support

adaptation

via

object

mobility

for

mobile

applications.

This

problem

is

non-trivial

since

typical

mobile

object

frameworks

consist

of

multiple

application

and

middleware

components,

and

thus

metrics

collection

must

be

performed

at

different

locations

and

the

results

efficiently

propagated

to

the

adaptation

engine.

Furthermore,

in

some

cases

the

location

where

each

metric

should

be

collected

is

not

fixed

(i.e.

it

could

be

done

in

several

places)

and

thus

a

decision

must

be

made

based

on

the

efficiency

of

the

chosen

solution

(see

section

The

rest

of

this

paper

is

organised

as

follows:

Section

describes

the

general

structure

and

implementation

of

mobile

object

frameworks

in

order

to

understand

the

challenges

related

to

the

collection,

propagation

and

delivery

of

metrics

as

described

in

section

Section

describes

some

initial

testing

and

results

and

section

closes

with

a

summary,

conclusions

and

discussion

of

future

work.

BACKGROUND

In

general,

an

object-oriented

application

consists

of

objects

collaborating

to

provide

the

functionality

required

by

a

given

problem

domain.

Mobile

object

frameworks

allow

some

of

these

objects

to

be

tagged

as

mobile

objects,

providing

middleware

support

for

such

objects

to

be

moved

at

runtime

to

other

hosts.

At

a

minimum,

a

mobile

object

framework

with

at

least

one

running

mobile

application

consists

of

the

following

components:

runtimes,

mobile

objects,

and

proxies

although

the

terminology

used

by

individual

frameworks

can

differ

A

runtime

is

a

container

process

for

the

management

of

mobile

objects.

For

example,

in

FarGo

this

component

is

known

as

a

core

and

in

most

systems

separate

runtimes

are

required

to

allow

different

applications

to

run

independently,

although

this

is

not

the

case

with

MobJeX,

which

can

run

multiple

applications

in

a

single

runtime

using

threads.

The

applications

themselves

comprise

mobile

objects,

which

interact

with

each

other

through

proxies

Proxies,

which

have

the

same

method

interface

as

the

object

itself

but

add

remote

communication

and

object

tracking

functionality,

are

required

for

each

target

object

that

a

source

object

communicates

with.

Upon

migration,

proxy

objects

move

with

the

source

object.

The

Java

based

system

MobJeX,

which

is

used

as

the

implementation

platform

for

the

metrics

collection

solution

described

in

this

paper,

adds

a

number

of

additional

middleware

components.

Firstly,

a

host

manager

(known

as

a

service

in

MobJeX)

provides

a

central

point

of

communication

by

running

on

a

known

port

on

a

per

host

basis,

thus

facilitating

the

enumeration

or

lookup

of

components

such

as

runtimes

or

mobile

objects.

Secondly,

MobJeX

has

a

per-application

mobile

object

container

called

a

transport

manager

(TM).

As

such

the

host

and

transport

managers

are

considered

in

the

solution

provided

in

the

next

section

but

could

be

omitted

in

the

general

case.

Finally,

depending

on

adaptation

mode,

MobJeX

can

have

a

centralised

system

controller

incorporating

a

global

adaptation

engine

for

performing

system

wide

optimisation.

METRICS

COLLECTION

This

section

discusses

the

design

and

derivation

of

a

solution

for

collecting

metrics

in

order

to

support

the

adaptation

of

applications

via

object

migration.

The

solution,

although

implemented

within

the

MobJeX

framework,

is

for

the

most

part

discussed

in

generic

terms,

except

where

explicitly

stated

to

be

MobJeX

specific.

Metrics

Selection

The

metrics

of

Ryan

and

Rossi

have

been

chosen

as

the

basis

for

this

solution,

since

they

are

specifically

intended

for

mobile

application

adaptation

as

well

as

having

been

derived

from

a

series

of

mathematical

models

and

empirically

validated.

Furthermore,

the

metrics

were

empirically

shown

to

improve

the

application

performance

in

a

real

adaptation

scenario

following

a

change

in

the

execution

environment.

It

would

however

be

beyond

the

scope

of

this

paper

to

implement

and

test

the

full

suite

of

metrics

listed

in

and

thus

in

order

to

provide

a

useful

non-random

subset,

we

chose

to

implement

the

minimum

set

of

metrics

necessary

to

implement

local

and

global

adaptation

and

thereby

satisfy

a

range

of

real

adaptation

scenarios.

As

such

the

solution

presented

in

this

section

is

discussed

primarily

in

terms

of

these

metrics,

although

the

structure

of

the

solution

is

intended

to

support

the

implementation

of

the

remaining

metrics,

as

well

as

other

unspecified

metrics

such

as

those

related

to

quality

and

resource

utilisation.

This

subset

is

listed

below

and

categorised

according

to

metric

type.

Note

that

some

additional

metrics

were

used

for

implementation

purposes

in

order

to

derive

core

metrics

or

assist

the

evaluation,

and

as

such

are

defined

in

context

where

appropriate.

Software

metrics

Number

of

Invocations

(NI),

the

frequency

of

invocations

on

methods

of

a

class.

Performance

metrics

Method

Execution

Time

(ET),

the

time

taken

to

execute

a

method

body

(ms).

Method

Invocation

Time

(IT),

the

time

taken

to

invoke

a

method,

excluding

the

method

execution

time

(ms).

Resource

utilization

metrics

Memory

Usage

(MU),

the

memory

usage

of

a

process

(in

bytes).

Processor

Usage

(PU),

the

percentage

of

the

CPU

load

of

a

host.

Network

Usage

(NU),

the

network

bandwidth

between

two

hosts

(in

bytes/sec).

Following

are

brief

examples

of

a

number

of

these

metrics

in

order

to

demonstrate

their

usage

in

an

adaptation

scenario.

As

Processor

Usage

(PU)

on

a

certain

host

increases,

the

Execution

Time

(ET)

of

a

given

method

executed

on

that

host

also

increases

thus

facilitating

the

decision

of

whether

to

move

an

object

with

high

ET

to

another

host

with

low

PU.

Invocation

Time

(IT)

shows

the

overhead

of

invoking

a

certain

method,

with

the

invocation

overhead

of

marshalling

parameters

and

transmitting

remote

data

for

a

remote

call

being

orders

of

magnitude

higher

than

the

cost

of

pushing

and

popping

data

from

the

method

call

stack.

In

other

words,

remote

method

invocation

is

expensive

and

thus

should

be

avoided

unless

the

gains

made

by

moving

an

object

to

a

host

with

more

processing

power

(thereby

reducing

ET)

outweigh

the

higher

IT

of

the

remote

call.

Finally,

Number

of

Invocations

(NI)

is

used

primarily

as

a

weighting

factor

or

multiplier

in

order

to

enable

the

adaptation

engine

to

predict

the

value

over

time

of

a

particular

adaptation

decision.

Metrics

Measurement

This

subsection

discusses

how

each

of

the

metrics

in

the

subset

under

investigation

can

be

obtained

in

terms

of

either

direct

measurement

or

derivation,

and

where

in

the

mobile

object

framework

such

metrics

should

actually

be

measured.

Of

the

environmental

resource

metrics,

Processor

Usage

(PU)

and

Network

Usage

(NU)

both

relate

to

an

individual

machine,

and

thus

can

be

directly

measured

through

the

resource

monitoring

subsystem

that

is

instantiated

as

part

of

the

MobJeX

service.

However,

Memory

Usage

(MU),

which

represents

the

memory

state

of

a

running

process

rather

than

the

memory

usage

of

a

host,

should

instead

be

collected

within

an

individual

runtime.

The

measurement

of

Number

of

Invocations

(NI)

and

Execution

Time

(ET)

metrics

can

be

also

be

performed

via

direct

measurement,

however

in

this

case

within

the

mobile

object

implementation

(mobject)

itself.

NI

involves

simply

incrementing

a

counter

value

at

either

the

start

or

end

of

a

method

call,

depending

upon

the

desired

semantics

with

regard

to

thrown

exceptions,

while

ET

can

be

measured

by

starting

a

timer

at

the

beginning

of

the

method

and

stopping

it

at

the

end

of

the

method,

then

retrieving

the

duration

recorded

by

the

timer.

In

contrast,

collecting

Invocation

Time

(IT)

is

not

as

straight

forward

because

the

time

taken

to

invoke

a

method

can

only

be

measured

after

the

method

finishes

its

execution

and

returns

to

the

caller.

In

order

to

collect

IT

metrics,

another

additional

metric

is

needed.

Ryan

and

Rossi

define

the

metric

Response

Time

(RT),

as

the

total

time

taken

for

a

method

call

to

finish,

which

is

the

sum

of

IT

and

ET.

The

Response

Time

can

be

measured

directly

using

the

same

timer

based

technique

used

to

measure

ET,

although

at

the

start

and

end

of

the

proxy

call

rather

than

the

method

implementation.

Once

the

Response

Time

(RT)

is

known,

IT

can

derived

by

subtracting

RT

from

ET.

Although

this

derivation

appears

simple,

in

practice

it

is

complicated

by

the

fact

that

the

RT

and

ET

values

from

which

the

IT

is

derived

are

by

necessity

measured

using

timer

code

in

different

locations

i.e.

RT

measured

in

the

proxy,

ET

measured

in

the

method

body

of

the

object

implementation.

In

addition,

the

proxies

are

by

definition

not

part

of

the

MobJeX

containment

hierarchy,

since

although

proxies

have

a

reference

to

their

target

object,

it

is

not

efficient

for

a

mobile

object

(mobject)

to

have

backward

references

to

all

of

the

many

proxies

which

reference

it

(one

per

source

object).

Fortunately,

this

problem

can

be

solved

using

the

push

based

propagation

mechanism

described

in

section

in

which

the

RT

metric

is

pushed

to

the

mobject

so

that

IT

can

be

derived

from

the

ET

value

stored

there.

The

derived

value

of

IT

is

then

stored

and

propagated

further

as

necessary

according

to

the

criteria

of

section

the

structural

relationship

of

which

is

shown

in

Figure

Measurement

Initiation

The

polling

approach

was

identified

as

the

most

appropriate

method

for

collecting

resource

utilisation

metrics,

such

as

Processor

Usage

(PU),

Network

Usage

(NU)

and

Memory

Usage

(MU),

since

they

are

not

part

of,

or

related

to,

the

direct

flow

of

the

application.

To

measure

PU

or

NU,

the

resource

monitor

polls

the

Operating

System

for

the

current

CPU

or

network

load

respectively.

In

the

case

of

Memory

Usage

(MU),

the

Java

Virtual

Machine

(JVM)

is

polled

for

the

current

memory

load.

Note

that

in

order

to

minimise

the

impact

on

application

response

time,

the

polling

action

should

be

done

asynchronously

in

a

separate

thread.

Metrics

that

are

suitable

for

application

initiated

collection

(i.e.

as

part

of

a

normal

method

call)

are

software

and

performance

related

metrics,

such

as

Number

of

Invocations

(NI),

Execution

Time

(ET),

and

Invocation

Time

(IT),

which

are

explicitly

related

to

the

normal

invocation

of

a

method,

and

thus

can

be

measured

directly

at

this

time.

Metrics

Aggregation

In

the

solution

presented

in

this

paper,

all

metrics

collected

in

the

same

location

are

aggregated

in

a

MetricsContainer

with

individual

containers

corresponding

to

functional

components

in

the

mobile

object

framework.

The

primary

advantage

of

aggregating

metrics

in

containers

is

that

it

allows

them

to

be

propagated

easily

as

a

cohesive

unit

through

the

components

of

the

mobility

framework

so

that

they

can

be

delivered

to

the

adaptation

engine,

as

discussed

in

the

following

subsection.

Note

that

this

containment

captures

the

different

granularity

of

measurement

attributes

and

their

corresponding

metrics.

Consider

the

case

of

measuring

memory

consumption.

At

a

coarse

level

of

granularity

this

could

be

measured

for

an

entire

application

or

even

a

system,

but

could

also

be

measured

at

the

level

of

an

individual

object;

or

for

an

even

finer

level

of

granularity,

the

memory

consumption

during

the

execution

of

a

specific

method.

As

an

example

of

the

level

of

granularity

required

for

mobility

based

adaptation,

the

local

adaptation

algorithm

proposed

by

Ryan

and

Rossi

requires

metrics

representing

both

the

duration

of

a

method

execution

and

the

overhead

of

a

method

invocation.

The

use

of

metrics

containers

facilitates

the

collection

of

metrics

at

levels

of

granularity

ranging

from

a

single

machine

down

to

the

individual

method

level.

Note

that

some

metrics

containers

do

not

contain

any

Metric

objects,

since

as

previously

described,

the

sample

implementation

uses

only

a

subset

of

the

adaptation

metrics

from

However,

for

the

sake

of

consistency

and

to

promote

flexibility

in

terms

of

adding

new

metrics

in

the

future,

these

containers

are

still

considered

in

the

present

design

for

completeness

and

for

future

work.

Propagation

and

Delivery

of

Metrics

The

solution

in

this

paper

identifies

two

stages

in

the

metrics

collection

and

delivery

process.

Firstly,

the

propagation

of

metrics

through

the

components

of

the

mobility

framework

and

secondly,

the

delivery

of

those

metrics

from

the

host

manager/service

(or

runtime

if

the

host

manager

is

not

present)

to

the

adaptation

engine.

Regarding

propagation,

in

brief,

it

is

proposed

that

when

a

lower

level

system

component

detects

the

arrival

of

a

new

metric

update

(e.g.

mobile

object),

the

metric

is

pushed

(possibly

along

with

other

relevant

metrics)

to

the

next

level

component

(i.e.

runtime

or

transport

manager

containing

the

mobile

object),

which

at

some

later

stage,

again

determined

by

a

configurable

criteria

(for

example

when

there

are

a

sufficient

number

of

changed

mobjects)

will

get

pushed

to

the

next

level

component

(i.e.

the

host

manager

or

the

adaptation

engine).

A

further

incentive

for

treating

propagation

separately

from

delivery

is

due

to

the

distinction

between

local

and

global

adaptation

Local

adaptation

is

performed

by

an

engine

running

on

the

local

host

(for

example

in

MobJeX

this

would

occur

within

the

service)

and

thus

in

this

case

the

delivery

phase

would

be

a

local

inter-process

call.

Conversely,

global

adaptation

is

handled

by

a

centralised

adaptation

engine

running

on

a

remote

host

and

thus

the

delivery

of

metrics

is

via

a

remote

call,

and

in

the

case

where

multiple

runtimes

exist

without

a

separate

host

manager

the

delivery

process

would

be

even

more

expensive.

Therefore,

due

to

the

presence

of

network

communication

latency,

it

is

important

for

the

host

manager

to

pass

as

many

metrics

as

possible

to

the

adaptation

engine

in

one

invocation,

implying

the

need

to

gather

these

metrics

in

the

host

manager,

through

some

form

of

push

or

propagation,

before

sending

them

to

the

adaptation

engine.

Consequently,

an

abstract

representation

or

model

of

the

system

needs

to

be

maintained.

Such

a

model

would

contain

model

entities,

corresponding

to

each

of

the

main

system

components,

connected

in

a

tree

like

hierarchy,

which

precisely

reflects

the

structure

and

containment

hierarchy

of

the

actual

system.

Attaching

metrics

containers

to

model

entities

allows

a

model

entity

representing

a

host

manager

to

be

delivered

to

the

adaptation

engine

enabling

it

to

access

all

metrics

in

that

component

and

any

of

its

children

(i.e.

runtimes,

and

mobile

objects).

Furthermore

it

would

generally

be

expected

that

an

adaptation

engine

or

system

controller

would

already

maintain

a

model

of

the

system

that

can

not

only

be

reused

for

propagation

but

also

provides

an

effective

means

of

delivering

metrics

information

from

the

host

manager

to

the

adaptation

engine.

The

relationship

between

model

entities

and

metrics

containers

is

captured

in

Figure

Propagation

and

Delivery

Criteria

This

subsection

proposes

flexible

criteria

to

allow

each

component

to

decide

when

it

should

propagate

its

metrics

to

the

next

component

in

line

(Figure

in

order

to

reduce

the

overhead

incurred

when

metrics

are

unnecessarily

propagated

through

the

components

of

the

mobility

framework

and

delivered

to

the

adaptation

engine.

This

paper

proposes

four

different

types

of

criterion

that

are

executed

at

various

stages

of

the

measurement

and

propagation

process

in

order

to

determine

whether

the

next

action

should

be

taken

or

not.

This

approach

was

designed

such

that

whenever

a

single

criterion

is

not

satisfied,

the

subsequent

criteria

are

not

tested.

These

four

criteria

are

described

in

the

following

subsections.

Measure

Metric

Criterion

This

criterion

is

attached

to

individual

Metric

objects

to

decide

whether

a

new

metric

value

should

be

measured

or

not.

This

is

most

useful

in

the

case

where

it

is

expensive

to

measure

a

particular

metric.

Furthermore,

this

criterion

can

be

used

as

a

mechanism

for

limiting

storage

requirements

and

manipulation

overhead

in

the

case

where

metric

history

is

maintained.

Simple

examples

would

be

either

time

or

frequency

based

whereas

more

complex

criteria

could

be

domain

specific

for

a

particular

metric,

or

based

upon

information

stored

in

the

metrics

history.

Notify

Metrics

Container

Criterion

This

criterion

is

also

attached

to

individual

Metric

objects

and

is

used

to

determine

the

circumstances

under

which

the

Metric

object

should

notify

its

MetricsContainer.

This

is

based

on

the

assumption

that

there

may

be

cases

where

it

is

desirable

to

measure

and

store

a

metric

in

the

history

for

the

analysis

of

temporal

behaviour,

but

is

not

yet

significant

enough

to

notify

the

MetricsContainer

for

further

processing.

A

simple

example

of

this

criterion

would

be

threshold

based

in

which

the

newest

metric

value

is

compared

with

the

previously

stored

value

to

determine

whether

the

difference

is

significant

enough

to

be

of

any

interest

to

the

MetricsContainer.

A

more

complex

criterion

could

involve

analysis

of

the

history

to

determine

whether

a

pattern

of

recent

changes

is

significant

enough

to

warrant

further

processing

and

possible

metrics

delivery.

Notify

Model

Entity

Criterion

Unlike

the

previous

two

criteria,

this

criterion

is

associated

with

a

MetricsContainer.

Since

a

MetricsContainer

can

have

multiple

Metric

objects,

of

which

it

has

explicit

domain

knowledge,

it

is

able

to

determine

if,

when,

and

how

many

of

these

metrics

should

be

propagated

to

the

ModelEntity

and

thus

become

candidates

for

being

part

of

the

hierarchical

ModelEntity

push

process

as

described

below.

This

decision

making

is

facilitated

by

the

notifications

received

from

individual

Metric

objects

as

described

above.

A

simple

implementation

would

be

waiting

for

a

certain

number

of

updates

before

sending

a

notification

to

the

model

entity.

For

example,

since

the

MobjectMetricsContainer

object

contains

three

metrics,

a

possible

criteria

would

be

to

check

if

two

or

more

of

the

metrics

have

changed.

A

slightly

more

advanced

implementation

can

be

done

by

giving

each

metric

a

weight

to

indicate

how

significant

it

is

in

the

adaptation

decision

making

process.

Push

Criterion

The

push

criterion

applies

to

all

of

the

ModelEntites

which

are

containers,

that

is

the

TransportManagerModelEntity,

RuntimeModelEntity

and

ServiceModelEntity,

as

well

as

the

special

case

of

the

ProxyMetricsContainer.

The

purpose

of

this

criterion

is

twofold.

For

the

TransportManagerModelEntity

this

serves

as

a

criterion

to

determine

notification

since

as

with

the

previously

described

criteria,

a

local

reference

is

involved.

For

the

other

model

entities,

this

serves

as

an

opportunity

to

determine

both

when

and

what

metrics

should

be

pushed

to

the

parent

container

wherein

the

case

of

the

ServiceModelEntity

the

parent

is

the

adaptation

engine

itself

or

in

the

case

of

the

ProxyMetricsContainer

the

target

of

the

push

is

the

MobjectMetricsContainer.

Furthermore,

this

criterion

is

evaluated

using

information

from

two

sources.

Firstly,

it

responds

to

the

notification

received

from

its

own

MetricsContainer

but

more

importantly

it

serves

to

keep

track

of

notifications

from

its

child

ModelEntities

so

as

to

determine

when

and

what

metrics

information

should

be

pushed

to

its

parent

or

target.

In

the

specialised

case

of

the

push

criterion

for

the

proxy,

the

decision

making

is

based

on

both

the

ProxyMetricsContainer

itself,

as

well

as

the

information

accumulated

from

the

individual

ProxyMethodMetricsContainers.

Note

that

a

push

criterion

is

not

required

for

a

mobject

since

it

does

not

have

any

containment

or

aggregating

responsibilities

since

this

is

already

Service

Model

Entity

Service

Metrics

Container

Notify

Model

Entity

Criterion

Runtime

Model

Entity

Runtime

Metrics

Container

Notify

Model

Entity

Criterion

Transport

Manager

Model

Entity

Transport

Manager

Metrics

Container

Notify

Model

Entity

Criterion

Push

Criterion

Mobject

Model

Entity

Mobject

Method

Metrics

Notify

Model

Entity

Criterion

Push

Criterion

Push

Criterion

To

adaptation

engine

Mobject

Metrics

Container

Notify

Metrics

Container

Criterion

Measure

Metric

Criterion

Metric

NotifyMetrics

Container

Criterion

Notify

Metrics

Container

Criterion

Measure

Metric

CriterionProxyMethod

Metrics

Containers

RT

Metric

Notify

Metrics

Container

Criterion

ProxyMetrics

Container

Push

Criterion

Measure

Metric

Criterion

Metric

Measure

Metric

Criterion

Metric

not

currently

implemented

Notify

Metrics

Container

Criterion

Metric

Metric

Measure

Metric

Criterion

Measure

Metric

Criterion

Notify

Metrics

Container

Criterion

MU

Metric

Measure

Metric

Criterion

Notify

Metrics

Container

Criterion

ET

Metric

IT

Metric

NI

Metric

Measure

Metric

Criterion

Measure

Metric

Criterion

Measure

Metric

Criterion

Notify

Metrics

Container

Criterion

NU

Metric

PU

Metric

Measure

Metric

Criterion

Measure

Metric

Criterion

Figure

Structural

overview

of

the

hierarchical

and

criteriabased

notification

relationships

between

Metrics,

Metrics

Containers,

and

Model

Entities

handled

by

the

MobjectMetricsContainer

and

its

individual

MobjectMethodMetricsContainers.

Although

it

is

always

important

to

reduce

the

number

of

pushes,

this

is

especially

so

from

a

service

to

a

centralised

global

adaptation

engine,

or

from

a

proxy

to

a

mobject.

This

is

because

these

relationships

involve

a

remote

call

which

is

expensive

due

to

connection

setup

and

data

marshalling

and

unmarshalling

overhead,

and

thus

it

is

more

efficient

to

send

a

given

amount

of

data

in

aggregate

form

rather

than

sending

smaller

chunks

multiple

times.

A

simple

implementation

for

reducing

the

number

of

pushes

can

be

done

using

the

concept

of

a

process

period

in

which

case

the

model

entity

accumulates

pushes

from

its

child

entities

until

the

process

period

expires

at

which

time

it

pushes

the

accumulated

metrics

to

its

parent.

Alternatively

it

could

be

based

on

frequency

using

domain

knowledge

about

the

type

of

children

for

example

when

a

significant

number

of

mobjects

in

a

particular

application

(i.e.

TransportManager)

have

undergone

substantial

changes.

For

reducing

the

size

of

pushed

data,

two

types

of

pushes

were

considered:

shallow

push

and

deep

push.

With

shallow

push,

a

list

of

metrics

containers

that

contain

updated

metrics

is

pushed.

In

a

deep

push,

the

model

entity

itself

is

pushed,

along

with

its

metrics

container

and

its

child

entities,

which

also

have

reference

to

metrics

containers

but

possibly

unchanged

metrics.

In

the

case

of

the

proxy,

a

deep

push

involves

pushing

the

ProxyMetricsContainer

and

all

of

the

ProxyMethodMetricsContainers

whereas

a

shallow

push

means

only

the

ProxyMethodMetricsContainers

that

meet

a

certain

criterion.

EVALUATION

The

preliminary

tests

presented

in

this

section

aim

to

analyse

the

performance

and

scalability

of

the

solution

and

evaluate

the

impact

on

application

execution

in

terms

of

metrics

collection

overhead.

All

tests

were

executed

using

two

Pentium

GHz

PCs

with

MB

of

RAM,

running

Java

The

two

machines

were

connected

to

a

router

with

a

third

computer

acting

as

a

file

server

and

hosting

the

external

adaptation

engine

implemented

within

the

MobJeX

system

controller,

thereby

simulating

a

global

adaptation

scenario.

Since

only

a

limited

number

of

tests

could

be

executed,

this

evaluation

chose

to

measure

the

worst

case

scenario

in

which

all

metrics

collection

was

initiated

in

mobjects,

wherein

the

propagation

cost

is

higher

than

for

any

other

metrics

collected

in

the

system.

In

addition,

since

exhaustive

testing

of

criteria

is

beyond

the

scope

of

this

paper,

two

different

types

of

criteria

were

used

in

the

tests.

The

measure

metrics

criterion

was

chosen,

since

this

represents

the

starting

point

of

the

measurement

process

and

can

control

under

what

circumstances

and

how

frequently

metrics

are

measured.

In

addition,

the

push

criterion

was

also

implemented

on

the

service,

in

order

to

provide

an

evaluation

of

controlling

the

frequency

of

metrics

delivery

to

the

adaptation

engine.

All

other

(update

and

push)

criteria

were

set

to

always

meaning

that

they

always

evaluated

to

true

and

thus

a

notification

was

posted.

Figure

shows

the

metric

collection

overhead

in

the

mobject

(MMCO),

for

different

numbers

of

mobjects

and

methods

when

all

criteria

are

set

to

always

to

provide

the

maximum

measurement

and

propagation

of

metrics

and

thus

an

absolute

worst

case

performance

scenario.

It

can

be

seen

that

the

independent

factors

of

increasing

the

number

of

mobjects

and

methods

independently

are

linear.

Although

combining

these

together

provides

an

exponential

growth

that

is

approximately

n-squared,

the

initial

results

are

not

discouraging

since

delivering

all

of

the

metrics

associated

with

mobjects,

each

having

methods

(which

constitutes

quite

a

large

application

given

that

mobjects

typically

represent

coarse

grained

object

clusters)

is

approximately

which

could

reasonably

be

expected

to

be

offset

with

adaptation

gains.

Note

that

in

contrast,

the

proxy

metrics

collection

overhead

(PMCO)

was

relatively

small

and

constant

at

since

in

the

absence

of

a

proxy

push

criterion

(this

was

only

implemented

on

the

service)

the

response

time

(RT)

data

for

a

single

method

is

pushed

during

every

invocation.

Number

of

Mobjects/Methods

MobjectMetricsCollectionOverheadMMCO(ms)

Methods

Mobjects

Both

Figure

Worst

case

performance

characteristics

The

next

step

was

to

determine

the

percentage

metrics

collection

overhead

compared

with

execution

time

in

order

to

provide

information

about

the

execution

characteristics

of

objects

that

would

be

suitable

for

adaptation

using

this

metric

collection

approach.

Clearly,

it

is

not

practical

to

measure

metrics

and

perform

adaptation

on

objects

with

short

execution

times

that

cannot

benefit

from

remote

execution

on

hosts

with

greater

processing

power,

thereby

offsetting

IT

overhead

of

remote

compared

with

local

execution

as

well

as

the

cost

of

object

migration

and

the

metrics

collection

process

itself.

In

addition,

to

demonstrate

the

effect

of

using

simple

frequency

based

criteria,

the

MMCO

results

as

a

percentage

of

method

execution

time

were

plotted

as

a

graph

in

Figure

with

the

z-axis

representing

the

frequency

used

in

both

the

measure

metrics

criterion

and

the

service

to

adaptation

engine

push

criterion.

This

means

that

for

a

frequency

value

of

metrics

are

only

measured

on

every

fifth

method

call,

which

then

results

in

a

notification

through

the

model

entity

hierarchy

to

the

service,

on

this

same

fifth

invocation.

Furthermore,

the

value

of

was

also

applied

to

the

service

push

criterion

so

that

metrics

were

only

pushed

to

the

adaptation

engine

after

five

such

notifications,

that

is

for

example

five

different

mobjects

had

updated

their

metrics.

These

results

are

encouraging

since

even

for

the

worst

case

scenario

of

the

metric

collection

overhead

is

an

acceptable

for

a

method

of

duration

(which

is

relatively

short

for

a

component

or

service

level

object

in

a

distributed

enterprise

class

application)

with

previous

work

on

adaptation

showing

that

such

an

overhead

could

easily

be

recovered

by

the

efficiency

gains

made

by

adaptation

Furthermore,

the

measurement

time

includes

delivering

the

results

synchronously

via

a

remote

call

to

the

adaptation

engine

on

a

different

host,

which

would

normally

be

done

asynchronously,

thus

further

reducing

the

impact

on

method

execution

performance.

The

graph

also

demonstrates

that

even

using

modest

criteria

to

reduce

the

metrics

measurement

to

more

realistic

levels,

has

a

rapid

improvement

on

collection

overhead

at

for

of

ET.

MMCO

ET

(milliseconds)

N

(interval)

MMCO

Figure

Performance

characteristics

with

simple

criteria

SUMMARY

AND

CONCLUSIONS

Given

the

challenges

of

developing

mobile

applications

that

run

in

dynamic/heterogeneous

environments,

and

the

subsequent

interest

in

application

adaptation,

this

paper

has

proposed

and

implemented

an

online

metrics

collection

strategy

to

assist

such

adaptation

using

a

mobile

object

framework

and

supporting

middleware.

Controlled

lab

studies

were

conducted

to

determine

worst

case

performance,

as

well

as

show

the

reduction

in

collection

overhead

when

applying

simple

collection

criteria.

In

addition,

further

testing

provided

an

initial

indication

of

the

characteristics

of

application

objects

(based

on

method

execution

time)

that

would

be

good

candidates

for

adaptation

using

the

worst

case

implementation

of

the

proposed

metrics

collection

strategy.

A

key

feature

of

the

solution

was

the

specification

of

multiple

configurable

criteria

to

control

the

propagation

of

metrics

through

the

system,

thereby

reducing

collection

overhead.

While

the

potentially

efficacy

of

this

approach

was

tested

using

simple

criteria,

given

the

flexibility

of

the

approach

we

believe

there

are

many

opportunities

to

significantly

reduce

collection

overhead

through

the

use

of

more

sophisticated

criteria.

One

such

approach

could

be

based

on

maintaining

metrics

history

in

order

to

determine

the

temporal

behaviour

of

metrics

and

thus

make

more

intelligent

and

conservative

decisions

regarding

whether

a

change

in

a

particular

metric

is

likely

to

be

of

interest

to

the

adaptation

engine

and

should

thus

serve

as

a

basis

for

notification

for

inclusion

in

the

next

metrics

push.

Furthermore,

such

a

temporal

history

could

also

facilitate

intelligent

decisions

regarding

the

collection

of

metrics

since

for

example

a

metric

that

is

known

to

be

largely

constant

need

not

be

frequently

measured.

Future

work

will

also

involve

the

evaluation

of

a

broad

range

of

adaptation

scenarios

on

the

MobJeX

framework

to

quantity

the

gains

that

can

be

made

via

adaptation

through

object

mobility

and

thus

demonstrate

in

practise,

the

efficacy

of

the

solution

described

in

this

paper.

Finally,

the

authors

wish

to

explore

applying

the

metrics

collection

concepts

described

in

this

paper

to

a

more

general

and

reusable

context

management

system

REFERENCES

Katz,

R.H.,

Adaptation

and

Mobility

in

Wireless

Information

Systems.

IEEE

Personal

Communications,

p.

Hirschfeld,

R.

and

Kawamura,

K.

Dynamic

Service

Adaptation.

in

ICDCS

Lemlouma,

T.

and

Layaida,

N.

Context-Aware

Adaptation

for

Mobile

Devices.

in

Proceedings

of

IEEE

International

Conference

on

Mobile

Data

Management

Noble,

B.D.,

et

al.

Agile

Application-Aware

Adaptation

for

Mobility.

in

Proc.

of

the

ACM

Symposium

on

Operating

Systems

and

Principles

SOSP.

Saint-Malo,

France.

Rossi,

P.

and

Ryan,

C.

An

Empirical

Evaluation

of

Dynamic

Local

Adaptation

for

Distributed

Mobile

Applications.

in

Proc.

of

International

Symposium

on

Distributed

Objects

and

Applications

(DOA

Larnaca,

Cyprus:

SpringerVerlag.

Ryan,

C.

and

Westhorpe,

C.

Application

Adaptation

through

Transparent

and

Portable

Object

Mobility

in

Java.

in

International

Symposium

on

Distributed

Objects

and

Applications

(DOA

Larnaca,

Cyprus:

SpringerVerlag.

da

Silva

e

Silva,

F.J.,

Endler,

M.,

and

Kon,

F.

Developing

Adaptive

Distributed

Applications:

A

Framework

Overview

and

Experimental

Results.

in

On

The

Move

to

Meaningful

Internet

Systems

CoopIS,

DOA,

and

ODBASE

(LNCS

Rossi,

P.

and

Fernandez,

G.

Definition

and

validation

of

design

metrics

for

distributed

applications.

in

Ninth

International

Software

Metrics

Symposium.

Sydney:

IEEE.

Ryan,

C.

and

Rossi,

P.

Software,

Performance

and

Resource

Utilisation

Metrics

for

Context

Aware

Mobile

Applications.

in

Proceedings

of

International

Software

Metrics

Symposium

IEEE

Metrics

Como,

Italy.

Recursion

Software

Inc.

Voyager

URL:

Holder,

O.,

Ben-Shaul,

I.,

and

Gazit,

H.,

System

Support

for

Dynamic

Layout

of

Distributed

Applications.

TechinonIsrael

Institute

of

Technology.

p.

Holder,

O.,

Ben-Shaul,

I.,

and

Gazit,

H.

Dynamic

Layout

of

Distributed

Applications

in

FarGo.

in

Int'l

Conf.

Software

Engineering

ACM

Press.

Philippsen,

M.

and

Zenger,

M.,

JavaParty

Transparent

Remote

Objects

in

Java.

Concurrency:

Practice

and

Experience,

p.

Shapiro,

M.

Structure

and

Encapsulation

in

Distributed

Systems:

the

Proxy

Principle.

in

Intl.

Conference

on

Distributed

Computing

Systems.

Cambridge,

Mass.

(USA):

IEEE.

Gazit,

H.,

Ben-Shaul,

I.,

and

Holder,

O.

Monitoring-Based

Dynamic

Relocation

of

Components

in

Fargo.

in

Proceedings

of

the

Second

International

Symposium

on

Agent

Systems

and

Applications

and

Fourth

International

Symposium

on

Mobile

Agents.

Lindholm,

T.

and

Yellin,

F.,

The

Java

Virtual

Machine

Specification

Edition.

Addison-Wesley.

Randell,

L.G.,

Holst,

L.G.,

and

Bolmsjö,

G.S.

Incremental

System

Development

of

Large

Discrete-Event

Simulation

Models.

in

Proceedings

of

the

conference

on

Winter

Simulation.

Phoenix,

Arizona.

Waldo,

J.,

Remote

Procedure

Calls

and

Java

Remote

Method

Invocation.

IEEE

Concurrency,

p.

Rolia,

J.

and

Lin,

B.

Consistency

Issues

in

Distributed

Application

Performance

Metrics.

in

Proceedings

of

the

Conference

of

the

Centre

for

Advanced

Studies

on

Collaborative

Research.

Toronto,

Canada.

Henricksen,

K.

and

Indulska,

J.

A

software

engineering

framework

for

context-aware

pervasive

computing.

in

Proceedings

of

the

IEEE

Conference

on

Pervasive

Computing

and

Communications

(PerCom).

Orlando.

O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B I I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B I O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B I O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O

data
mobile object framework
mobjex
java
metricscontainer
metric collection
proxy
performance and scalability
measurement
propagation and delivery
framework
adaptation
mobile object


Addressing Strategic Behavior in a Deployed Microeconomic Resource Allocator ABSTRACT While market-based systems have long been proposed as solutions for distributed resource allocation , few have been deployed for production use in real computer systems . Towards this end , we present our initial experience using Mirage , a microeconomic resource allocation system based on a repeated combinatorial auction . Mirage allocates time on a heavily-used 148-node wireless sensor network testbed . In particular , we focus on observed strategic user behavior over a four-month period in which 312,148 node hours were allocated across 11 research projects . Based on these results , we present a set of key challenges for market-based resource allocation systems based on repeated combinatorial auctions . Finally , we propose refinements to the system 's current auction scheme to mitigate the strategies observed to date and also comment on some initial steps toward building an approximately strategyproof repeated combinatorial auction . 1 . INTRODUCTION Market-based systems have long been proposed as solutions for resource allocation in distributed systems including computational Grids -LSB- 2 , 20 -RSB- , wide-area network testbeds -LSB- 9 -RSB- , and peer-to-peer systems -LSB- 17 -RSB- . Yet , while the theoretical underpinnings of market-based schemes have made significant strides in recent years , practical integration of market-based mechanisms into real computer systems and empirical observations of such systems under real workloads has remained an elusive goal . Towards this end , we have designed , implemented , and deployed a microeconomic resource allocation system called Mirage -LSB- 3 -RSB- for scheduling testbed time on a 148-node wireless sensor network -LRB- SensorNet -RRB- testbed at Intel Research . The system , which employs a repeated combinatorial auction -LSB- 5 , 14 -RSB- to schedule allocations , has been in production use for over four months and has scheduled over 312,148 node hours across 11 research projects to date . In designing and deploying Mirage , we had three primary goals . First , we wanted to validate whether a market-based resource allocation scheme was necessary at all . An economic problem only exists when resources are scarce . Therefore , a key goal was to first measure both resource contention and the range of underlying valuations users place on the resources during periods of resource scarcity . Second , we wanted to observe how users would actually behave in a market-based environment . Much of economic theory is predicated on rational user behavior , which forms the basis for motivating research efforts such as strategyproof mechanism design -LSB- 4 , 6 , 15 , 16 , 19 -RSB- . With Mirage , we wanted to observe to what extent rationality held and in what ways users would attempt to strategize and game the system . Finally , we wanted to identify what other practical problems would emerge in a deployment of a market based system . In this paper , we report briefly on our first goal while focusing primarily on the second . The third is left for future work . Empirical results based on four months of usage have validated the key motivating factors in using an auction-based scheme -LRB- i.e. , significant resource contention and widely varying valuations -RRB- but have also pointed to real-world observations of strategic user behavior . In deploying Mirage , we made the early decision to base the system on a repeated combinatorial auction known not to be strategyproof . That is , self-interested users could attempt to increase their personal gain , at the expense of others , by not revealing their true value to the system . We made this decision mainly because designing a strategyproof mechanism remains an open , challenging problem and we wanted to deploy a working system and gain experience with real users to address our three goals in a timely manner . Deploying a non-strategyproof mechanism also had the benefit of testing rationality and seeing how and to what extent users would try to game the system . The key contribution of this paper is an analysis of such strategic behavior as observed over a four-month time period and proposed refinements for mitigating such behavior en route to building an approximately strategyproof repeated combinatorial auction . The rest of this paper is organized as follows . In Section 2 , we present an overview of Mirage including high-level observations on usage over a four-month period . In Section 3 , we examine strategic user behavior , focusing on the four primary types of strategies employed by users in the system . Based on these results , Section 4 presents a set of key challenges for market-based resource allocation systems based on repeated combinatorial auctions . As a first step in addressing some of these challenges , we describe refinements to Mirage 's current auction scheme that mitigate the strategies observed to date and also comment on some initial steps towards building an approximately strategyproof repeated combinatorial auction for Mirage . Finally , in Section 5 , we conclude the paper . 2 . THE MIRAGE SYSTEM SensorNet testbeds are a critical tool for developing and evaluating SensorNet technology in a controlled and instrumented environment . As with many large-scale systems , however , resource management is a key problem given that it is not economical for users to each build and operate their own testbed . In Mirage -LSB- 3 -RSB- , testbed resources are spaceshared and allocated using a repeated combinatorial auction in a closed virtual currency environment . Users compete for testbed resources by submitting bids which specify resource combinations of interest in space/time -LRB- e.g. , `` any 32 MICA2 motes for 8 hours anytime in the next two days '' -RRB- along with a maximum value amount the user is willing to pay . A combinatorial auction is then periodically run to determine the winning bids based on supply and demand while maximizing aggregate utility delivered to users . Figure 1 : Testbed utilization for 97 MICA2 motes . In Mirage , resources are allocated using a first-price combinatorial auction which clears every hour . In each round of the auction , a rolling window of future testbed resources is available for allocation with subsets of that window being removed from the pool as resources get allocated . In our initial deployment , we used a 72-hour window and deployed the system on a testbed consisting of 148 nodes -LRB- 97 MICA2 -LSB- 1 -RSB- and 51 MICA2DOT sensor nodes or `` motes '' -RRB- . In each round of the auction , users bid for subsets of resources available in the current window . When the system is first brought online , a full 148-node × 72-hour window is available , where each row of the window represents the availability of a particular node across time , and each column represents the availability of the testbed for a given hour . The leftmost column of the window represents node availability for the hour immediately following the auction ; these node/hours will never again be available for auction . All other node/hours not allocated at this or previous auctions continue to be offered for sale at subsequent auctions . In each subsequent round -LRB- i.e. , every hour -RRB- , portions of the current window are allocated as bids are matched to available resources and a new rightmost 148-node × 1-hour column of resources rolls in and replaces the leftmost column of resources which expires . There is no time sharing of nodes : given limited local computation and communication power , once a sensor is allocated to a user for a particular time period , it is unavailable to all other users . In Mirage , users place combinatorial bids specifying resource combinations of interest in space/time along with a maximum value amount the user is willing to pay . More specifically , a bid bi = -LRB- vi , si , ti , di , fmin , fmax , ni , oki -RRB- indicates the user wants any combination of ni motes from the set oki simultaneously for a duration of di hours -LRB- di ∈ -LCB- 1 , 2 , ... , 32 -RCB- -RRB- , a start time anywhere between si and ti , and a radio frequency in the range -LSB- fmin , fmax -RSB- . ' The user also is willing to pay up to vi units of virtual currency for these resources . In essence , each bid specifies in a succinct manner what subsets of the resource window would serve as acceptable resources that meet the user 's constraints and how important the desired resource allocation is to the user . We deployed Mirage on December 9 , 2004 and the system has been in continuous production use for over four months . In the process , its lifetime has overlapped with several periods of significant resource contention including the SIGCOMM ’05 and SenSys ’05 conference deadlines . Overall , the system has 18 research projects registered to use the system spanning a variety of academic and commercial institutions . Of these , 11 have actively bid and received time on the system . As of April 8 , 2005 , the system has received 322 bids , and allocated 312,148 node hours over the testbed 's 148 nodes . Figure 2 : Bid value distributions by user . As a measure of contention , Figure 1 shows the utilization of the 97 MICA2 motes over the past four months . It depicts periods of significant contention extending over multiple consecutive days , in particular near major deadlines .2 To quantify user valuations for resources , Figure 2 ` The frequency constraints are used to schedule testbed allocations such that allocations co-scheduled in time do not collide by using the same radio frequency . In practice , distinct frequencies have not been a scarce resource . 2Results for the 51 MICA2DOT motes are similar and omitted for space . plots distributions of bid values per node hour for the seven most active users in the system . This graph shows that user valuations for testbed resources varied substantially , spanning over four orders of magnitude . Valuations are also distributed relatively evenly across each order of magnitude , suggesting that these ranges are not due to a few anomalous bids but rather to a wide range of underlying user valuations for testbed resources . These dual observations -- significant resource contention and a wide range of valuations -- support the use of an auction , which is designed precisely to harness such widely varying valuations to compute an efficient and user utility-maximizing node allocation . Lastly , as another measure of resource contention and the utility of driving resource allocation via user-specified valuations , Figure 3 plots the median per-node clearing price for both MICA2 and MICA2DOT motes over time . To compute these prices , we price an allocated node-hour for a winning bid with value v for n nodes for k hours as v/nk . Unallocated node-hours are assigned a price of 0 . For a given hour , we examine all MICA2 motes and plot the median node-hour price for that hour and do the same for MICA2DOT motes . Of particular interest in this graph are the sequence of prices from days 45 -- 60 and days 105 -- 120 -LRB- i.e. , periods leading up to conference deadlines -RRB- . These sequences show that the value of testbed resources , as measured by market prices for motes , increases exponentially -LRB- logarithmic y-axis -RRB- during times of peak contention . This suggests that allowing users to express valuations for resources to drive the resource allocation process is important for making effective use of the testbed -LRB- e.g. , to distinguish important use from low priority activities -RRB- . However , it also suggests that users become exponentially desperate to acquire resources as deadlines approach . As it turns out , it is precisely during these times that users will try their hardest to game the system and , therefore , when the efficacy of a market-based mechanism can be best evaluated . Figure 3 : Median node-hour market prices . 3 . OBSERVED STRATEGIC BEHAVIOR During the past four months of operation , Mirage has employed two distinct auction mechanisms and observed four primary types of strategic behavior from users . The first auction mechanism , A1 , was deployed from December 9 , 2004 to March 28 , 2005 . During this time period , we observed three different types of strategic behavior -LRB- S1-S3 -RRB- , the most recent of which -LRB- S3 -RRB- resulted in significant gaming of the system . In response to the impact of S3 , we deployed a second mechanism , A2 , on March 29 , 2005 -LRB- Day 111 in Figures 1 and 3 -RRB- . While A2 mitigated or eliminated the known shortcomings of A1 -- in particular the vulnerability strategy S3 exploited that prompted the change in the first place -- it was soon discovered that A2 remained vulnerable to another strategy , S4 , which was predictably discovered and exploited by a motivated user community . We are currently in the process of designing a mechanism to address the weakness in A2 that is abused by S4 . Of course , ideally we would develop a provably strategyproof mechanism . However , this remains an open research problem for repeated combinatorial auctions . In this section , we describe the two auction mechanisms A1 and A2 , Mirage 's virtual currency policy , the four types of observed strategic behavior S1 -- S4 , and their impact on aggregate utility delivered . 3.1 Auctions and virtual currency Our first auction mechanism , A1 was a first-price , openbid -LRB- i.e. , users can see all outstanding bids from competing users -RRB- combinatorial auction that cleared every hour based on a greedy algorithm . In each round of auction , the current set of bids was sorted by value per node hour and bids were greedily fit into the remaining portion of the current window of available resources . Like A1 , our second auction , A2 , was also based on a greedy clearing algorithm . Its key differences were that -LRB- i -RRB- it was a sealed-bid auction and -LRB- ii -RRB- it allocated resources over a 148-node × 104-hour window with bid start times constrained to be within the next 72 hours -LRB- the reason for this will become apparent when we discuss strategy S3 -RRB- . In both auctions , winning bids from previous auctions were publicly visible for price feedback and the same virtual currency policy was used . Our virtual currency policy assigns two numbers to each user 's bank account : a baseline value and a number of shares . When created , each bank account is initialized to its baseline value . Once funded , a user can then begin to bid and acquire testbed resources through Mirage . In each round of the auction , accounts for winning bids are debited and the proceeds are redistributed through a proportional profit-sharing policy based on bank account share values . The primary purpose of this policy is to reward users who refrain from using the system during times of peak demand and penalize those who use resources aggressively during periods of scarcity . These rewards result in transient bursts of credit and are balanced by another mechanism , a savings tax , to prevent idle users from sitting on large amounts of excess credit forever -LRB- a `` use it or lose it '' policy -RRB- . In our deployment , an administrator set the virtual currency policy . Bank accounts for external users were assigned baseline and shares value set to 1000 , while bank accounts for internal users -LRB- U4 and U5 -RRB- were assigned larger allocations with baseline and share values set to 2000 . 3.2 Strategic behavior The following are descriptions of the four primary bidding strategies observed over the past four-months . S1 : underbidding based on current demand . In A1 , all outstanding bids were publicly visible . Consequently , when users would observe a lack of demand , some users would bid correspondingly low amounts rather than their true values . For example , one user would frequently bid 1 or 2 when no other bids were present . While underbidding in the absence of competition is not a problem per se , it does raise two issues . First , if a seller was collecting revenue for profit , such bidding leads to suboptimal outcomes for the seller . Second , should other users enter competing bids before the auction clears , users will need to refine their bids to allow the system to compute an allocation that maximizes aggregate utility . This second problem then leads to strategy S2 . S2 : iterative bidding . Because users are allowed to modify their bids and A1 was an open auction , iteratively refining one 's bid value in response to other users ' bid values should , in theory , have no effect on who wins the auction ; users with higher valuations -- who may also be underbidding -- should eventually outbid those with lower valuations after sufficient iteration . The problem is that users do not behave this -LRB- rational -RRB- way . Usability overhead matters : users in Mirage bid once and perhaps modify their bid a second time . The end result is that inefficiencies may arise since the auction may clear with bid values that are understated . While bidding proxies that automatically adjust user bids in response to other bids are effective in single-good auctions , it is unclear how such proxies could be generally effective in a combinatorial auction without actually implementing the same clearing algorithm used by Mirage -LRB- which could be computationally expensive -RRB- . In summary , S1 and S2 both point toward the need for a strategyproof auction mechanism in Mirage . In such an auction , a user 's optimal strategy is always to bid truthfully the first time . Thus , rational users will never underbid and iterative bidding is unnecessary . S3 : rolling window manipulation . Unlike auctions for tangible goods , resource allocation in computer systems fundamentally involves time , since sharing of resources implies that a resource can not be assigned to a given user/process forever . In Mirage , we addressed the issue of time by selling resources over a rolling window 72 hours into the future with users able to bid for blocks 1 , 2 , ... , or 32 hours in length . What we did not anticipate , however , was what would happen when the entire window of resources becomes fully allocated . In this scenario , which was the norm near the recent SenSys ’05 deadline , the entire 148-node × 72hour window is allocated . A user bidding for , say , 32 hours thus needs to minimally wait 32 hours for 32 new 148-node × 1-hour columns of resources to become available . The problem here is that a user can game the system by observing other bids and simply requesting fewer hours . Since 16 columns will roll into the resource window before 32 columns , a user bidding for 16 hours outbids a 32-hour bid independent of each bid 's value because resources for the 32-hour bid are not available when the auction clears . Of course , if other users also begin bidding for 16 hours , this opportunity disappears but then moves to durations shorter than 16 hours . In the limit , all users bid for 1-hour blocks , thereby eliminating the possibility of obtaining longer resource allocations which may be critical to the underlying SensorNet experiment . In practice , we observed this type of gaming push winning bid durations down to 2 hours . With rampant gaming of the system occurring through S3 , we responded by implementing and deploying auction A2 . As mentioned , a key difference of A2 compared to A1 is that it allocates resources over a 104-hour window with bid start times constrained to be within the next 72 hours . In expanding the window and expanding -LRB- while still constraining -RRB- the range of start times , A2 eliminates strategy S3 . When the entire 148-node × 72-hour window is allocated , a pending 16-hour bid and a pending 32-hour bid will both Table 1 : Strategy S4 on 97 MICA2 motes . have their first opportunity for an allocation when 32 new columns become available . At that point , both the 16-hour bid and the 32-hour bid will have an opportunity to obtain an allocation . Such allocations are then determined by the usual greedy clearing algorithm . S4 : auction sandwich attack . While A2 eliminated S3 and significantly reduced S1 and S2 , it still retained a weakness of A1 that had yet to be discovered and exploited . In the auction sandwich attack , a user exploits two pieces of information : -LRB- i -RRB- historical information on previous winning bids to estimate the current workload and -LRB- ii -RRB- the greedy nature of the auction clearing algorithm . In this particular case , a user employs a strategy of splitting a bid for 97 MICA2 motes across several bids , only one of which has a high value per node hour . Since the high value bid is likely to win due to the greedy nature of the auction clearing algorithm and since all other users at the time were all requesting 97 motes -LRB- based on the historical information and the fact that the SenSys ’05 deadline was imminent requiring experiments at scale -RRB- , no other bids could backfill the remaining slots ; the user 's remaining bids would then fit those slots at a low price . An actual occurrence is shown in Table 1 . Here , user U1 uses three bids , the main one being a bid with value 130 -LRB- value per node hour 130 / -LRB- 4 · 40 -RRB- = 0.813 -RRB- which is used to outbid a bid with value 1590 -LRB- value per node hour 1590 / -LRB- 32 · 97 -RRB- = 0.0512 -RRB- . Once the high valued 40-node bid has occupied its portion of the resource window , no other 97-node bids can be matched . Consequently , the user backfills on the remaining 57 nodes using two bids : a 24-node bid and a 33-node bid , both at low valuations . 4 . CHALLENGES AND REFINEMENTS Designing an appropriate auction mechanism is key to addressing the above strategies . Specifically , our goals for such a mechanism include : -LRB- i -RRB- strategyproofness , -LRB- ii -RRB- computational tractability , and -LRB- iii -RRB- optimal allocation . The Generalized Vickrey Auction -LRB- GVA -RRB- -LSB- 8 , 18 -RSB- is the only known combinatorial mechanism that provides both strategyproofness and optimal allocation . However , it also is computationally intractable as it is NP-hard to calculate the allocations as well as individual payments . Other VCG-based mechanisms exist that replace the allocation algorithms in GVA with approximate ones to provide tractability . In this case , however , strategyproofness is no longer available -LSB- 16 -RSB- . These goals are in conflict for VCG and in general -LSB- 10 -RSB- . We thus must make certain trade-offs . With this in mind , we now present a two-phase roadmap for improving Mirage : -LRB- i -RRB- short-term improvements to the current mechanism that mitigate the effects of existing strategies ; and -LRB- ii -RRB- designing a new mechanism that approximately achieves our three goals simultaneously . 4.1 Ongoing improvements Our first improvement is a mixed-integer programming -LRB- MIP -RRB- formulation as an alternative to the greedy algorithm . This is aimed directly at eliminating strategy S4 . While the MIP does not provide strategyproofness , it is able to compute approximately-optimal allocations . Like the GVA , however , the MIP is computationally demanding and thus careful formulation of the MIP and optimizations based on the observed workloads from Mirage will be required to ensure timely clearing of the auction . Our first step is to test and optimize our MIP-based algorithm on auction data from the past four months . We can then run both the MIP alongside the greedy algorithm in parallel and select the higher quality result each time the auction clears . Second , we can also augment the auction with additional rules and fees to further mitigate strategic behavior . To eliminate S4 , two possibilities are to restrict each user to having either one outstanding bid at a time or to mandate that users are not allowed to have multiple overlapping allocations in time . To mitigate S1 and S2 , we could add transaction fees . With such fees in place , a user who understates a bid and intends to iteratively refine it will have a disincentive to do so given that each iteration incurs a fee . Finally , another approach to eliminating S4 is to modify the greedy algorithm such that if users do have bids whose allocations could overlap in time , then those potential allocations are considered from lowest to highest value per node hour . In effect , this allows bids for overlapping allocations but creates a disincentive for users to place such bids . 4.2 Towards a strategyproof mechanism Clearly , we need to evaluate our goals and identify where we can make trade-offs in designing a new mechanism . Computational tractability is a fundamental requirement for operational reasons . Strategyproofness or , minimally , making the system hard to manipulate is also key given the behavior we have observed . Finally , our mechanism should compute near-optimal allocations given our compute time budget . Among the potential mechanisms we can extend , the LOS scheme -LSB- 12 -RSB- seems to be a good starting point . It is a fast algorithm as the allocation rule is a greedy mechanism , ranking bids with some `` norm '' such as value per node hour . The advantage of LOS is its special payment scheme that is tightly linked to the greedy allocation . Essentially , a winner i pays the `` norm '' of the first bidder denied access times the amount of units -LRB- i.e. node hours -RRB- that i won . This feature makes it strategyproof . The main downside , however , is that it assumes users are single-minded , meaning that each bidder only cares about a specific set of goods -LRB- e.g. , a specific list of nodes for specific durations -RRB- and they do not value anything else . Unfortunately , this is highly restrictive and contradicts what Mirage currently offers its users , namely the ability to select any subset of nodes for any slots and submit multiple bids . Thus , LOS is vulnerable to S4 and to avoid it we must find a way to extend LOS and its strategyproof property to satisfy complex-bidders . Realistically , even with a strategyproof LOS scheme for complex bidders there will likely be further strategies we have yet to encounter and that we should consider in our design . For instance , our discussion so far focuses on strategyproofness within a single auction . Across auctions , however , there may be temporal strategies that are possible . For example , in a particular auction , suppose the highest bidder wants all nodes and pays , using GVA payment scheme for simplicity , the next bidder 's value . This same bidder may be better off by waiting until the next auction , if the user can still win and face bidders that have even lower values . In this case , the user will gain additional utility due to a lower payment . This , however , may create various problems as total revenue , total value , as well as allocative efficiency across the auctions may be adversely affected . There are two techniques we can use to address temporal strategies . The first is a `` wrapper '' scheme such as the one employed by Virtual Worlds -LRB- VW -RRB- -LSB- 13 -RSB- that makes sequences of individually strategyproof auctions -LRB- e.g. , LOS -RRB- strategyproof . What VW does is , after bidder i wins , it tracks what would have happened if i had submitted in a subsequent auction instead . Specifically , it tracks what i would have paid in all following auctions during i 's patience -LRB- i.e. , the maximum time i is willing to wait for an allocation -RRB- and keeps track of the lowest possible payment . i will instead be charged the lowest payment and will thus have no incentive to temporally game the system . Alternatively , the new class of online mechanisms -LSB- 7 , 11 -RSB- assumes dynamic arrival and departure of bidders and does not hold auctions at fixed intervals . Instead , the mechanism is a continuous scheme that accepts bids as they arrive and makes allocation decisions immediately , thus removing any need to `` clear '' auctions . The challenge is that the current literature is still restricted to non-combinatorial settings . 5 . CONCLUSION Despite initially using a repeated combinatorial auction known not to be strategyproof , Mirage has shown significant promise as a vehicle for SensorNet testbed allocation . The dual observations of significant resource contention and a wide range of valuations suggest that auction-based schemes can deliver large improvements in aggregate utility when compared to traditional approaches such as proportional share allocation or batch scheduling . Fully realizing these gains , however , requires addressing key problems in strategyproof mechanism design and combinatorial optimization . The temporal nature of computational resources and the combinatorial resource demands of distributed applications adds an additional layer of complexity . Nevertheless , we remain optimistic and believe that a pragmatic mix of theory and practice combined with iterative improvements on real deployments provides one promising avenue toward bringing market-based resource allocation into the mainstream .

O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O

market-based system
distribution system
strategy behavior
ration
auction-based scheme
node-hour price
usable overhead
batch schedule

resource allocation system
combinatorial auction
mirage system
sensornet testbed
distributed applications
resource allocation