Self-calibration

and

visual

SLAM

with

a

multi-camera

system

on

a

micro

aerial

vehicle

The

use

of

a

multi-camera

system

enables

a

robot

to

obtain

a

surround

view,

and

thus,

maximize

its

perceptual

awareness

of

its

environment.

An

accurate

calibration

is

a

necessary

prerequisite

if

vision-based

simultaneous

localization

and

mapping

(vSLAM)

is

expected

to

provide

reliable

pose

estimates

for

a

micro

aerial

vehicle

(MAV)

with

a

multi-camera

system.

On

our

MAV,

we

set

up

each

camera

pair

in

a

stereo

configuration.

We

propose

a

novel

vSLAM-based

self-calibration

method

for

a

multi-sensor

system

that

includes

multiple

calibrated

stereo

cameras

and

an

inertial

measurement

unit

(IMU).

Our

selfcalibration

estimates

the

transform

with

metric

scale

between

each

camera

and

the

IMU.

Once

the

MAV

is

calibrated,

the

MAV

is

able

to

estimate

its

global

pose

via

a

multi-camera

vSLAM

implementation

based

on

the

generalized

camera

model.

We

propose

a

novel

minimal

and

linear

algorithm

that

uses

inertial

information

to

recover

the

relative

motion

of

the

MAV

with

metric

scale.

Our

constant-time

vSLAM

implementation

with

loop

closures

runs

on-board

the

MAV

in

real-time.

To

the

best

of

our

knowledge,

no

published

work

has

demonstrated

realtime

on-board

vSLAM

with

loop

closures.

We

show

experimental

results

in

both

indoor

and

outdoor

environments.

The

code

for

both

the

self-calibration

and

vSLAM

is

available

as

a

set

of

ROS

packages

at

MAVs

are

more

versatile

than

laser-based

MAVs.

Whereas

a

laser

only

provides

geometry

data,

a

camera

can

provide

both

geometry

data

via

stereo

and

structure-frommotion

techniques,

and

appearance

data.

A

camera

is

a

passive

sensor

while

a

laser

is

an

active

sensor

and

is

thus

susceptible

to

interference.

Furthermore,

a

camera

is

lighter

and

has

a

smaller

footprint.

However,

utmost

care

has

to

be

taken

when

choosing

the

camera

configuration

for

a

vision-based

MAV

expected

to

operate

robustly

in

challenging

environments.

A

single-camera

configuration

introduces

limited

perceptual

awareness,

and

in

turn,

flight

constraints

because

if

the

camera

observes

too

few

features

for

some

time,

localization

can

fail

and

lead

to

a

crash.

In

the

case

of

a

single

downward-looking

camera

the

MAV

cannot

fly

too

close

to

the

ground

which

often

has

little

texture,

and

at

the

same

time,

it

cannot

perform

obstacle

avoidance

due

to

the

absence

of

a

forward-looking

camera.

In

the

case

of

a

forward-looking

camera,

constraints

are

imposed

on

the

path

planning.

For

example,

in

any

planned

path

ensures

that

there

are

a

sufficient

number

of

features

for

localization.

In

addition,

in

a

path

is

planned

such

that

the

MAV

does

not

move

outside

the

camera's

field

of

view,

and

inadvertently

crash

into

an

unseen

obstacle.In

this

paper,

we

use

a

multi-camera

system

on

a

MAV;

together

with

the

use

of

fish-eye

lenses,

this

camera

configuration

provides

a

surround

view

of

the

vicinity.

Our

multi-camera

Vision-based

MAVs

are

more

versatile

than

laser-based

MAVs.

Whereas

a

laser

only

provides

geometry

data,

a

camera

can

provide

both

geometry

data

via

stereo

and

structure-frommotion

techniques,

and

appearance

data.

A

camera

is

a

passive

sensor

while

a

laser

is

an

active

sensor

and

is

thus

susceptible

to

interference.

Furthermore,

a

camera

is

lighter

and

has

a

smaller

footprint.

However,

utmost

care

has

to

be

taken

when

choosing

the

camera

configuration

for

a

vision-based

MAV

expected

to

operate

robustly

in

challenging

environments.

A

single-camera

configuration

introduces

limited

perceptual

awareness,

and

in

turn,

flight

constraints

because

if

the

camera

observes

too

few

features

for

some

time,

localization

can

fail

and

lead

to

a

crash.

In

the

case

of

a

single

downward-looking

camera

the

MAV

cannot

fly

too

close

to

the

ground

which

often

has

little

texture,

and

at

the

same

time,

it

cannot

perform

obstacle

avoidance

due

to

the

absence

of

a

forward-looking

camera.

In

the

case

of

a

forward-looking

camera,

constraints

are

imposed

on

the

path

planning.

For

example,

in

any

planned

path

ensures

that

there

are

a

sufficient

number

of

features

for

localization.

In

addition,

in

a

path

is

planned

such

that

the

MAV

does

not

move

outside

the

camera's

field

of

view,

and

inadvertently

crash

into

an

unseen

obstacle..

In

this

paper,

we

use

a

multi-camera

system

on

a

MAV;

together

with

the

use

of

fish-eye

lenses,

this

camera

configuration

provides

a

surround

view

of

the

vicinity.

Our

multi-camera

Fig.

We

show

an

image

from

each

of

the

four

fish-eye

cameras

on

our

MAV

platform.

Our

camera

configuration

provides

a

surround

view.

Each

pair

of

cameras

is

arranged

in

a

stereo

configuration

so

that

scene

information

is

available

at

all

times.

approach

is

similar

in

spirit

to

Furgale

et

al.

who

uses

a

car

equipped

with

multiple

cameras

configured

to

provide

a

surround

view.

The

use

of

such

a

multi-camera

system

eliminates

the

constraints

on

path

planning

associated

with

a

single

camera

as

discussed

earlier,

and

allows

for

immediate

path

planning

in

all

directions.

Furthermore,

localization

is

more

robust

as

there

is

always

at

least

one

camera

that

observes

a

sufficient

number

of

features.

Another

benefit

of

using

multiple

cameras

is

that

we

directly

infer

metric

scale..

Our

MAV

is

self-contained

in

the

sense

that

all

algorithms

necessary

for

autonomous

flight

are

run

on-board.

With

this

self-contained

MAV,

we

circumvent

latency

and

reliability

issues

related

to

off-board

computing,

especially

in

areas

where

Wi-Fi

reception

is

poor.

On

the

other

hand,

more

cameras

correlate

to

a

requirement

for

more

computational

resources,

but

with

recent

advances

in

computing

hardware,

especially

in

multi-core

processors,

it

is

now

computationally

feasible

to

run

image

processing

algorithms

on-board

a

MAV

that

uses

multiple

cameras..

For

robust

operation

in

the

field,

we

arrange

each

camera

pair

in

a

stereo

configuration

such

that

scene

information

is

always

available.

Thus,

we

avoid

the

use

of

failure-prone

map

initialization

methods

required

for

monocular

cameras

and

multi-camera

systems

with

non-overlapping

fields

of

view..

The

use

of

multiple

cameras

on

a

MAV

raises

two

issues

we

have

to

resolve

before

we

can

realize

autonomous

flight:

finding

the

inter-sensor

transforms,

and

estimating

the

MAV's

pose

with

multiple

synchronized

sensors.

We

come

up

with

novel

methods

for

self-calibration

and

pose

estimation

based

on

the

generalized

camera

model

Our

self-calibration

method

is

designed

for

a

multi-sensor

system

with

multiple

calibrated

stereo

cameras

and

an

IMU;

we

do

not

assume

overlapping

fields

of

view

between

any

two

stereo

cameras.

This

method

is

based

on

vSLAM,

only

assumes

that

each

stereo

camera

is

pre-calibrated,

and

does

not

require

operator

input,

fiducial

markers

or

external

positioning

systems.

We

are

able

to

estimate

the

extrinsic

calibration

parameters

with

metric

scale

which

is

inferred

from

stereo.

We

do

not

know

of

other

existing

vSLAM-based

self-calibration

methods

for

multiple

calibrated

stereo

cameras.

We

propose

a

minimal

and

linear

solution

for

motion

estimation

that

uses

inertial

information

to

recover

the

relative

motion

with

metric

scale.

We

incrementally

build

a

graph

of

keyframes

and

constraints,

and

use

the

double-window

optimization

method

to

optimize

this

graph.

This

graph

optimization

enables

real-time

on-board

SLAM

with

loop

closures

on

a

vision-based

MAV,

which

to

the

best

of

our

knowledge,

has

not

been

shown

before

in

published

works.

There

is

an

extensive

body

of

work

on

calibration

for

multisensor

systems.

For

tractability,

we

look

at

current

state-of-theart

methods

that

involve

cameras.

Kelly

and

Sukhatme

perform

visual-inertial

SLAM

with

a

camera-IMU

system,

and

at

the

same

time,

estimate

the

calibration

parameters

including

the

camera-IMU

transform.

However,

the

calibration

parameters

are

only

accurate

as

long

as

the

camera-IMU

system

experiences

constant

excitation.

Furgale

et

al.

proposes

an

offline

method

to

find

an

accurate

estimate

of

the

transform

and

the

temporal

offset

between

a

camera

and

an

IMU

using

a

calibration

pattern.

Here,

this

method

makes

use

of

temporal

basis

functions

for

parameterization

of

continuoustime

variables,

and

continuous-time

batch

estimation.

If

we

use

this

approach

to

calibrate

each

camera-IMU

pair,

the

intercamera

transforms

may

not

be

accurate

as

we

do

not

consider

camera-camera

constraints

in

the

form

of

scene

points

mutually

observed

by

multiple

cameras.

Brookshire

and

Teller

take

a

more

generic

approach

similar

to

hand-eye

calibration

by

calibrating

a

multi-sensor

system

based

on

relative

motion

measurements

for

each

sensor.

In

contrast,

in

our

selfcalibration,

we

exploit

scene

points

mutually

observed

by

multiple

cameras

to

obtain

a

more

accurate

estimate

of

inter-camera

transforms.

Carrera

et

al.

Heng

et

al.

develop

self-calibration

methods

for

multi-camera

systems

that

are

based

on

vSLAM.

However,

only

estimates

the

inter-camera

transforms

up

to

scale

while

infers

metric

scale

from

odometry

data.

In

contrast,

by

utilizing

calibrated

stereo,

our

self-calibration

method

estimates

the

inter-camera

transforms

with

metric

scale

without

requiring

odometry

data..

We

explore

existing

work

on

SLAM

with

a

multi-camera

system

Kaess

and

Dellaert

solve

an

optimization

problem

comprising

pose-point

constraints

and

odometry

constraints

in

order

to

obtain

the

pose

of

the

multicamera

system,

and

do

not

perform

loop

closure

detection.

Kim

et

al.

models

a

multi-camera

system

as

a

spherical

camera.

The

drawback

with

this

spherical

model

is

that

the

relative

motion

of

the

system

can

only

be

estimated

up

to

scale.

Furthermore,

they

do

not

show

experiments

with

their

multi-camera

system

on

unmanned

aerial

vehicles.

Carrera

et

al.

uses

a

forward-looking

camera

and

a

backwardlooking

camera

on

a

ground

robot,

and

implements

pose-graph

SLAM

based

on

odometry

data

and

loop

closure

detection.

The

absence

of

bundle

adjustment

in

any

form

limits

the

metric

accuracy

of

the

map.

Tribou

implements

a

multi-camera

version

of

PTAM

that

makes

use

of

a

three-camera

system

with

non-overlapping

fields

of

view

on

a

MAV.

As

initial

scene

point

depths

are

not

known,

an

initialized

map

does

not

have

accurate

metric

scale,

and

bundle

adjustment

that

runs

in

a

separate

thread

is

relied

on

to

gradually

recover

metric

scale.

Due

to

the

cubic

complexity

of

bundle

adjustment,

the

approach

of

Tribou

does

not

scale

to

large

environments.

Furthermore,

pose

estimates

with

incorrect

metric

scale

can

cause

control

instability

issues.

In

addition,

loop

closures

are

not

performed.

Li

et

al.

shows

that

in

the

case

of

only

intra-camera

feature

correspondences,

and

the

relative

rotation

being

an

identity

matrix,

the

generalized

epipolar

constraint

reduces

to

the

epipolar

constraint

which

only

allows

us

to

recover

the

relative

pose

up

to

scale.

Due

to

the

absence

of

inter-camera

feature

correspondences

from

non-overlapping

fields

of

view,

metric

scale

cannot

be

recovered

if

the

MAV

moves

with

minimal

rotation.

In

contrast,

wide

overlapping

fields

of

views

for

each

stereo

camera

on

our

MAV

ensure

that

we

always

have

inter-camera

feature

correspondences,

and

thus,

are

able

to

obtain

pose

estimates

with

metric

scale

all

the

time.

Furthermore,

our

use

of

double-window

optimization

allows

our

SLAM

implementation

to

run

in

constant

time,

and

thus,

scale

to

large

environments.

We

then

look

at

vision-based

MAVs

that

run

real-time

pose

estimation

algorithms

on-board.

Schauwecker

and

Zell

deploy

one

downward-looking

stereo

camera

and

one

forwardlooking

stereo

camera

on

a

MAV,

independently

estimate

the

MAV's

pose

from

each

camera,

and

fuse

both

pose

estimates.

In

contrast,

we

use

all

cameras

to

obtain

a

single

pose

estimate.

Weiss

et

al.

utilizes

both

an

IMU

and

downwardlooking

camera

together

with

a

modified

version

of

PTAM

to

estimate

the

MAV's

pose,

and

infer

scale

from

accelerometer

measurements.

Schmid

et

al.

uses

a

FPGA

board

to

compute

depth

maps

from

a

forward-looking

stereo

camera

and

relies

on

stereo

visual

odometry

for

pose

estimation.

Shen

et

al.

uses

a

forward-looking

stereo

camera

with

fish-eye

lenses.

Here,

they

use

gyroscopic

measurements

to

filter

out

incorrect

feature

correspondences,

and

uses

a

local

map

to

estimate

the

MAV's

pose.

We

note

that

the

pose

estimation

in

all

discussed

works

is

susceptible

to

drift

as

loop

closure

detection

is

not

carried

out.

To

the

best

of

our

knowledge,

there

is

no

published

work

on

real-time

on-board

SLAM

with

loop

closures

for

vision-based

MAVs..

A

number

of

works

exploit

the

vertical

direction

information

provided

by

the

IMU

to

simplify

motion

estimation

algorithms;

the

vertical

direction

is

not

susceptible

to

drift

unlike

gyroscopic

measurements.

However,

the

IMU

present

on

our

MAV

is

of

the

MEMS

type,

and

the

maximum

error

of

the

vertical

direction

is

degrees

as

measured

with

a

Vicon

motion

capture

system.

As

a

result,

we

did

not

get

accurate

relative

pose

estimates

from

using

the

known

vertical

in

our

motion

estimation

algorithm.

B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Factors of characteristic words of irrational numbers Let be an irrational number between 0 and 1. The characteristic word f ( ) of is defined to be the infinite word over {0, 1} whose nth letter is [(n + 1) ] − [n ], n 1. It is well known that, for each m 1, f ( ) has exactly m + 1 distinct factors of length m. In this paper, we shall develop a method to construct these factors. Under our construction, the 1-sets of these m + 1 factors x (m) 0 , x (m) 1 , . . . , x (m) m are determined, these factors are increasing in the lexicographic order and their moments M(x (m) 0 ), M(x (m) 1 ), . . . , M(x (m) m ) form an increasing sequence of m + 1 consecutive integers. Some known results about generating factors of f ( ) using the unbordered -words and their conjugates turn out to be consequences of our main theorem. Without loss of generality, we consider only binary words throughout this paper. For w = c 1 c 2 · · · c n , where c i ∈ {0, 1}, n 1, let T (w) = c 2 · · · c n c 1 , w = c n · · · c 2 c 1 , . The nonnegative number M(w) is called the moment of w. The set E(w) is called the 1-set of w. Let T 0 (w) = w and T i (w) = T (T i−1 (w)), where i 1. The words T i (w), i 0, are called the conjugates of w. The positive integer n is called the length of w and is denoted by |w|. The length of the empty word is defined to be 0. The number of occurrences of the letter 1 in w is denoted by |w| 1 , and is called the height of w.. Let w and x be nonempty words. The word x is said to be a proper prefix (resp., suffix) of w, denoted by x < p w (resp., x < s w), if there is a nonempty word y such that w = xy (resp., w = yx); x is said to be a factor of w if there are words y and z such that w = yxz;. x is said to be a cyclic factor of w if |x| |w| and x is a factor of w 2 ; x is said to be a border of w if x < p w and x < s w; the word w is said to be unbordered if it has no border. If a word w = xy, where x, y are nonempty, we write wy −1 = x and x −1 w = y. Let w = c 1 c 2 · · · c n ,where c i ∈ {0, 1}. A positive integer s is said to be a period of w if c i+s = c i , for i = 1, 2, . . . , n − s. Denote by F (w) the set of all factors of the word w. For any set B of words, define F (B) = ∪{F (w) : w ∈ B}.. Let be a real number with 0 < < 1. Let f ( ) be an infinite binary word whose nth letter is [(n + 1) ]− [n ] (n 1). The infinite words f ( ) is called the characteristic word of . A characteristic word is also called a standard Sturmian word. The characteristic words have been studied extensively (see for example [5, 7, [10] [11] [12] [17] [18] [19] 22, 24, 26] ). The finite factors of characteristic words are called finite Sturmian words. An enumeration formula for finite Sturmian words was first proved by Mignosi [23] . Other proofs were given in [2, 15] . Let St denote the set of all finite Sturmian words. A 1-1 correspondence was established between St and a set of triplets of integers [3, 20] . See also [16] . Let denote the set of all words w of the form w = Cxy = AB, where A, B, C are palindromes, x, y ∈ {0, 1}, and x = y. Let PER denote the set of all words w having two periods s and t such that gcd(s, t) = 1, and |w| s + t − 2. Using the theorem of Fine and Wilf, de Luca and Mignosi [15] showed that St = F ( ) = F (PER).. The following theorem is well known [24] .. Theorem A. Let be an irrational number between 0 and 1. For each m 1, the characteristic word f ( ) has exactly m + 1 distinct factors of length m.. To generate factors of f ( ), the present author makes use of the unbordered -words and their conjugates [10] . A word w is said to be an -word if either w ∈ {0, 1} or there are positive integers n, a 1 , a 2 , . . . , a n and integers r i (1 i n), with 0 r 1 a 1 −1, 0 r i a i (2 i n), and words w 0 , w 1 , . . . , w n with w 0 = 0, w 1 = 0 a 1 −1−r 1 10 r 1 , w n = w, and. In the latter case, the word w is denoted by w(a 1 , . . . , a n ; r 1 , . . . , r n ). It has length q and height p, where p and q are relatively prime integers with p/q = [0, a 1 , a 2 , . . . , a n ]. The -word w(a 1 , . . . , a n ; 0, 0, . . . , 0) is also called a finite standard Sturmian word. It is known that each -word w(a 1 , . . . , a n ; r 1 , . . . , r n ) is a conjugate of the -word w(a 1 , . . . , a n ; 0, 0, . . . , 0) [10] . If p/q = [0, a 1 , a 2 , . . . , a n ] = [0, b 1 , b 2 , . . . , b m ], where n is odd and m is even, then w(a 1 , . . . , a n ; 0, a 2 , 0, . . . , a n−1 , [1, 4] . Denote this word by w(p/q). Clearly w(p/q) starts with the letter 0 and ends with the letter 1 if q 2. It is a Lyndon Sturmian word as well as a Christoffel primitive word [1, 4] . Its conjugates are the only -words having length q and height p. Among these conjugates, w(p/q) and its reversal are the only unbordered -words [12] . Now let be an irrational number with continued fraction expansion [0, a 1 , a 2 , . . .]. Let p −1 = 1, p 0 = 0, q −1 = 0, q 0 = 1, p n,r = rp n−1 + p n−2 , q n,r = rq n−1 + q n−2 , 1 r a n , p n = p n,a n , q n = q n,a n , n 1.. (1.1). Let w n,r = w(p n,r /q n,r ) and w n = w n,a n (n 1, 1 r a n ). For n 1, 1 r a n and q n,r 2, the first (resp., last) letter of w n,r is 0 (resp., 1). Let y n,r = w n,r 1 −1 0 (n is odd), 10 −1 w n,r (n is even), y n = y n,a n , n 1, 1 r a n . Clearly |y n,r | = q n,r .. Let S = {0} ∪ {w n,r : n 1, 1 r a n }. The set S consists of all Lyndon -words that are factors of f ( ) [7, 8] ; elements of S and their conjugates are building blocks of the characteristic word f ( ) and their suffixes [11] ; a factors of f ( ) is unbordered if and only if either w ∈ S or w ∈ S [12] . The unborderedness of these words leads to the following theorem, which has some interesting reformulations and consequences about factors of f ( ) of any given length [10] .. Theorem B. Let w = w n,r or w n,r , y = y n,r , q = q n,r 2 and 1 m < q. In this paper, we shall develop another method for generating the m + 1 factors of f ( ) of length m (Theorem 3.1). As we shall see, the 1-sets of the m+1 factors x (m). will be determined, the factors so constructed will be an increasing sequence in the lexicographic order and their moments will form an increasing sequence of consecutive integers. More precise description of these phenomenon will be given in Corollaries 4.1 and 4.3. The proof of Theorem 3.1 is independent of Theorem B. As a matter of fact, Theorem B is a consequence of Theorem 3.1 (see Remark 4.4).

O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Which process metrics can significantly improve defect prediction models? An empirical study Abstract The knowledge about the software metrics which serve as defect indicators is vital for the efficient allocation of resources for quality assurance. It is the process metrics, although sometimes difficult to collect, which have recently become popular with regard to defect prediction. However, in order to identify rightly the process metrics which are actually worth collecting, we need the evidence validating their ability to improve the product metric-based defect prediction models. This paper presents an empirical evaluation in which several process metrics were investigated in order to identify the ones which significantly improve the defect prediction models based on product metrics. Data from a wide range of software projects (both, industrial and open source) were collected. The predictions of the models that use only product metrics (simple models) were compared with the predictions of the models which used product metrics, as well as one of the process metrics under scrutiny (advanced models). To decide whether the improvements were significant or not, statistical tests were performed and effect sizes were calculated. The advanced defect prediction models trained on a data set containing product metrics and additionally Number of Distinct Committers (NDC) were significantly better than the simple models without NDC, while the effect size was medium and the probability of superiority (PS) of the advanced models over simple ones was high (p ¼ :016, r ¼ À:29, PS ¼ :76), which is a substantial finding useful in defect prediction. A similar result with slightly smaller PS was achieved by the advanced models trained on a data set containing product metrics and additionally all of the investigated process metrics (p ¼ :038, r ¼ À:29, PS ¼ :68). The advanced models trained on a data set containing product metrics and additionally Number of Modified Lines (NML) were significantly better than the simple models without NML, but the effect size was small (p ¼ :038, r ¼ :06). Hence, 393-422 DOI 10.1007/s11219-014-9241-7 it is reasonable to recommend the NDC process metric in building the defect prediction models. Software development companies are seeking for ways to improve the quality of software systems without allocating too many resources in the quality assurance activities such as testing. Applying the same testing effort to all modules of a software system is not an optimal approach, since the distribution of defects among individual parts of a system is not uniform. According to Pareto-Zipf-type law (Boehm and Papaccio 1988; Denaro and Pezzè 2002; Endres and Rombach 2003) , the 80:20 empirical rule is operating here, i.e., a small amount of code (often quantified as 20 % of the code) is responsible for the majority of software faults (often quantified as 80 % of the faults). Therefore, it is possible to test only a small part of a software system and find most of the defects. Defect prediction models, in turn, may be used to find the defect-prone classes. Hence, the quality assurance efforts should be focused (unless for critical projects) on the most defect-prone classes in order to save valuable time and financial resources, and, at the same time, to increase the quality of delivered software products.. The defect prediction models built on the basis of product metrics are already well known (Basili et al. 1996; Denaro and Pezzè 2002; Gyimothy et al. 2005; Tang et al. 1999) ; however, also the process metrics have recently become popular 1 . Fenton was not only among the first who have criticized the product metric-based approach (Fenton and Ohlsson 2000) , but also the one who suggested a model based only on the project and the process metrics (Fenton et al. 2007 ). There are also other studies in which the process metrics are investigated (Illes-Seifert and Paech 2010; Schröter et al. 2006) , as well as used in the model (Graves et al. 2000; Weyuker et al. 2008 Weyuker et al. , 2010 . Nevertheless, there are no conclusive results. Usually, only the correlations between some process metrics and the defect count are investigated, e.g. (Illes-Seifert and Paech 2010; Schröter et al. 2006) . When defect prediction models are built, they are either not compared with a productbased approach (e.g., Bell et al. 2006; Hassan 2009; Ostrand et al. 2005; Weyuker et al. 2006 Weyuker et al. , 2007 , they are built on a small sample (e.g., Graves et al. 2000; Moser et al. 2008) or do not perform statistical tests and effect size calculations to conclude whether the improvements obtained through adding the process metrics were of both, statistical and practical significance even when improvements were impressive (e.g., Nagappan et al. 2008) . Effect size is an index that quantifies the degree of practical significance of study results, i.e., the degree to which the study results should be considered important, or negligible, regardless of the size of the study sample. Further discussion of related work is given in detail in Sect. 3. This paper presents the results of an empirical study exploring the relationship between the process metrics and the number of defects. For that purpose, the correlations between particular process metrics and the number of defects were calculated. Subsequently, the simple defect prediction models were built on the basis of the product metrics. With those simple models, we were able to build advanced defect prediction models by introducing, additionally, one of the process metrics at a time. As a result, we were able to compare the simple and the advanced models and answer the question whether or not the introduction of the selected process metric improved the adequacy of the predictions. Statistical methods were used to evaluate the significance of that improvement. The approach used in this study can be easily put into practice, which is its distinct advantage. Moreover, no sophisticated methods were used to build the prediction models, but the ordinary stepwise linear regression. Even though they are probably neither best nor the most effective for this purpose, stepwise linear regression methods are widely known and, therefore, reduce the learning effort.. The derivation of the baseline model, as well as the experiments presented in this paper, intend to reflect the industrial reality. Since the product metrics have a very long history (e.g., McCabe 1976), they enjoy a good tool support (e.g., the Ckjm tool used in this study) and are well understood by practitioners. We may assume that there are companies interested in defect prediction which have already launched a metric program and collect the product metrics. The assumption is plausible, as such companies are already known to the authors of this paper. A hypothetical company as described above is using product metrics for the aforementioned reasons (mainly tool support). Unfortunately, the prediction results are often unsatisfactory; therefore, new metrics may be employed in order to improve the prediction. The process metrics can be particularly useful, since they reflect the attributes different from those associated with the product metrics, namely the product history, which is (hopefully) an extra source of information. Nevertheless, it is still not obvious what the company should do, as there are a number of process metrics which are being investigated with regard to defect prediction. Furthermore, the results are sometimes contradictory (see Sect. 3 for details). Moreover, the tool support for the process metrics is far from being perfect, e.g., for the sake of this study, the authors had to develop their own solution to calculate these metrics. Bearing in mind that hypothetical situation in an industrial environment and relying on their direct and indirect experience, the authors of this study chose as its main objective to provide assistance in making key decisions regarding which metric (or metrics) should be chosen and added to the metric program in order to improve the predictions and not to waste financial resources on checking all the possibilities. Therefore, we have analyzed which of the frequently used process metrics can significantly improve defect prediction-on the basis of a wide range of software projects from different environments. The construction of the models made use solely of the data which were historically older than the ones used in prediction (model evaluation). For example, the model built on the data from the release i was used to make predictions in release i þ 1. The data from ith release are usually (or at least can be) available during the development of ði þ 1Þth release. Hopefully, on the basis of the empirical evaluations presented in this paper, development teams may take informed decisions (at least to some extent, as the number of analyzed projects, although large, is not infinite) about the process metrics which may be worth collecting in order to improve the defect prediction models based on product metrics. Additionally, the framework of the empirical evaluation of the models presented in this paper can be reused in different environments to evaluate new kinds of metrics and to improve the defect prediction models even further. This paper is organized as follows: The descriptions of all the investigated product and process metrics, as well as the tools employed for data collection and the investigated software projects are described in Sect. 2. Related empirical studies concerning the process metrics are presented in Sect. 3. Section 4 contains the detailed description of our empirical investigation aimed at identifying the process metrics which may significantly improve the defect prediction models based on the product metrics. The obtained results are reported in Sect. 5, while threats to validity are discussed in Sect. 6. The discussion of results in Sect. 7 is followed by the conclusions and contributions in Sect. 8. In this section, we analyze the relationship between the number of defects in a software class and the values of process metrics.

O B I O O O B I I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I I O O O O O O O O O B I O O O O O O O O O O O O B I I O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O B I I O O O O O O O B I O O O O O O O O O O B I I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O B I I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I


Look-Ahead Dynamic Threshold Voltage Control Scheme for Improving Write Margin of SOI-7T-SRAM Instability of SRAM memory cells derived from aggressive technology scaling has been recently one of the most significant issues. Although a 7T-SRAM cell with an area-tolerable separated read port improves read margins even at sub-1V, it unfortunately results in degradation of write margins. In order to assist the write operation, we address a new memory cell employing a look-ahead body-bias which dynamically controls the threshold voltage. Simulation results have shown improvement in both the write margins and access time without increasing the leakage power derived from the body-bias. key words: PD-SOI, body-bias, SRAM, low power design In a previous scenario of SRAM development, technology scaling had been able to achieve the higher performance, larger memory capacity, and lower power consumption at the same time so far. However, when it comes to a sub100nm era, an unexpected random variation in the threshold voltage (V th ) derived from the process variation and the lowered supply voltage drastically deteriorates the stability of SRAM memory cells. Then, we propose a 7T-SRAM cell adopting the active body-biasing control on PD-SOI, where the V th of each transistor can be dynamically controlled through the direct body contact [1] , [2] . The use of word/bit line signals as the control signals for memory cells improves both the write margins and access time. In order to improve the deteriorated write margin of a 7T-SRAM cell, we have proposed a memory cell adopting a look-ahead body-bias which dynamically controls the threshold voltage in order to assist the write operation. Simulation results have shown improvement in both the write margins and access time without increasing the leakage power derived from the body-bias.

O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B B B B I I O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B


Toward a higher-level visual representation for content-based image retrieval Having effective methods to access the desired images is essential nowadays with the availability of a huge amount of digital images. The proposed approach is based on an analogy between content-based image retrieval and text retrieval. The aim of the approach is to build a meaningful mid-level representation of images to be used later on for matching between a query image and other images in the desired database. The approach is based firstly on constructing different visual words using local patch extraction and fusion of descriptors. Secondly, we introduce a new method using multilayer pLSA to eliminate the noisiest words generated by the vocabulary building process. Thirdly, a new spatial weighting scheme is introduced that consists of weighting visual words according to the probability of each visual word to belong to each of the n Gaussian. Finally, we construct visual phrases from groups of visual words that are involved in strong association rules. Experimental results show that our approach outperforms the results of traditional image retrieval techniques. Due to the explosive spread of digital devices, the amount of digital content grows rapidly. The increasing need for automatic processing, description, and structuring of large digital archives motivates to have an effective content-based image retrieval (CBIR) [29] . In typical CBIR systems, it is always important to select an appropriate representation for images. Indeed, the quality of the retrieval depends on the quality of the internal representation for the content of the visual documents [2] . Recently, many image retrieval systems have shown that the partbased representation for image retrieval [37] is much superior to traditional global features. Indeed, one single image feature computed over the entire image is not sufficient to represent important local characteristics of different objects within the image.. Nowadays, bag-of-visual-words [17, 28, 32] has drawn much attention. Analogous to document representation in terms of words in text domain, the bag-of-visual-words approach models an image as an unordered bag of visual words, which are formed by vector quantization of local region descriptors. This approach achieves good results in representing variable object appearances caused by changes in pose, scale and translation. Despite the success of the bag-of-visual-words approach in recent studies, the precision of image retrieval is still incomparable to its analogy in text domain, i.e. document retrieval, because of many important drawbacks.. Firstly, most of the local descriptors are based on the intensity or gradient information of images, so neither shape nor color information is used. In the proposed approach, in addition to the SURF descriptor that was proposed by Bay et al. [3] , we introduce a novel descriptor (Edge context) that is based on the distribution of edge points.. Secondly, since the bag-of-visual-words approach represents an image as a collection of local descriptors, ignoring their order within the image, the resulting model provides a rare amount of information about the spatial structure of the image. In this paper, we propose a new spatial weighting scheme that consists of weighting visual words according to the probability of each visual word to belong to one of the n Gaussian in the 5-dimensional color-spatial feature space.. Thirdly, the low discrimination power of visual words leads to low correlations between the image features and their semantics. In our work, we build a higher-level representation, namely visual phrase from groups of adjacent words using association rules extracted with the Apriori algorithm [1] . Having a higher-level representation, from mining the occurrence of groups of low-level features (visual words), enhances the image representation with more discriminative power since structural information is added.. The remainder of the article is structured as follows: Section 2 reviews related works to the proposed approach. In Section 3, we describe the method for constructing visual words from images. In Section 4, we describe the method for mining visual phrases from visual words to obtain the final image representation. In Section 5, we present an image similarity method based on visual words and visual phrases. We report the experimental results in Section 6, and we give a conclusion to this article in Section 7. We successfully drew out an analogy between the techniques in text retrieval and image retrieval by building visual words and phrases. Visual words were constructed by extracting features based on the fusion of descriptors. A new "spatial weighting" technique has been introduced, which enhances the basic 'bag of-visualwords' approach by using spatial relations. Finally, we devised the methods to construct visual phrases based on association rule technique. Our experimental studies showed that a combined use of words and phrases could perform better than using them separately. It also showed good performance compared to similar recent approaches.. In our future work, we will investigate the usage of such model on proposing computer vision solutions like human behavior analysis from video. We will work on further justification based on other datasets such as TRECVID and NUS-WIDE. This section describes the set of experiments we have performed to explore the performance of the proposed methodology. Firstly, we investigate the performance of the proposed approach and the average number of visual words on each class of images after filtering the noisy visual words. Secondly, we evaluate the effect of fusion Edge context descriptor with SURF. Thirdly, we compare the performance of the spatial weighting scheme with the traditional bag-of-visual-words. Fourthly, we evaluate the contribution between visual words and visual phrases. Finally, we compare our approach to Zheng et al. approach [38] .

O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O B O O O O O O O B I O O O O O O O O O O O B I O B I O O O O O O O O O O O O


An FSM Reengineering Approach to Sequential Circuit Synthesis by State Splitting demonstrated that in on-demand RF read architectures, there is scope for further rfa reduction and proposed several bypass-aware instruction scheduling techniques aimed at reducing the number of access to the register file. Our experiments on the Intel XScale processor pipeline with on-demand RF read running MiBench benchmarks show that up to 26% and, on average, 12% rfa can be reduced. Further, one of our scheduling techniques, which is RFPN2, is an effective heuristic to reduce the number of rfa (11.4% on average) without much loss in performance (less than 1% on average) and within a reasonable compilation time. We have demonstrated that our compilation technique consistently reduces the number of the rfa on various bypass configurations. REFERENCESLin Yuan, Member, IEEE, Gang Qu, Senior Member, IEEE, Tiziano Villa, and Alberto Sangiovanni-Vincentelli, Fellow, IEEE Abstract-This paper presents a finite-state machine (FSM) reengineering method that enhances the FSM synthesis by reconstructing a functionally equivalent but topologically different FSM based on the optimization objective. This method enables the FSM synthesis algorithms to explore a set of functionally equivalent FSMs and obtain better solutions than those in the original FSM. To demonstrate the effectiveness of the proposed method, we apply it to popular power-and area-driven FSM synthesis algorithms, respectively. Our method achieves an average of 5.5% power reduction and 2.7% area reduction, respectively, on 25 Microelectronics Center of North Carolina (MCNC) FSM benchmarks, where the proposed method is applicable. This is a significant performance improvement for the power-and area-driven FSM synthesis algorithms being used. Our method has a negligible run-time overhead, and it maintains the quality of the synthesis solutions.Index Terms-Finite-state machine (FSM) encoding, power minimization, sequential logic synthesis, state splitting. Finite-state machine (FSM) synthesis is a well-studied problem. It consists of state minimization (SM) and state encoding (SE) procedures. SM finds a functionally equivalent FSM that has the minimum number of states. SE assigns distinct binary codes to each state of the FSM such that the sequential circuit modeled by the FSM can be efficient in terms of area, performance, and/or power.. The SM problem can be optimally solved in completely specified FSMs [6] , and there are standard approaches to solving the SM problem in incompletely specified machines [9] . On the other hand, there have been many techniques to solve the SE problem based on different optimization objectives and implementation technologies.. In area-driven FSM synthesis, De Micheli et al. proposed an SE algorithm to minimize area in a programmable logic array implementation by generating a minimum (multivalued) symbolic cover of the FSM followed by a step of satisfying the encoding constraints [13] . Successive extensions also introduced output constraints and more efficient algorithms to satisfy the input and output encoding constraints [18] , [19] . MUSTANG [3] is one of the earliest state encoding techniques for multilevel logic minimization; it assigns a weight to each pair of symbols and gives adjacent codes to pairs of states with large weight. JEDI [11] adopts a weighted graph model similar to the one in MUSTANG, but it uses a simulated annealing algorithm to perform the embedding. Instead, MUSE [4] and MIS-MV [12] apply multivalued minimization followed by extraction and satisfaction of encoding constraints, generalizing it to multilevel logic.. In power-driven FSM synthesis, the optimization objective of SE is often formulated as minimizing the total switching activity in the circuit in light of the fact that dynamic power is proportional to the switching activity. Roy and Prasad proposed a simulated annealing algorithm to reduce the switching activity in conventional area-driven state encoding algorithms [15] . Washabaugh et al. assigned weights to each pair of states based on the state transition probability between them. Then, they used a branch-and-bound algorithm to minimize the total weighted sum of state transitions [20] . Based on the same weighted state transition graph (STG) model, Olson and Kang applied a genetic algorithm to optimize the SE solution; in addition, they also considered area minimization during encoding to achieve different area-power tradeoffs [14] . Later, Benini and De Micheli proposed POW3, a greedy algorithm that assigns binary codes to states bit by bit. At each step, the codes are selected to minimize the number of states with different partial codes [2] . This algorithm has low run-time and very good synthesis results. Moreover, Iman and Pedram developed a complete low-power FSM synthesis framework that minimizes the switching activity both in the state registers and combinational circuit [8] .. All previous FSM synthesis approaches start with SM and look for the best encoding solution in the minimized FSM. Such serial strategy, however, is a heuristic that disregards the fact that a state-minimized FSM is not necessarily the best starting point for SE. Concurrent SM and SE were proposed in [1] , [5] , [7] , and [10] , with little success. Hallbauer [7] proposed a method for asynchronous circuits based on pseudodichotomies trying to perform SM while heuristically reducing the encoding length, with no reported results. To explore the solution space in the nonminimized FSM, the method of Lee and Perkowski [10] employed a branch-and-bound technique; however, it is only feasible for very small machines (no more than 16 states). Avedillo et al. [1] presented a heuristic method in which the encoding is incrementally generated and may create incompletely specified codes for the states in the original FSM. Although reasonably efficient, the experimental results on a subset of the Microelectronics Center of North Carolina (MCNC) benchmarks do not show improvements over a serial synthesis strategy. Fuhrer and Nowick proposed OPTIMIST [5] , a concurrent SM and SE algorithm for two-level logic implementation. It provides an exact solution to FSM optimization for two-level logic implementation. However, the largest FSM in the reported table of results has only nine states, and the authors pointed out that their algorithm does not scale well.. To overcome the pitfalls of locality for a serial strategy and computational bottlenecks in concurrent approaches, we propose FSM reengineering, a performance enhancement method to FSM synthesis. Our method consists of three phases. First, we encode a minimized FSM. Then, based on the encoded solution, we reconstruct a functionally equivalent FSM with increased number of states. Finally, we reencode the new enlarged FSM. This method enables the SE algorithms to explore a larger solution space composed of functionally equivalent but nonminimized FSMs. Unlike the concurrent SM and SE approaches, our method has much lower runtime complexity and, therefore, is capable of handling large FSMs. Furthermore, this method can be applied on top of an existing FSM synthesis flow to improve the solution quality. An extended version of this paper can be found in [21] . We introduced an FSM reengineering method to improve the FSM synthesis tool. It reduces the effect of locality in a serial synthesis strategy and has lower complexity than concurrent SM and SE. Based on the existing power and area cost function, we have seen improvements on the existing power-and area-driven SE algorithms.

O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I B I I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Developing an optimized application hosting framework in Clouds A Cloud system, which is also often called a data center, may host a large number of application servers (termed as applications for short in this paper) that consume various types of resource, such as CPU, memory, and I/O. In such a scenario, the application placement strategy, which specifies how to host applications across the physical servers in the Cloud, is critical to achieve good system-and/or client-oriented performance. The existing application placement algorithms in literature assume that the consumption of system resources is proportional to the level of workloads submitted to the system. In this paper, we reveal that it may not be the case in some circumstances. Based on this observation, we design and implement an application placement framework, called EAPAC, for Clouds. The developed framework is able to achieve the optimized performance by judiciously allocating to the applications a proper mixture of different types of application request as well as an appropriate number of requests in each type. Further, we investigate the issue of resource conflicts among different applications when there exist concurrent requests in the system. We have conducted extensive experiments to evaluate the performance of the developed framework. The experiment results show that compared with the existing method in literature, EAPAC can improve the performance by 35% in terms of the reply rate. Application placement is an importance issue in Clouds (which are also called data centers). In this paper, we design, implement, and evaluate an application placement framework, EAPAC, for Cloud environments. EAPAC analyzes the Web requests before making application placement decisions, aiming to avoid resource competition among different applications. We carried out extensive experiments for performance evaluation. Compared with the existing application placement method in literature, EAPAC improves throughput by 30% for all applications we tested. Moreover, the workload is better balanced by EAPAC, and consequently it achieves more stable and shorter response time.. Our future work includes the following directions: (1) We will integrate the virtual machine technology into EAPAC, aiming to enable EAPAC to support resource provision [28, 24] ; (2) Currently, EAPAC handles the resource competition from different applications. We will address the resource co-allocation for applications [5, 25, 14] ; (3) In data centers, the fine-granularity monitoring can improve the back-end resource utilization. We will plan to strike a balance between the monitoring granularity and the monitoring overhead; (4) Currently, we only use a simple WLC load balancing algorithm. In the future, we will explore more advanced load balancing algorithms to study the relationship between the load balancing algorithm and the application placement algorithm. Existing application placement methods in literature [27] usually treat all types of request equally and estimate a server's resource consumption by simply adding up the resource consumption caused by each individual request when it is running in the server alone. However, our studies show that the resource consumptions estimated by this approach may not be very accurate when multiple requests are running in the system concurrently. The inaccuracy may reduce the efficiency of the request/application scheduling strategies proposed in literature.. We conducted the benchmarking experiments to demonstrate this situation. The experiment platform consists of two IBM HS21 nodes taken from High Performance Computing Center 1 at the Huazhong University of Science and Technology.. Each HS21 node is equipped with a 2-way, 4-core Intel Xeon CPU E5345 running at 2.33GHz, and with 8GB memory and a 73GB SCSI hard disk. The two nodes are interconnected via a Gigabit Ethernet network and a 10Gbps Infiniband. Both nodes run the RedHat Enterprise Linux 5 (Linux 2.6. 18-8.el5 ). An emulated data-intensive application on JBoss is implemented, which creates 100MB data when receiving a request. One node is used to generate and send the requests using Apache Benchmark (AB), 2 while the other node serves the requests, each of which invokes an instance of the emulated application.. In total, 1000 http requests are generated, and they are sent in the following four patterns, which represent different concurrency levels: (1) one request every time; (2) two concurrent requests every time; (3) four concurrent requests every time; (4) eight concurrent requests every time. Fig. 1 shows the results of the average processing time of one request in the aforementioned four scenarios. The x-axis represents the concurrency level, and y-axis is the average processing time of a request. Different legends in the figure correspond to the time spent in different types of operations in the benchmark application. As can be seen from this figure, the "usr", "iowait" and "sys" time dominate the processing time. The "iowait" time refers to the time when the CPU waits for the completion of I/O operations. The "usr" and "sys" time are the time when the application is running in the user and kernel space of the operation system, respectively. Note that the processing time in the figure is the time when the CPU is physically occupied for processing the request (not the duration between the time when the request starts execution and the time when it finishes). It can be observed from this figure that the physical processing time for a request increases as the concurrency level increases. For example, the processing time for the concurrency level of 8 is about 4 seconds, while the physical processing time for the concurrency level of 1 is about 1.2 seconds. This means that when there are 8 requests running in the server concurrently, their total CPU consumption is not 1.2 × 8 = 9.6 seconds, but 4 × 8 = 36 seconds. Fig. 2 shows the processing time of each request as the above experiments progressed. It can be verified from this figure that when the concurrency level increases, the processing time of the requests increases. Further observations in this figure show that when the concurrency level is 1, 2 and 4, the processing times of the requests are mostly distributed around certain values (which are about the average processing time shown in Fig. 1 ). However, when the concurrency level is 8, the processing times become volatile, which suggests that the system is now in an unstable state. The results observed in this figure verify those obtained in Fig. 1 . Moreover, they indicate that the system may become unstable when the concurrency level is too big (this phenomenon will be discussed in more detail later in this paper). The results obtained in the above benchmarking experiments deviate from the assumption widely made in the existing studies of application scheduling. The existing studies assume that the system employs a perfect time-sharing fashion to run multiple application instances concurrently and therefore, the processing time of an instance increases linearly with the number of instances running in the system. From another perspective, the benchmarking results obtained in this paper suggest that the CPU consumption is not necessarily proportional to the level of workload submitted to the system. The underlying reason for the non-linearity may be due to the workload management strategies employed in the operating system. There are two possible solutions for this problem. One is to revise the operation system kernel. The other lies in the application level by judiciously allocating to application servers both an appropriate amount and combination of different types of request. This paper focuses on the latter approach and develops a new application placement framework, EAPAC. There is a growing interest in the efficient management of data centers and hosting platforms. Some of them target the algorithms of hosting applications. For example, Kimbrel et al. [9] and Tang et al. [27] proposed the dynamic application placement algorithms in response to the changes in application demands. Their work is close to EAPAC. However, as illustrated in Section 1, EAPAC analyzes the requests and make application placement decisions by taking into account both concurrency of the requests and resource conflicts of the applications. The algorithm proposed by Urgaonkar et al. [29] allows the applications to share machines. However, the algorithm only considers a single bottleneck resource, does not dynamically change the number of instances of an application. Moreover, the algorithm does not aim to minimize the placement changes. The placement problems have also been modeled using various approaches, including bin packing, multiple knapsack, and multi-dimensional knapsack, etc. [19] . These modeling techniques are the theoretical foundation of the application placement strategy used in EAPAC. Some research works target different resource-sharing environments [6, 15, 22, 23] . Qin et al. [15] propose two I/O-aware load-balancing schemes for two types of clusters, which address the scheduling algorithm for IO-intensive applications. In EAPAC, the application placement considers the IO impacts on CPU consuming. Jiang et al. [6] proposed a service-ondemand framework for application service with virtual machines. Chase et al. proposed a dynamic virtual cluster which shares resources at the granularity of the entire machines [2] . In EAPAC, however, the application servers are running in a native, non-virtualized environment.. Placing application replicas is one of the key aspects in a hosting platform. It received much attention in the past. Previous work in this area mostly focused on the algorithms of determining the number and location of the replicas, either in the wide area network [7, 17, 18, 8, 10] or within a given data center. In EAPAC, the applications can be replicated, but the requests are scheduled by the load balancer. The replication algorithm is outside the scope of this paper.. Al-Qudah et al. addressed the problem of efficient enactment of application placement when the number and location of application instances have been determined [1] . Some key technologies was adopted in [1] , such as prefetching [13] . In this paper, we do not address these problems. But these methods can be easily integrated into EAPAC. The agility of hosting platforms has been recently addressed by Qian et al. [16] . Unlike our work, the authors targeted the environments in which application instances are organized in the application-level clusters and hosted on virtual machines. Fig . 6 illustrates the architecture of EAPAC. As mentioned in the previous section, there are two parts in EAPAC: Load Balancer (LB) and Application Placement Manager (APM). The LB forwards the requests to the suitable applications, and the APM handles the application placement on the back-end physical nodes. Besides, there is a testing node in EAPAC, which Each functional module in EAPAC is discussed as follows.

O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O B O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O


Usability evaluation for enterprise SOA APIs SAP recently began offering access to web services through its Enterprise Service-Oriented Architecture (E-SOA) platform. It is in the best interest of SAP that its E-SOA service operations are easier for developers to use and understand, which will contribute to higher E-SOA adoption, and a more effective means of innovation on the part of business customers. To facilitate such a change, Carnegie Mellon University's Human-Computer Interaction Institute is working with SAP's E-SOA and Business Process Renovation Teams to analyze the E-SOA interfaces using HCI techniques and determine means by which developers assigned to create SOA APIs in general, and Enterprise SOA APIs in particular, can design superior interfaces. The identification of usable design patterns, and methodologies to determine these patterns, can streamline SOA projects for API developers and programmers who use SOA APIs. The interface between the developer and the library being utilized is known as the Application Programming Interface (API). Usability issues arising from tradeoffs in API design have severe implications for the productivity of developers. By studying API users, Human-Computer Interaction (HCI) techniques may be applied to evaluate and improve APIs. In our previous studies, we have successfully accomplished this, while also designing effective and valid methodologies for measuring the effects of API design choices [5, 14, 15] .. By analyzing the E-SOA web services APIs, SAP hopes to increase API usability, and therefore the speed, efficiency, and viability of projects which create SOA composite applications. To this end, we are adapting Human-Computer Interaction methodologies and techniques to inform and support the design of SOA APIs.. Although many resources focus on the design of web services, few tested design guidelines exist to enable speedy, intuitive, and effective discovery and consumption of web services by programmers [2, 6, 13] . It is in the best interest of SAP and other web services API designers to increase API usability, so as to reduce the costs and increase the effectiveness of all SOA projects using these APIs.. SAP's E-SOA usability effort faces interesting and novel challenges that may yield general solutions to other SOA API design efforts. First, SAP possesses a massive backend used to create composite applications, which require substantial complexity. There are over 1700 web services available, and these services are grouped in at least two hierarchies: one by licensed process component (Enterprise Resource Planning, Customer Relationship Management, and more) and the other by likely scenario of use. Relationships between service operations complicate their use, and are not always clear. Little is known about how to organize large API frameworks to maximize usability. We are studying SAP's Enterprise SOA service hierarchies to better understand this challenge.. Second, SAP's users may experience conceptual challenges that are not normally anticipated or understood. Part of SAP's business model relies on users licensing software that is installed on servers owned by the customer, resulting in the interesting situation that the service provider and consumer are technically the same, although the users did not create and are not necessarily familiar with the services. The Enterprise SOA user experience is one of both the provision, and the consumption, of hundreds of available services. We plan to explore the usability implications of SOA's asymmetry between creation and consumption.. Third, current stub generators result in API designs that mimic the creation of XML documents, a form of programming that is inconsistent with what is commonly used in the Java and .NET environments. It may be necessary to adhere to document-style Web Service Description Language (WSDL) conventions because the Remote Procedural Call (RPC) format, designed to encourage consistency with nondistributed method calls, is criticized as too tightly-coupled to be interoperable enough to support SOA [1, 9] . However, the design flexibility of stub generators raises the possibility that XML document-style API design does not necessarily need to be imposed by document-style WSDL conventions. Usability research into the generation of API designs from WSDL may result in improvements that could speed the adoption of both E-SOA and SOA as a whole.. Fourth, use of APIs is significantly affected by the IDE used, as well as the documentation provided. After the needed service is located in the documentation, stub generation tools integrated into the IDE are required to produce the client-side code needed to call it. Support for both discovery of services and their technical configuration with the user's IDE is necessary to ensure intuitive consumption.. By examining web service provisioning and consumption in an Enterprise SOA environment, we hope to identify pain points, barriers, and means of improvement that could result in generalized solutions, and support API designers across other circumstances. This paper begins by describing the usability challenges that have been identified so far in our ongoing research, and next describes HCI techniques that are being altered specifically to support API designers in judging design tradeoffs and making informed and justified decisions. We conclude with topics for future research. By adapting Human-Computer Interaction techniques to the unique demands of software development tasks, valuable insights can be gained to help improve E-SOA web service API design, increasing the rate of SOA adoption, lowering the costs of business process redesign, and speeding business innovation. We hope that these insights will generalize to help others create more usable SOA APIs.

B O O B I O B O O O O O B I O O O B I O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B O O B I O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B I O O B O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O