Behavioral clusters in dynamic graphs This paper contributes a method for combining sparse parallel graph algorithms with dense parallel linear algebra algorithms in order to understand dynamic graphs including the temporal behavior of vertices. Our method is the first to cluster vertices in a dynamic graph based on arbitrary temporal behaviors. In order to successfully implement this method, we develop a feature based pipeline for dynamic graphs and apply Nonnegative Matrix Factorization (NMF) to these features. We demonstrate these steps with a sample of the Twitter mentions graph as well as a CAIDA network traffic graph. We contribute and analyze a parallel NMF algorithm presenting both theoretical and empirical studies of performance. This work can be leveraged by graph/network analysts to understand the temporal behavior cluster structure and segmentation structure of dynamic graphs. There are many domains of data analysis that can be modeled with the graph abstraction. In particular we are interested in social networks and internet connection networks. These networks are collections of interactions occurring in complex patterns. Analyzing these patterns is essential to leveraging the information contained in these networks. Because the most important networks are the networks that are in heavy use right now, methods to understand temporal patterns in dynamic networks are important.. The availability of big data has driven an adoption of large scale statistical techniques, both classical and modern. These techniques are not immediately applicable to graph data and this leaves analysts separated from their familiar software tools. In order to connect graph analysis and statistical reasoning, we introduce vertex features which can be calculated efficiently and then analyzed using familiar large scale statistical software tools. This connection is bidirectional because statistical analysis of vertex features informs the computation of additional features. The observed difficulty of writing scalable parallel graph algorithms for scale-free and irregular graphs advises against writing inferential and mathematical code to analyze the graphs directly. In this paper we address this gap by first applying non-inferential graph code to generate vectorial data that is statistically well behaved, then applying a state of the art vectorial technique to this data, which provides insight into the original graph. A representation of this framework is presented in Fig. 1 .. In the massive streaming data analytics model [11] , we view the graph of network events as an unending stream of new edge updates. For each interval of time, we have the static graph, which represents the previous state of the network, and a sequence of edge updates that represent the events since the previous state was recorded. An update can take the form of inserting a new edge, a changing the weight of an existing edge, or a deleting an existing edge. Some networks do not naturally handle deletions, for example Twitter and IP networks where messages are sent and received. In these cases we count the number of messages as the edge weight.. Early work on the theory of streaming algorithms involves summarizing data streams. In a seminal paper by Flajolet and Martin [15] , the data is presented in a streaming context and the number of distinct elements must be counted. The algorithms in this field are streaming but the analysis of that data is not necessarily temporal. Feigenbaum et al. [14] have contributed to one model of streaming graph analysis by considering the ''semi-streaming model'' where graphs are presented ''as a stream of edges in adversarial order'' and the goal is to compute properties of the graph in one or sub-linearly many passes over the edge stream. This semi-streaming model takes the perspective of a fixed graph with limited access to the data. The work addresses the theoretical issues in computing solutions to ''classical graph problems'' with necessary approximations due to the constraints on accessing the edges.. With a dynamically changing graph where only those edges occurring in the past can be accessed, there are a new set of temporal queries to answer. Our work contributes to the analysis of modern graph problems that only appear when the edge set is fluctuating over time. We provide insight into applications of temporal data analysis techniques to large data sets that are well represented by the dynamic graph abstraction. Previous approaches to dynamic graph analysis have leveraged traditional, static graph analysis algorithms to compute an initial metric on the graph and then a final metric on the graph after all updates are processed. The underlying assumption is that the time window is large and the network changes substantially so that the entire metric must be recomputed. However, in the massive streaming data analytics model, algorithms react to much smaller changes on smaller time-scales. For example, given a graph with billions of edges, inserting 100,000 new edges might have a small impact on the overall graph, but it might have a large impact on a small subset of the graph. To accommodate this, in our approach an efficient streaming algorithm recomputes metrics on only the affected regions of the graph. For instance, when considering the betweenness centrality of vertices in IP networks each batch of edges represents approximately 36 s of internet traffic. This approach has shown large speed-ups for evaluating clustering coefficients and connected components on scale-free networks [11, 13] .. In this work we show that the vertex features, as explained above, can be used to generate an understanding of the vertex behavior as well as the behavior of the entire graph as a whole. The nonnegative factorization 1 of these feature matrices provides a clustering of the vertices into groups and a segmentation of the edge stream into phases, which are two important data analysis tasks. These feature matrices are broadly applicable and many applications are beyond the scope of this paper, including tensor factorizations which will provide latent feature based understanding of the three way interaction between the vertices, the different features, and the time-steps. We contribute a novel framework for making inferences about dynamic vertex behavior based on streaming graph computations. This framework provides a new method for detecting clusters of vertices based on temporal behavior. The inferred temporal behavior is interpretable, easy to visualize, and found without explicitly searching for a predefined pattern. We developed a successful feature based pipeline for dynamic graph streams and applied NMF to these features. The computed low rank approximation is useful for several graph analysis tasks including temporal similarity at the graph level, vertex clustering and graph temporal segmentation. We contribute a theoretical and empirical evaluation of a new parallel algorithm for NMF based clustering, which scales up to 12 cores and exhibits good performance up to 50 clusters, even on graphs with large vertex sets. These techniques yield an algorithm which is linear in the number of vertices m and time-steps n when the number of temporal behavior clusters is Oððm þ nÞ 1=4 Þ. A case study analyzing CAIDA internet traces provides understanding into the behavior of the ubiquitous internet connection graph. This feature based pipeline takes advantage of longitudinal studies of vertex behavior over time to inform the generation of derived features. A low rank approximation to the matrix of features produces clusters of vertices that rise and fall in influence together. The clusters of vertices correlate with sharp boundaries in time which form phases of the network's evolution. Our approach loosely couples parallel graph algorithms with machine learning algorithms, which allows the application of both systems effectively.. Future work includes tensor analysis on these temporal features. By restricting to methods using matrices instead of tensors, we loose the coupled relationship between two features evaluated on the same vertex. Future work with tensors is expected to recover this relationship. By leveraging a diverse collection of matrix factorizations, one can extract distinct insights into the vertex and graph behavior from the same vertex features. Determination of the most useful features is still an open subject. The ANLS-BPP algorithm uses nonnegative least squares with multiple right hand sides where the grouping of right hand sides is currently chosen for simplicity of implementation. In a more refined future implementation, an optimal grouping of right hand sides could be used.

B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


dbVOR:

a

database

system

for

importing

pedigree,

phenotype

and

genotype

data

and

exporting

selected

subsets

Background:

When

studying

the

genetics

of

a

human

trait,

we

typically

have

to

manage

both

genome-wide

and

targeted

genotype

data.

There

can

be

overlap

of

both

people

and

markers

from

different

genotyping

experiments;

the

overlap

can

introduce

several

kinds

of

problems.

Most

times

the

overlapping

genotypes

are

the

same,

but

sometimes

they

are

different.

Occasionally,

the

lab

will

return

genotypes

using

a

different

allele

labeling

scheme

(for

example

vs

A/C).

Sometimes,

the

genotype

for

a

person/marker

index

is

unreliable

or

missing.

Further,

over

time

some

markers

are

merged

and

bad

samples

are

re-run

under

a

different

sample

name.

We

need

a

consistent

picture

of

the

subset

of

data

we

have

chosen

to

work

with

even

though

there

might

possibly

be

conflicting

measurements

from

multiple

data

sources.

Results:

We

have

developed

the

dbVOR

database,

which

is

designed

to

hold

data

efficiently

for

both

genome-wide

and

targeted

experiments.

The

data

are

indexed

for

fast

retrieval

by

person

and

marker.

In

addition,

we

store

pedigree

and

phenotype

data

for

our

subjects.

The

dbVOR

database

allows

us

to

select

subsets

of

the

data

by

several

different

criteria

and

to

merge

their

results

into

a

coherent

and

consistent

whole.

Data

may

be

filtered

by:

family,

person,

trait

value,

markers,

chromosomes,

and

chromosome

ranges.

The

results

can

be

presented

in

columnar,

or

PLINK

format.

Conclusions:

dbVOR

serves

our

needs

well.

It

is

freely

available

from

Documentation

for

dbVOR

can

be

found

at

might

be

carried

out

on

a

subset

of

the

samples.

Managing

and

reconciling

these

multiple

sources

of

data,

often

with

different

sample

IDs,

allele

labels,

and

marker

names,

is

best

managed

using

a

database

system.While

several

database

systems

have

been

developed

for

managing

genetic

when

we

tried

some

of

these,

we

found

that

some

relied

on

commercial

database

systems

that

were

so

complicated

that

they

required

a

database

administrator

to

routinely

maintain

and

apply

regular

security

updates.

Others

did

not

scale

well

as

the

numbers

of

markers

genotyped

per

experiment

rapidly

increased.

Some

were

written

with

sophisticated

webbased

interfaces,

that,

while

elegant,

were

difficult

to

extend

and

customize

as

needed.

Therefore,

we

set

out

to

build

our

own

database

system,

based

on

open-source

We

have

created

a

series

of

programs

and

a

database

called,

dbVOR,

to

store,

manage,

and

retrieve

genetic

data.

Figure

illustrates

the

overall

operations

of

dbVOR.

The

raw

data

flows

into

the

database

tables

using

one

of

several

programs.

Finally,

the

genout

program

can

be

used

to

filter

and

merge

data

to

be

output

in

one

of

several

file

formats..

The

rest

of

this

document

is

organized

as

follows:

in

the

Implementation

section,

we

first

describe

dbVOR's

ancillary

database

tables

used

to

store

pedigree,

trait,

and

marker

data.

Then

we

show

the

tables

used

for

storing

genotype

data:

one

for

targeted

experiments

and

the

other

set

for

genome-wide

experiments.

The

key

difference

between

these

is

that

for

genome-wide

data

we

store

multiple

genotypes

per

record,

in

blocks.

We

next

discuss

the

programs

that

are

used

to

enter

data

into

the

aforementioned

tables.

A

subsequent

program

is

presented

that

extracts

data

from

the

database

tables.

The

next

few

sections

are

intended

to

give

a

feel

for

details

of

the

implementation:

configuration

files,

log

files,

note

storage,

and

database

commit

policy.

The

Example

section

illustrates

the

format

and

use

of

the

targeted

and

genome-wide

genotype

tables.

It

also

illustrates

how

multiple

conflicting

genotypes

for

the

same

person

and

marker

coming

from

multiple

or

even

the

same

experiment

get

resolved.

The

Data

statistics

section

show

how

conflicting

genotype

information

is

summarized

for

user

review.

The

Results

section

compares

the

efficiency

of

the

genome-wide

multiple

genotypes

per

record

representation

to

that

of

the

simpler

single

genotype

per

record

representation.

The

performance

is

also

illustrated

in

the

Discussion

section

via

comparisons

with

existing

systems.

Finally,

the

Appendix

shows

the

full

database

schema..

We

maintain

the

following

typographic

conventions:

All

programs

and

subcommands

are

shown

in

a

bold

typeface.

The

names

of

database

tables

are

shown

in

bold

italic.

Our

dbVOR

database

provides

useful

solutions

to

data

management

problems

commonly

encountered

during

an

ongoing

study

of

the

genetics

of

a

complex

trait.

In

such

a

study,

oftentimes

genome-wide

data

are

first

generated,

and

then

later

regions

of

interest

are

followed

up

via

targeted

custom

genotyping

using

a

different

technology.

Differing

technologies

may

return

different

allele

labels

for

a

given

marker.

Genotyping

experiments

may

partially

fail,

and

so

need

to

be

redone.

Clinical

data

may

be

presented

with

identifiers

that

differ

from

the

identifiers

used

in

the

laboratory.

Accordingly,

dbVOR

was

designed

to

handle

data

from

multiple

sources

with

possibly

conflicting

measurements

as

well

as

conflicting

codings

when

the

underlying

data

are

actually

the

same.

It

was

designed

to

process

not

only

targeted

experiments

but

also

genome-wide

experiments

and

to

merge

their

results

into

a

coherent

whole.

dbVOR

allows

you

to

select

subsets

of

the

data

by

several

different

criteria

and

to

output

the

results

in

tabular,

or

PLINK

format..

We

now

enumerate

some

of

the

strengths

of

our

dbVOR

database:.

dbVOR

was

designed

to

be

easy

to

install,

maintain,

and

modify.

It

is

purposely

lean,

requiring

just

two

software

components

(Python

and

MySQL),

and

it

is

open

source..

Our

dbVOR

database

handles

both

genome-wide

scale

data,

as

well

as

targeted

follow-up

data

of

limited

scale.

While

the

genome-wide

data

are

stored

in

a

special

way

that

maximizes

efficiency,

our

database

is

capable

of

merging

the

two

types

of

data

together

during

export.

dbVOR

handles

not

only

unrelated

individuals,

but

also

handles

family

data.

It

can

keep

family

structures

intact

after

filtering.

Our

database

is

capable

of

handling

multiple

different

phenotypes,

both

categorical

and

quantitative,

and

provides

summary

statistics.

dbVOR

is

designed

to

handle

the

reconciliation

of

overlapping

genotype

data

from

different

experiments.

In

the

process,

it

provides

feedback

to

the

user

in

the

form

of

agreement

matrices,

clearly

showing

how

many

repeated

genotypes

agreed

with

each

other

and

how

many

did

not.

Our

database

can

resolve

multiple

ID

systems,

alternate

marker

names,

and

differing

marker

allele

labels.

dbVOR

retains

all

original

data

while

providing

the

ability

to

selectively

turn

off

untrusted

portions

of

the

data.

Our

database

supports

flexible

and

powerful

filtering

during

export.

For

example,

specific

pedigrees

or

persons

can

be

excluded

or

included,

and

markers

can

be

selected

by

name,

chromosomal

position,

or

chromosomal

range.

dbVOR

supports

the

storage

of

multiple

builds

of

map

information,

as

well

as

supports

genetic

maps.

Our

database

uses

a

command-line

interface,

augmented

with

flexible

configuration

files.

dbVOR

has

the

ability

to

do

a

test

run

of

requested

actions

before

committing

such

changes

to

the

database.

This

permits

iterative

improvement

to

the

choice

of

configuration

options.

dbVOR

provides

a

system

for

storing

and

retrieving

notes.

The

data

analyst

can

use

these

notes

to

document

data

cleaning

decisions

and

data

manipulation

choices.

dbVOR

supports

Illumina,

Affymetrix,

and

flat

file

input,

largely

by

letting

the

user

choose

columns

from

the

input

files.

It

also

supports

input

in

PLINK

binary

'bed'

format..

As

with

all

software

projects,

dbVOR

is

a

work

in

progress,

and

so

there

are

a

number

of

possible

improvements

that

could

be

made

in

the

future.

These

include:.

dbVOR

could

be

extended

to

use

multiple

cores

in

parallel

when

available.

Such

an

extension

should

enable

the

speed

up

of

some

currently

slow

operations.

A

number

of

database

systems

for

handling

human

genetics

data

have

been

proposed,

starting

as

early

as

when

the

"Human

Genetics

Database

Management

System"

was

developed;

this

database

system

handled

not

only

marker

data

on

pedigrees,

but

also

clinical

and

laboratory

data

Before

embarking

on

writing

our

own

database

software,

we

explored

other

options.

As

a

small

research

group

with

limited

funds,

the

capable

Integrated

Genotyping

System

was

not

an

option

for

us

because

we

were

not

prepared

to

set

up

and

maintain

a

Windows

server

machine.

However,

we

did

have

a

Sun

Solaris

machine,

and

so

explored

using

GeneLink

GeneLink

was

originally

developed

using

Sybase

SQL

server,

but

we

chose

to

get

the

Oracle

version

working,

as

we

had

a

site

license

for

Oracle.

While

we

got

it

working,

we

found

that

keeping

Oracle

secure

and

updated

was

a

major

undertaking,

requiring

the

skills

of

an

experienced

database

system

administrator.

We

failed

at

our

attempts

to

extend

GeneLink

to

handle

non-integer

pedigree

IDs

-changing

the

pedigree

ID

type

in

the

database

tables

was

simple,

but

adjusting

the

sophisticated

web

interface

to

accommodate

this

change

was

beyond

our

skill

set.

These

experiences

led

us

to

focus

on

creating

a

database

system,

modeled

on

GeneLink,

that

used

open

source

free

components,

was

easy

to

install

and

maintain,

and

has

a

simple

interface,

enhancing

the

ability

to

subsequently

modify

it

as

needed..

In

passing,

we

note

that

PLINK

is

a

formidable

data

analysis

tool

and

it

provides

at

least

as

much

filtering

capabilities

as

dbVOR.

PLINK

can

do

a

variety

of

tasks

-it

can

recode

allele

labels,

flip

strands,

update

individual

information,

zero

out

a

set

of

genotypes,

and

merge

sets

of

genotypes.

However,

PLINK

only

supports

flat

files;

it

has

no

database

capabilities.

Further,

it

cannot

easily

manage

collections

of

experiments

to

process.

Merging

data

sets

with

PLINK

requires

that

both

data

sets

use

the

same

sets

of

person

IDs

and

allele

labels,

while

dbVOR

allows

one

to

resolve

these

issues

via

the

use

of

appropriate

aliases.

And,

having

been

written

primarily

for

data

sets

of

unrelated

individuals,

when

working

with

family

data

it

can

be

challenging

with

PLINK

to

avoid

filtering

out

ungenotyped

individuals

required

to

maintain

family

structures.

While

our

final

merged

and

cleaned

data

sets

are

often

analyzed

with

PLINK,

in

our

experience

dbVOR

greatly

facilitates

the

process

of

preparing

a

single

PLINK-ready

data

set..

We

have

compared

dbVOR

to

SNPpy,

a

recently

developed

database

system

Both

SNPpy

and

dbVOR

are

database

systems

that

store

and

retrieve

genotype

information

from

experiments.

Both

systems

access

their

database

in

Python

and

by

a

command

line

interface.

They

both

deal

with

the

large

amount

of

marker

data

in

current

genome-wise

studies,

but

in

different

ways.

SNPpy

more

heavily

uses

database

features

and

uses

C++

code

embedded

in

Python

to

improve

database

performance.

In

addition,

SNPpy

can

take

advantage

of

multiple

cores,

when

available.

However,

SNPpy

supports

only

one

phenotype

and

no

parental

information.

dbVOR

is

specifically

designed

for

linkage

studies

containing

families,

and

so

stores

parent

data.

It

also

stores

marker

genetic

positions

(as

well

as

base

pair

positions);

in

addition,

it

stores

map

information

for

multiple

builds.

dbVOR

stores

all

the

data

as

it

was

originally

sourced

from

flat

files,

Illumina

files,

or

Affymetrix

files.

But

it

allows

for

later

marker

aliasing,

person

name

aliasing,

and

allele

value

remapping.

It

provides

statistics

to

show

patterns/anomalies

in

the

data

without

judging

them

as

errors..

On

data

extraction,

dbVOR

allows

poor

data

to

be

marked

and

ignored.

It

resolves

multiple

measurements

of

the

same

person/marker

in

an

automated

but

customizable

way.

dbVOR

can

filter

subsets

of

the

database

via

simple

lists

and

ranges

and

does

not

required

the

construction

of

complicated

SQL

expressions

(as

is

necessary

for

SNPpy).

Further,

filtering

can

be

constrained

to

keep

the

family

structures

intact.

dbVOR

reports

data

out

in

the

less

compact

text-based

PLINK

formats

like

SNPpy

does,

as

well

as

in

the

compact

binary

PLINK

format.

It

also

can

output

files

dbVOR's

data

representation

is

particularly

efficient

for

fetching

all

the

markers

from

a

given

chromosome

or

chromosome

range.

Table

compares

the

Illumina

HapMap

loading

times

with

the

corresponding

times

for

SNPpy.

Recall,

these

data

contain

markers

and

and

samples

for

CEU,

YRI,

and

CHB+JPT,

respectively.

We

use

an

iMac

with

a

GHz

Intel

Core

Duo

processor

having

between

bp

to

write

to

ped

file.

The

timings

under

Cold

Database

are

for

performing

a

task

just

after

the

database

server

is

started.

The

Hot

Database

timing

are

for

repeating

the

task

a

second

(or

third)

time.

The

Cold

numbers

are

averaged

over

three

runs

and

the

Hot

numbers

are

averaged

over

six

runs..

GB

memory.

The

dbVOR

configuration

files

used

in

these

comparisons

are

provided

in

Additional

file

the

process

for

developing

the

timing

data

is

described

in

detail

in

Additional

file

The

dbVOR

calculation

is

done

with

the

genotypes

stored

in

the

genome-wide

block

representation.

The

SNPpy

calculation

is

done

with

genotypes

stored

in

the

shard

representation

with

multiprocessing

activity

set

to

one

processor

(j

Regarding

the

average

loading

times

seen

in

the

first

line

of

Table

SNPpy

is

faster

than

dbVOR.

Similarly,

when

selecting

and

writing

out

all

the

genotypes,

SNPpy

is

much

faster

than

dbVOR.

However,

when

smaller

portions

of

the

data

are

selected,

dbVOR

is

faster

than

SNPpy.

The

'cold'

timings

indicate

performance

right

after

the

database

has

been

started,

while

the

'hot'

timings

indicate

the

effect

of

repeating

a

task

that

has

previously

been

carried

out.

Both

systems

show

improvements

in

'hot'

mode

when

reading

a

chromosome

or

less;

presumably

because

data

are

cached

in

the

server

and

so

do

not

have

to

be

fetched

from

the

database

files

One

might

question

why

our

database

system

contains

two

different

subsystems

for

storing

genotypes:

the

Genotypes

table

is

used

for

storing

small

volumes

of

data,

while

the

Genotypesblocks

table

approach

is

used

for

genomewide

large-scale

genotype

data.

One

reason

is

historical:

we

developed

the

Genotypes

table

approach

first

for

targeted

experiments.

The

other

reason

is

that

in

our

experience,

in

a

given

research

project,

it

is

common

to

start

with

genome-wide

large-scale

data

and

then

followup

with

much

smaller

scale

targeted

follow-up

data.

The

genome-wide

representation

is

more

efficient

for

reading

and

writing

large-scale

genotype

data;

however,

it

is

less

space

efficient

for

targeted

data

(as

storing

small-scale

data

in

a

large

block

structure

would

waste

space).

The

Genotypes

table

representation

is

just

the

opposite;

it

is

efficient

where

the

genome-wide

representation

is

not

and

vice

versa.

The

genome-wide

representation

for

reading

and

writing

uses

fewer

database

records.

The

database

must

maintain

indices,

consistency

constraints

and

atomicity

of

operations

for

each

record.

With

fewer

records,

there

is

less

total

time

overhead..

Finally,

it

is

important

to

note

that

efficiency

is

context

dependent:

the

genome-wide

representation

is

inefficient

in

the

case

when

one

is

selecting

a

small

number

of

markers,

each

in

a

separate

block.

Such

a

selection

would,

when

using

the

Genotypeblocks

representation,

require

extraction

and

handling

of

a

lot

of

unneeded

marker

data,

as

each

block

containing

a

selected

marker

would

need

to

be

pulled

in

its

entirety

from

the

database.

Here

we

discuss

some

timing

data

based

on

a

real

data

set,

as

well

as

compare

dbVOR

to

other

previously

published

database

software.

Genetic

studies

of

complex

human

diseases

typically

involve

multiple

iterations

of

genetic

marker

generation.

For

example,

a

set

of

individuals

may

first

be

assayed

with

a

genome-wide

chip

that

measures

genotypes

at

hundreds

of

thousands

of

markers.

Then

a

subset

of

interesting

signals

may

be

followed

up

by

custom

genotyping

using

a

different

technology.

If

there

are

technical

problems,

yet

another

round

of

custom

genotyping

using

a

third

MySQL,

driven

by

command-line

programs

written

in

Python.

We

named

our

database

system

"dbVOR",

after

Vör,

the

inquiring

Norse

goddess

of

wisdom

from

whom

nothing

can

be

concealed.

O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O B O B O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O B O O O O O O O O O O O O O O O O


On stable social laws and qualitative equilibria This paper introduces and investigates the notion of qualitative equilibria, or stable social laws, in tbe context of qualitative decision making. Previous work in qualitative decision theory has used the maximin decision criterion for modelling qualitative decision making. When several decision-makers share a common environment, a corresponding notion of equilibrium can be defined. This notion can be associated with the concept of a stable social law. This paper initiates a basic study of stable social laws; in particular, it discusses the stability benefits one obtains from using social laws rather than simple conventions, the existence of stable social laws under various assumptions, the computation of stable social laws, and the representation of stable social laws in a graph-theoretic framework. 0 1998 Elsevier Science B.V. All rights reserved. General coordination mechanisms are essential tools for efficient reasoning in multiagent AI systems. Coordination mechanisms are a major issue of study in the fields of mathemati,cal economics and game theory as well. Much work in these fields concentrates on the notion of an equilibrium. An equilibrium is a joint behavior of agents, where it is irrational for each agent to deviate from that behavior. The notion of an equilibrium discussed in the game theory and mathematical economics literature refers to agents which are expected utility maximizers. However, much work in AI has been concerned with more qualitative forms of rational decision making. In particular, work in AI has been concerned with agents which attempt to maximize their worst case payoff. Although, at first, this behavior may look questionable from a decision-theoretic perspective, it is known to capture the behavior of risk-averse agents [9, 19, 38] , and it is appropriate in the context of qualitative decision theory [8, 15, 19, 59] . Moreover, in [9] Brafman and Tennenholtz have shown general conditions under which an agent can be viewed as if it were a maximin agent (i.e., an agent which maximizes its worst case payoff). However, the corresponding notion of equilibrium has not yet been investigated. In this paper we introduce this notion and investigate its properties. The concept of qualitative equilibrium turns out to coincide with the notion of a stable social law, to be introduced later in this paper. For ease of exposition we introduce the notion of a stable social law in a self-contained fashion, as an extension to previous work on artificial social systems.. Some work on multi-agent systems assumes that agents are controlled by a single entity which dictates their behavior at each point in time, while some other work is concerned with decentralized systems where no global controller exists. A significant part of the theory developed for decentralized multi-agent systems [7, 17] deals with conflict resolution in multi-agent encounters. The basic theme of work on this subject is that in decentralized systems agents will reach states of conflict and appropriate negotiation mechanisms would be needed in order to resolve these conflicts. The result of the negotiation process is a deal that the agents will follow. Work in AI has been mostly concerned with agents that conform to agreed-upon deals. Agents may not follow irrational negotiation protocols, but will conform to deals obtained by following rational negotiation protocols [20, 37, 62] .2 This differs from work in gametheory [25, 48] where a joint strategy is considered unstable (and therefore unsatisfactory from a design perspective) if an agent has a rational incentive to deviate from it. The Artificial Social Systems approach (e.g., [45, 57] ) exposes a spectrum between a totally centralized approach and a totally decentralized approach to coordination. The basic idea of the Artificial Social Systems approach is to add a mechanism, called a social law, that will minimize the need for both centralized control and on-line resolution of conflicts. In a mobile robots setting, for example, such a social law may consist of various traffic constraints [56] . More generally, a social law is a set of restrictions on the agents' activities which allow them enough freedom on one hand, but at the same time constrain them so that they will not interfere with each other. In particular, a social law makes certain conflicts unreachable, and as a result improves the system efficiency. Notice that mechanisms for conflict resolution can serve as part of the social law; they will be used in situations where conflicts can't be prevented in advance.. The motivation for the theory of artificial social systems has been the design of artificial multi-agent systems, and as such it assumes that the agents will obey the law supplied by the designer. However, if each agent is designed by a different designer then some laws might be considered irrational. Therefore, at the current stage, the artificial social systems approach and approaches to conflict resolution are somewhat complementary; the resolution of conflicts in multi-agent encounters is part of a more general theory of social laws, but the theory of artificial social systems has neglected the stability of social laws in * See [53] for a detailed discussion of this point. multi-agent encounters. In this paper we wish to bridge part of the gap between the theory of artificial social systems and the theory of conflict resolution in multi-agent encounters, by considering stable social laws for multi-agent encounters. A social law for a multiagent encounter is a restriction of the set of available actions (in the encounter) to a set of socially allowed actions. Stable social laws make deviation from them irrational. Notice that a convention is a particular type of a social law; a convention determines a particular joint action for the agents to follow (e.g., keep the right of the road), while a social law allows several such actions and prohibits others. As it turns out, this distinction is quite important and useful.. We will discuss social laws for multi-agent encounters using a game-theoretic framework which is tailored for assumptions made in the AI literature, and especially in recent work on qualitative decision making. In particular, in most of this paper we will assume that the agents are risk-averse agents, which use the maximin decision criterion.. More specifically, given a set of possible behaviors of the other agents, the aim of an agent is to optimize its worst case outcome assuming the other agents may follow any of these behaviors. This kind of behavior is appropriate where there is some ordinal relation on possible outcomes. In such situations all that matters to agents is the order of payoffs and not their exact value. The precise conditions under which such modelling is an appropriate: one are discussed in [9] . Moreover, as we discuss in Section 4, this modeling perspective is quite natural in many multi-agent systems, where a payoff corresponds to the achievement of a particular user's specification. We will require that a social law suggested for a particular encounter will guarantee to the agents a certain payoff, and that it will be stable; there should be no incentive to deviate from it assuming the agents are riskaverse agents. Hence, a stable social law corresponds to a notion of qualitative equilibrium for risk-averse agents. In a later stage, we show how our discussion and results can be extended to other basic qualitative decision-making settings. We start by introducing our framework. In particular, in Section 3 we define the notion of stable social laws. In Section 4 we discuss the intuition and formal adequacy of maximin in multi-agent systems. Having the basic framework, in Section 5 we show that the set of multi-agent encounters for which a stable convention exists is a strict subset of the set of multi-agent encounters for which there is an appropriate stable social law; however, we show that there exists situations where no stable social law exists. Then, in Section 6 we initiate a computational study of stable social laws; we formulate the corresponding computational problem and show that the general problem of coming up with a stable social law :IS intractable; the proof of our result sheds light on the structure of stable social laws; in addition, we point to an interesting restriction on our framework under which the synthesis of stable social laws is polynomial. We then return back to the question of the existence of stable social laws; in Section 7 we first show how this question can be formulated in standard graph-theoretic terms, and then expose a class of encounters where simple graph-theoretic conditions imply the existence of stable social laws. In Section 8 we discuss how our ideas can be applied in other qualitative decision making contexts, and in Section '9 we further discuss the meaning of our study and results and the connection of our work to the existing literature. In this work we have introduced a theory of stable social laws, or qualitative equilibria, for qualitative decision makers. Our work bridges some of the gap between work on Artificial Social Systems and work on conflict resolution in game theory and AI. Social laws have been shown to be a basic and useful tool for the coordination of multi-agent systems [4, 5, 12, 42, 45, 47, 57] .. However, the stability of social laws in a system of rational agents has been neglected so far. This work extends previous work on social laws for artificial agent societies by considering stable social laws for multi-agent encounters.. Two major lines of research related to our work are work in the field of game theory and work in the field of Distributed Artificial Intelligence [DAI] . Related work on rational deals and negotiations in DA1 (e.g., [37, 51, 62] ) can be viewed as contributions to game theory. A very interesting property of this work is that it considers deals among rational agents who will not deviate from agreed-upon deals. to In difference to this assumption, our work is concerned with agents who will deviate from agreed-upon deals if they have a rational incentive to do so. The safety level kind of reasoning/analysis is not new of course to computer science. This is also true for its (somewhat implicit) use in multi-agent systems. In fact, work in software engineering has used a social laws paradigm for the imposition of protocols in distributed systems [42, 43] . The idea in this work is that a desired behavior of an agent should be guaranteed regardless of the behavior of other agents (obeying the corresponding law). Naturally, the different specifications which may be satisfied (i.e., the lo This is not to say that other assumptions are not treated by the DA1 literature; see, for example [53] . sets of goals/tasks which may be obtained) by each agent might be of different quality, which can be captured by corresponding payoffs. In this case the performance guarantee, that is a crucial factor in computerized systems, can be captured by maximin-like analysis.. Much work in game theory has been concerned with devising rational conventions for a group of rational agents; a rational agent may deviate from a prescribed joint strategy if this deviation will improve its own situation. More specifically, much work in game theory [25, 38, 48] has been devoted to the study of equilibrium in games; an equilibrium will have the property that there is no rational incentive for an agent to deviate from the equilibriurn as long as other agents stick to it. The notion of an equilibrium has been adopted to the AI and DA1 literature in various settings (e.g., [61] ), as part of a general and important attempt to introduce social and organizational metaphors into the AI context [14,16,18,:21,24,28,35,36,39,58] .. A central notion in this regard is the notion of a rational agent adopted from the decision/game theory literature. Most work in game theory has associated the notion of a rational agent with the notion of expected utility maximization. This is not however the usual way a rational agent is viewed in the AI literature, such as in work on conditional planning [22, 29, 50, 52, 60] . Moreover, much of the field of knowledge representalion and reasoning is concerned with qualitative notions of decision making, which are different from expected utility maximization; this is true for work in belief revision [26] , nonmonotonic reasoning [30] , qualitative physics [23] , as well as for work on qualitative decision theory [8, 11, 15, 19, 38, 59] . Work on safety level reasoning/analysis gets increasing attention in mainstream game theory in the recent years. Several works, including [3, 6, 32, 40] , and more recently [1, 34] , are concerned with long-run solution concepts, in which agents attempt to get closer to a safety level of a game. In the related work an agent is not able to observe the utility function of the other agents (although he may be familiar with the related strategies). Although one way to address the related issue is by using the theory of types developed by Harsanyi [33] , the use of safety level analysis in such settings is a natural approach. Notice that our work may fit nicely also with the assumption that each agent knows only its utility function.. One may wish to explore the connections between the notion of stable social laws and the notion of mixed strategies (and equilibria in mixed strategies) discussed in game theory. However, ,a direct comparison between these concepts seems quite problematic. This is due to the fact that the interpretations of the utility function for maximin agents and for expected utility maximizers are different (see the discussion is Section 4, as well as [lo] ). Notice thar in our context the actual selection among the strategies available to an agent is modelled als a nondeterministic choice rather than a probabilistic one. The nondeterminism refers to parameters which might be unknown to the designer and to the other agents and which may change from time to time. One may however wish to find some functional connections between mixed strategies and stable social laws, e.g., a stable social law may be the support of a mixed strategy equilibrium. Unfortunately, we were not able to prove or disprove such a claim. On the other hand, the idea of considering sets of strategies rather than a mixed strategy in equilibrium is consistent with the theory of social situations [31] . The restriction to a set of strategies (i.e., prohibiting some of the strategies) rather than pointing to a probabilistic mixture is a basic idea in the work of Minsky on social rules in the context of software engineering [42, 43] . Needless to say that the enforcement of a social law is much more practical than the enforcement of a mixed strategy, due to the fact that coin flippings can not be usually observed by an outside observer.. Using a game-theoretic terminology, in this work we developed an equilibrium theory for qualitative decision makers, and in particular for risk-averse agents [9, 19, 38] . Our theory and results can therefore be interpreted both as an extension to the theory of Social Laws presented in the AI literature, as well as a contribution to the foundations of discrete/qualitative decision/game theory. We hope it can lead to further cross-fertilization between these fields.

O O B I O O O O O O O O O O O O O O O B I O O O O B I I O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Information infrastructure for inter-organizational mental health services: An actor network theory analysis of psychiatric rehabilitation In the supply of mental health services to communities, data and information are managed not only by clinical organizations, but also by welfare state agencies and charities. The aim of this study is to use methods of analysis from actor network theory to identify organizational interventions necessary for the development of an information infrastructure for inter-organizational mental health services. Data was collected in a project aimed at developing an information system that supports inter-organizational psychiatric rehabilitation in a Swedish municipality. Three organizational interventions were identified: an integrated service policy defined by the national government, a common legal framework allowing sharing of high-level client data, and commissioned support for local inter-agency workspaces. It is concluded that organizational interventions must be regarded when configuring an information infrastructure for mental health services. Organizational interventions should also routinely be addressed in systems design methods to be used in inter-organizational settings. In most countries, health-related services are provided not only by clinical organizations, but also by different welfare state agencies. Together, these institutions share between them the responsibility of supporting community members in their total life situation. From a welfare state perspective, an additional intention is to regard an individual's health problems in light of, for instance, social and labor market issues and vice versa [1, 2] . To be able to make decisions with a focus on particular individuals and their total situation, the practitioners working in health care organizations and welfare state agencies must be able to co-operate. For this collaboration, information sharing and management are essential issues. The development of information systems that support the provision of inter-organizational health services thus requires that designers study and understand the social environment and the work practices in these settings. Paradoxically, however, research has shown that social aspects are currently seldom included when designing and implementing new technology to support co-operation between people within the same organization, or between people in different organizations [3] . The aim of this study is, therefore, to use actor network theory for the identification of organizational interventions needed for the development of a unified information infrastructure for inter-organizational mental health services. These interventions can then be addressed in system development planning. Psychiatric rehabilitation is used in the study as an example application domain. High-level social and organizational aspects must be considered in an information infrastructure for inter-organizational health services. In this study, we have used actor network theory to identify three organizational interventions necessary to consider when developing such an infrastructure for psychiatric rehabilitation. These interventions should be highlighted in system design processes involving several healthcare and welfare state agencies in the tax-financed setting. In order to establish the general validity of the findings, future studies must address the research question in health service financing contexts other than the Scandinavian setting. The aim of this study was to identify organizational interventions needed for the development of a unified information infrastructure for inter-organizational mental health services. Three interventions were identified: an integrated service policy defined by the national government, a common legal framework allowing sharing of high-level client data, and commissioned support for local inter-agency workspaces. From these results, two implications for the planning of a unified information infrastructure can be derived.. First, inconsistencies in mental health policies and strategies between agencies and government departments must be corrected. The structural gaps between the healthcare and welfare state agencies providing health-related services were at least partially caused by contradictory directives from the different national government departments involved. These structural gaps were predisposed for conflicts between the agencies in their view of single clients. Individuals were at constant risk of falling through the protective web provided by the social welfare system because no agency had an interest in collecting and managing data on the individual's situation as a whole. Even a technically unified information infrastructure will fail to provide added value to the services supplied to the community if the strategies and policies of government departments, welfare state agencies, and healthcare providers are disjointed and the services not aligned. Second, the information infrastructure must be developed in parallel to modifications of the formal and informal social norms surrounding the inter-organizational practices. Although the welfare state agencies received directives from different government departments, the inter-organizational team members were dependent on each other's organizational resources when adopting a rehabilitation process that would best suit a specific client. However, the practitioners were hindered in their efforts by the prevalent formal and informal social norms. The current legislation, in our case the Swedish Personal Integrity Act, did not allow free sharing of client data between welfare state agencies, and the organizational culture at the social welfare agencies was not supportive of inter-organizational cooperation. Consequently, the results of this study point out that development of a health information infrastructure requires integration across two boundaries, i.e. between different service policies and between service policies and the information infrastructure. This observation has relevance for high-level policy-making regarding health services. For instance, in the US, it has recently been concluded that competition in the medical marketplace, coupled with fiscal pressures affecting providers and health systems, makes it necessary that leadership for regional and national coordination of an information infrastructure for health services needs to come from governments [29] . Accordingly, a position as the National Coordinator for Health Information Technology has been created at the Department of Health and Human Services. The results of the present study have relevance for the type of policymaking and organizational context that can be expected to develop in other countries and regions in the near future.. As regards service policy integration, only voluntary agreements allowed the healthcare and welfare state agencies included in the case study setting to provide integrated services. The present data suggest that the lack of structured and contracted relationships formalizing the agreements between the local agencies was an impediment to teamwork in general and to information management in particular. Previous studies have shown that when local administrators have formalized a mandated relationship into a contract, the perceived effectiveness of that relationship becomes greater [30] . Nonetheless, policy-makers at the national government level still often choose to support improvised relationships between welfare state agencies. One exception is Northern Ireland, where one of the most structurally integrated models of health and personal social services in Europe has been implemented. Social and health services are jointly administered and this arrangement intends to promote collaborative working and interdisciplinary arrangements. A recent comparative study suggests that health services in Northern Ireland, in practice, have distinct advantages over their counterparts in comparable areas [31] . The results indicate that the benefits are associated more with integrated management systems and less with integrated practice-related activities. In the US, promising advances have been reported from programs based on intergovernmental partnerships in which administrators at different levels of government prod each other to 'try and do more' [32] . Thus, even in this market-oriented environment, an intergovernmental partnership has been found to be the best model for innovative health policy leadership. Nevertheless, further research is required into the effectiveness of high-level integration of health services in general, and the relation between integration and information systems development in particular.. The task of protecting client integrity while developing electronic data interchange between healthcare and welfare state agencies has proved to be more complex than expected. Recent studies have reported significant problems, e.g. application restrictions and incompatibilities, technical malfunctions, changing standards, and insufficient dedicated resources [33] . The key challenge, however, is to coordinate enterprise security aspects with client integrity in a legal framework. Health consumer advocates have long argued that lack of trust in caregiver confidentiality frightens individuals with stigmatizing health disorders and thus deters them from seeking help, and there is today broad consensus among both policy-makers and practitioners on that confidentiality is a fundamental right of clients in the health service setting [34] . Nevertheless, while breaching confidential health information is an issue that has received considerable theoretical attention more research is required from practice settings.. Whether providing clients with online access to their own records or enabling members of inter-organizational teams to share high-level case management data, there are several benefits to online electronic systems that can ultimately lead to improved service outcomes. Therefore, as welfare state agencies move closer to a paperless environment and Internet-based applications, they must also be able to understand the increased risks to privacy and security that are incurred by using electronic systems [35] . Thereafter regulations must be developed that properly address system application, authentication, authorization and service provision practices, and most importantly, that the individual client's life situation is not compromised.. When conducting research in organizational contexts, one must make decisions on the primary unit of analysis [36] . Most studies of health information systems have focused on the micro politics of technology design and implementation [37, 38] . Even in this study, we chose activities within inter-organizational teams as the basic unit. This delimitation made it possible to categorize case study findings as external or internal to these teams. However, micro-studies of design practice, while invaluable in addressing interactions between technologies, users and work practices, often fail to account for the historic reach of decision-making at higher societal levels. Nevertheless, it is often these historic circumstances that contribute to present-day interactions between user, information system and organization that find expression-often indirectlyin daily work practices [39] . Therefore, a second order of data analysis was performed using actor network theory, whereby the higher-level social structures and informatics issues could be incorporated into a common model.. Finally, the forces we describe pay little attention to the viewpoints of the individual citizen. If some of these organizational pressures were put to the test of the patient and family perspective, radical change might result, whether in a welfare state or one governed by market forces [40] . With the growing international influence of consumer movements, yet another actor may soon take centre stage. Moreover, of course, sharing high-level information among organizations is from the perspective of a unified health information infrastructure only the beginning. Organizations and clinicians taking care of real patients and clients will need far more than high-level summaries. Optimal care will require timely and accurate laboratory results, medication lists, and other information collected by service providers, as well as the current information about the client. The long-term goal must therefore still be integrated services and electronic records that make it feasible to allow authorized users from authorized agencies to both access information and update it. One can envision providers ordering clinical treatments and social services on behalf of a client, clients registering their experiences with the services, and providers revising them, all done virtually, online. A case study design [23] was used to perform an actor network analysis of the information infrastructure underpinning the psychiatric rehabilitation services in a Swedish community. Case study methods were used for collection and preliminary structuring of data, while the theory of global and local actor networks [16] was introduced for the final analyses. The primary data were collected by video recording real-life information system design meetings in an inter-organizational setting. The final analysis of the data was performed in two phases. In the first phase, researchers with backgrounds in medical informatics and social sciences analyzed the data from the case study in order to construct a representation of the current actor network in psychiatric rehabilitation. Thereafter, additional clinical researchers joined the analysis to create an aligned actor network upon which to base a unified information infrastructure.

B I O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I I O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I


A Storm of Feasibility Pumps for Nonconvex MINLP One of the foremost difficulties in solving Mixed Integer Nonlinear Programs, either with exact or heuristic methods, is to find a feasible point. We address this issue with a new feasibility pump algorithm tailored for nonconvex Mixed Integer Nonlinear Programs. Feasibility pumps are successive projection algorithms that iterate between solving a continuous relaxation and a mixed-integer relaxation of the original problems; such approaches currently exist in the literature for Mixed-Integer Linear Programs and convex Mixed-Integer Nonlinear Programs. Both cases exhibit the distinctive property that the continuous relaxation can be solved in polynomial time. In nonconvex Mixed Integer Nonlinear Programming such a property does not hold and the main innovations in this paper are tailored algorithmic methods to overcome such a difficulty. We present extensive computational results on the MINLPLib, showing the effectiveness and efficiency of our algorithm. Mixed Integer Nonlinear Programming (MINLP) problems are Mathematical Programs (MP) of the following form: min f (x, y) g(x, y) ≤ 0. where x are integer decision variables, y are continuous decision variables, X, Y are two polyhedra (which possibly include variable bounds), f : R p+q → R and g : R p+q → R m . We remark that f can be assumed convex without loss of generality (if it were not, we might replace it by an added variable v and adjoin the constraint f (x) − v ≤ 0 as a further component of g).. The exact solution of nonconvex MINLP is only possible for certain classes of functions f, g (e.g. if f is linear and g involve bilinear terms xy [2, 10] ). In general, the spatial Branch-and-Bound (sBB) algorithm is used to obtain ε-approximate solutions for a given positive constant ε. The sBB computes upper and lower bounds to the objective function value within sets belonging to an iteratively refined partition of the feasible region. The search is pruned when the lower bound on the current set is worse The rest of this paper is organized as follows. In Section 2 we frame the FP algorithm within the class of Successive Projection Methods, describing their convergence properties. In Section 3 we discuss the use of different norms within the two subproblems of the FP algorithm. In Section 4 we list our solution strategies for both subproblems. In Section 5 we present comparative computational results illustrating the efficiency of the proposed approach. Section 6 concludes the paper. We have presented the theoretical foundation of an abstract feasibility pump scheme interpreted as a Successive Projection Method in which, roughly speaking, the set of constraints of the original problem is split in two sets and the overall algorithm aims at deciding if the feasibility space given by the intersection of such two sets is empty. Such a scheme has been specialized for dealing with nonconvex Mixed Integer Nonlinear Programming problems, the hardest class of (deterministic) optimization problems.. Because evil is in the details, we analyzed a large number of options for (i) formulating and solving the two distinct problems originated by the above split and (ii) guaranteeing convergence of the global algorithm. The result has been more than twenty feasibility pump variants which have been computationally tested on a large number of MINLP instances from the literature to assert the viability of FP both as a stand-alone approximation algorithm and as a primal heuristic within a global optimization solver. Four especially interesting of these variants have been discussed in detail and extensive results have been presented on a set of 65 "hard" instances. The results show that feasibility pumps are indeed successful in finding feasible solutions for nonconvex MINLPs. The four surviving FP variants have been extensively tested on the full set of 243 MINLP instances and, in particular, we discuss the results on the 65 "hard" instances introduced in Section 5.2. More precisely, the four variants have the characteristics reported in Table 1 . Table 2 reports the aggregated results on the 65 "hard" instances. In particular, we consider for each FP variant the number of times the algorithm terminated with a feasible solution within the 2-hours of user CPU time limit (successes), the number of times the algorithm was the only one to find a feasible solution (successes alone), the number of times the time limit was reached without a feasible solution (time limit reached), the number of times the algorithm encountered numerical issues (fails), the number The detailed results are reported in Table 3 where, for each variant, we give the solution value (value), the computing time (time) and the number of iterations (it.s) which are roughly equal to the number of problems (8) and (9) solved. In case of numerical issues for a pair instance / FP variant, we report in all entries for such an instance some "++", whereas in case of time limit reached the entry value is set to "-" (while we correctly report the computing time of 7,200 CPU seconds and the number of iterations within such a time).. The results of Tables 2 and 3 show that FP-1 is the most successful FP variant and is remarkably able to find a feasible solution in limited CPU time on 75% of the "hard" instances in the testbed. A direct comparison with the closest variant, namely FP-2, shows that the use of the norm constraint is useful: although FP-1 does not dominate FP-2, it is overall superior on all entries and there are many instances in which FP-2 converges slowly whereas FP-1 reaches feasibility in a very small number of iterations. Variant FP-3 is very fast but seems to be a bit "unsophisticated" for those instances which look more difficult (in the "hard" testbed). However, it might be a viable option for a "cheap" FP variant executed extensively within a GO solver. Finally, variant FP-4 does not look-at the moment-very competitive, although it is not fully dominated because it finds the smallest solution four times, in one case (deb8) a much smaller one, with respect to the other variants. One relevant issue for FP-4 seems that the MIQP solved as problem (9) is time consuming thus allowing only a limited number of FP iterations. Things might change in the future, depending on the solver or its settings.

O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A Hybrid Speech Emotion Recognition System Based on Spectral and Prosodic Features In this paper, we present a hybrid speech emotion recognition system exploiting both spectral and prosodic features in speech. For capturing the emotional information in the spectral domain, we propose a new spectral feature extraction method by applying a novel non-uniform subband processing, instead of the mel-frequency subbands used in MelFrequency Cepstral Coefficients (MFCC). For prosodic features, a set of features that are closely correlated with speech emotional states are selected. In the proposed hybrid emotion recognition system, due to the inherently different characteristics of these two kinds of features (e.g., data size), the newly extracted spectral features are modeled by Gaussian Mixture Model (GMM) and the selected prosodic features are modeled by Support Vector Machine (SVM). The final result of the proposed emotion recognition system is obtained by combining the results from these two subsystems. Experimental results show that (1) the proposed non-uniform spectral features are more effective than the traditional MFCC features for emotion recognition; (2) the proposed hybrid emotion recognition system using both spectral and prosodic features yields the relative recognition error reduction rate of 17.0% over the traditional recognition systems using only the spectral features, and 62.3% over those using only the prosodic features. key words: speech emotion recognition, non-uniform subband processing, spectral feature, prosodic feature As one of the most natural and important means in "humanhuman" interaction, speech communication consists of two channels, the explicit channel carrying the linguistic information (i.e., the content of the conversation) and the implicit channel carrying the non-linguistic information (e.g., gender, age, emotion, dialect) [1] , [2] . Both linguistic and nonlinguistic information play crucial roles in human speech communication. Many studies in linguistic information processing (e.g., speech recognition and speech synthesis) have been done in the past several decades; however, the research on non-linguistic cues has only recently become popular [3] .. Non-linguistic cues generally include gender, age, emotion, stress and nervousness, dialect, and so on [1] . Among these properties, emotion plays a key role in many applications, for example, in text-to-speech systems to syn- thesize emotional speech [4] . So far, different approaches have been presented to model emotions: one approach is the definition of discrete basic emotions, for example, anger, disgust, fear, happiness, sadness, and surprise, as proposed by Ekman [5] ; another approach is the utilization of continuous emotional dimensions, for instance, the threedimensional emotional space: arousal (activation), potency, and valence, as proposed by Schlosberg [6] .. In this paper, we present our recent study on automatic emotion recognition from speech, which has received much attention for building more intuitive "human-machine" interfaces [7] . Emotion recognition is basically a statistical pattern classification problem, which consists of two major steps, feature extraction and classification. While the theory of classification is pretty well developed [2] , the extraction of distinctive features is a highly empirical issue [2] . Therefore, our main focus in this research is to extract more effective features and apply them in one emotion recognition system. So far, many speech emotion recognition systems have already been reported [7] . In these existing systems, however, the features exploited for emotion recognition are generally the mel-frequency cepstrum coefficients (MFCC) [7] , [8] , which is often used in automatic speech recognition (ASR) systems. For speech recognition, the features should emphasize the content of speech (i.e., linguistic information). In contrast, the features to be used in emotion recognition systems should be able to highlight the discriminative cues among different emotions (i.e., non-linguistic information), rather than linguistic information of speech. This essential difference means that the MFCC features that are suitable for ASR systems do not satisfy the requirements for emotion recognition from speech. As a result, the systems using only MFCC features achieve very limited emotion recognition results, as reported in [7] - [9] . Furthermore, prosodic features of speech, which were found to be useful cues for representing speech emotion information in phonetics and linguistics [1] , were also used for speech emotion recognition systems [2] , [7] . However, the recognition systems using prosodic features alone demonstrated much poorer recognition performance than those using spectral features [2] . Many algorithms of utilizing both spectral and prosodic features have been proposed. In [10] , prosodic features, MFCC, and formant features were investigated, and only the mean and standard deviation of per-frame MFCC features were extracted for each utterance, then the spectral Copyright c 2010 The Institute of Electronics, Information and Communication Engineers and prosodic features were combined directly at the featurelevel, which might cause a loss of information. Many other systems use the per-frame MFCC features, and then integrate them with prosodic features, such as the phonemelevel modeling in [11] . In [11] , per-frame based MFCC features was suggested to be complementary to the suprasegmental prosodic features, however, the improvement is not obvious after combining the spectral features with prosodic features. Besides, the prosodic features used in this study mainly come from F0 and the performance could be further improved including much wider range of prosodic features [11] .. To address the problems of traditional emotion recognition systems, In this paper, we propose a new spectral feature extraction approach using a novel non-uniform subband processing technique that is designed based on the speech emotion production mechanism. Moreover, to make use of the complementary information prosodic features may provide, a much wider range of prosodic features closely related to emotion states in speech are selected from the previous studies [1] , [2] compared with [11] . Both the newly extracted spectral features and the selected prosodic features are exploited in the proposed emotion recognition system. Because of the differences in size of the feature vectors between the non-uniform spectral features and the prosodic features, two different classifiers are applied in our proposed emotion recognition system, namely, the Gaussian Mixture Model (GMM) classifier for the non-uniform spectral features, and the Support Vector Machine (SVM) classifier for the prosodic features. The proposed hybrid emotion recognition system yields the final decision through combining the results from both classifiers. Experimental results show that (1) the proposed non-uniform spectral feature is more effective than the traditional MFCC feature for emotion recognition; (2) the proposed emotion recognition system using both spectral and prosodic features yields the relative recognition error reduction rates of 17.0% over traditional recognition systems using only spectral features and 62.3% over using only the prosodic features.. The remainder of this paper is structured as follows. In Sect. 2, we show the overview of the proposed speech Fig. 1 Block diagram of the proposed hybrid emotion recognition system. emotion recognition system. In Sect. 3, we describe the GMM-based emotion recognition subsystem using the spectral features extracted by a new non-uniform subband processing technique. In Sect. 4, we introduce the SVM-based emotion recognition subsystem using prosodic features. In Sect. 5, the proposed hybrid emotion recognition system is described by combining the GMM-based subsystem and the SVM-based subsystem. In Sect. 6, experiments are performed to evaluate the proposed non-uniform spectral features and the proposed hybrid emotion recognition system. Finally, Sect. 7 draws some conclusions. This paper introduced a hybrid speech emotion recognition system based on both spectral and prosodic features. For spectral features, we employed a new feature extraction method based on a novel non-uniform sub-band processing technique. For prosodic features, a set of prosodic features that are highly correlated with speech emotional states was chosen. The proposed hybrid emotion system combines a GMM-based subsystem that exploits the non-uniform spectral features and a SVM-based subsystem that exploits the prosodic significance. Experimental results show that the proposed hybrid emotion recognition system outperforms the systems using only spectral features or prosodic features.

O O B I I O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I B I I B I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


An Effective Self-Adaptive Admission Control Algorithm for Large Web Caches In this paper, we propose an effective Web cache admission control algorithm. By selectively admitting objects into the cache, the proposed algorithm can significantly reduce the amount of disk I/O on a Web cache while maintaining a high hit ratio. The proposed algorithm adaptively adjusts its own admission control parameter, requiring no usersupplied parameters. Through extensive experiments, we show the effectiveness of the proposed algorithm. Over the past few years, most studies for Web cache management have focused on Web cache replacement algorithms, such as LRU, LFU, LRU-K, and GD-SIZE that determine which objects in the Web cache should be replaced when a new object is brought in [1] . However, most of them have not considered the issue of admission control, which determines whether a new object should be cached or not. It may not always be profitable to insert an object into the cache, because objects with low access frequency degrade the cache performance.. With the recent advent of user-created contents (UCC), Web object size is rapidly increasing [2] . If a Web cache admits every object merely because it is accessed once, the cache may spend a lot of time in writing large objects that would be rarely referenced in the future. This can impose heavy disk I/O overhead on the cache server while reducing cache hit ratio.. We propose an effective admission control algorithm for large Web caches, called Adaptive Frequency-based Admission Control (AFAC). When the Web object not in the cache is requested, admission control determines whether the object should get into the cache or not. In our algorithm, we use two criteria for admission control, i.e., the re-reference likelihood and the size of the object. We show through various experiments that, by selectively admitting objects based on these two criteria, the amount of disk I/O can be significantly reduced while a high hit ratio is maintained.. 2Q [3] is the most representative caching algorithm that utilizes the concept of admission control. When an object is first referenced, 2Q inserts the identifier of the object into an auxiliary FIFO queue called A1, instead of inserting the object into the cache. If the object is re-referenced while its identifier is in A1, it is considered as a hot, i.e., frequently requested, object and inserted into the cache. If an object is not re-referenced while its identifier is in A1, it is not inserted into the cache. Although this admission control is simple, it has been shown to provide a higher hit ratio than performing no admission control at all. However, the admission control in 2Q has two limitations. First, the size of A1 is difficult to tune. If the size of A1 is too large, many cold, i.e., infrequently requested, objects may have high chances to be admitted into the cache. If the size of A1 is too small, even hot objects may not be admitted into the cache, which results in a low utilization of the cache. Second, 2Q does not consider the non-uniformity of object sizes. Admitting one very large object into the cache may cause dozens of objects to be evicted from the cache. It also incurs a large amount of disk I/O. Thus, large objects should be admitted more carefully.. Note that cache admission control and cache replacement are orthogonal concepts. In other words, our proposed cache admission control algorithm will be used in combination with cache replacement algorithms such as LRU, LFU, LRU-K, and GD-SIZE, rather than substituting for those replacement algorithms. Figure 1 shows the proposed admission control architecture. In the figure the FIFO queue F that resides in main memory plays a central role in our proposed admission control method. Let N be the size of F, and F(n) denote a set of last n elements in F (1 ≤ n ≤ N). F(n) is called the selection window of F. Let d i and s i be the identifier and the size Fig. 1 The proposed cache admission control architecture. We have proposed an effective admission control algorithm for large Web caches, called AFAC. By selectively admitting objects into the cache, AFAC can significantly reduce disk I/O while maintaining a high hit ratio. We have shown through extensive experiments that our proposed algorithm can substantially improve the Web cache performance in practice.

O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O