Pairwise

protein

expression

classifier

for

candidate

biomarker

discovery

for

early

detection

of

human

disease

prognosis

Background:

An

approach

to

molecular

classification

based

on

the

comparative

expression

of

protein

pairs

is

presented.

The

method

overcomes

some

of

the

present

limitations

in

using

peptide

intensity

data

for

class

prediction

for

problems

such

as

the

detection

of

a

disease,

disease

prognosis,

or

for

predicting

treatment

response.

Data

analysis

is

particularly

challenging

in

these

situations

due

to

sample

size

(typically

tens)

being

much

smaller

than

the

large

number

of

peptides

(typically

thousands).

Methods

based

upon

high

dimensional

statistical

models,

machine

learning

or

other

complex

classifiers

generate

decisions

which

may

be

very

accurate

but

can

be

complex

and

difficult

to

interpret

in

simple

or

biologically

meaningful

terms.

A

classification

scheme,

called

ProtPair,

is

presented

that

generates

simple

decision

rules

leading

to

accurate

classification

which

is

based

on

measurement

of

very

few

proteins

and

requires

only

relative

expression

values,

providing

specific

targeted

hypotheses

suitable

for

straightforward

validation.Results:

ProtPair

has

been

tested

against

clinical

data

from

patients

following

a

bone

marrow

transplant,

of

which

progress

to

idiopathic

pneumonia

syndrome

(IPS).

The

approach

combines

multiple

peptide

pairs

originating

from

the

same

set

of

proteins,

with

each

unique

peptide

pair

providing

an

independent

measure

of

discriminatory

power.

The

prediction

rate

of

the

ProtPair

for

IPS

study

as

measured

by

leave-one-out

CV

is

which

can

be

very

beneficial

for

clinical

diagnosis

as

it

may

flag

patients

in

need

of

closer

monitoring.

The

"top

ranked"

proteins

provided

by

ProtPair

are

known

to

be

associated

with

the

biological

processes

and

pathways

intimately

associated

with

known

IPS

biology

based

on

mouse

models.

Conclusions:

An

approach

to

biomarker

discovery,

called

ProtPair,

is

presented.

ProtPair

is

based

on

the

differential

expression

of

pairs

of

peptides

and

the

associated

proteins.

Using

mass

spectrometry

data

from

"bottom

up"

proteomics

methods,

functionally

related

proteins/peptide

pairs

exhibiting

co-ordinated

changes

expression

profile

are

discovered,

which

represent

a

signature

for

patients

progressing

to

various

disease

conditions.

The

method

has

been

tested

against

clinical

data

from

patients

progressing

to

idiopthatic

pneumonia

syndrome

(IPS)

following

a

bone

marrow

transplant.

The

data

indicates

that

patients

with

improper

regulation

in

the

concentration

of

specific

acute

phase

response

proteins

at

the

time

of

bone

marrow

transplant

are

highly

likely

to

develop

IPS

within

few

weeks.

The

results

lead

to

a

specific

set

of

protein

pairs

that

can

be

efficiently

verified

by

investigating

the

pairwise

abundance

change

in

independent

cohorts

using

ELISA

or

targeted

mass

spectrometry

techniques.

This

generalized

classifier

can

be

extended

to

other

clinical

problems

in

a

variety

of

contexts.

A

methodology

to

molecular

classification

for

disease

progression

using

pairs

of

peptides/proteins

from

shotgun

mass

spectrometry

based

proteomics

data

has

been

presented.

The

strength

of

this

linear

approach

lies

in

its

design

for

being

able

to

handle

high

dimensional

data

with

small

sample

size

by

(i)

using

minimal

number

of

features

in

order

to

avoid

over

fitting

of

the

data

(ii)

observing

consistency

of

differential

expression

change

across

two

disease

conditions

(iii)

aggregating

peptide

pairs

originating

from

the

same

pair

of

proteins

to

classify

at

the

level

of

protein

pairs

through

joint

statistics.

Since

proteins

act

as

the

true

functional

machines

in

an

organism,

predictions

based

upon

ratios

of

protein

expression

levels

provide

a

natural

link

with

biological

phenomenon.

The

method

has

been

tested

using

clinical

proteomics

data

from

patients

following

a

bone

marrow

transplant,

differentiating

between

patients

progressing

to

IPS

versus

controls

that

remain

unaffected

by

IPS.

The

approach

gains

power

by

combining

multiple

peptide

pairs

originating

from

the

same

set

of

proteins,

with

each

unique

peptide

pair

providing

an

independent

measure

of

discriminatory

power.

The

prediction

rate

of

the

ProtPair

for

IPS

study

as

measured

by

leave-one-out

CV

is

which

may

be

very

beneficial

for

clinical

diagnosis

as

it

may

flag

patients

in

need

of

more

frequent

monitoring.

It

was

encouraging

to

find

that

the

"top

ranked"

proteins

provided

by

ProtPair

are

known

to

be

associated

with

the

biological

processes

and

pathways

intimately

associated

with

known

IPS

biology

based

on

both

mouse

models,

such

as

respiratory

disease,

inflammatory

responses,

and

acute

phase

response.

Proteins

from

top

pairs

were

imported

into

IPA

to

see

if

they

shared

common

biological

networks.

The

network

was

found

to

be

enriched

in

acute

phase

response

proteins

whose

concentration

is

known

to

change

significantly

in

response

to

inflammation,

and

is

associated

with

respiratory

disorders,

hematological

dysfunction,

cardiovascular

complications,

and

infectious

diseases.

The

data

indicates

that

patients

with

improper

regulation

in

the

concentration

of

specific

acute

phase

response

proteins

at

the

time

of

bone

marrow

transplant

are

highly

likely

to

develop

IPS

within

few

weeks.

The

results

lead

to

a

specific

set

of

protein

pairs

that

can

be

efficiently

verified

by

investigating

the

pairwise

abundance

change

in

independent

cohorts

using

ELISA

or

targeted

mass

spectrometry

techniques.

A

ranking/classification

methodology

for

biomarker

discovery

using

pairs

of

proteins

from

shotgun

mass

spectrometry

based

proteomics

data

has

been

introduced.

The

method

leads

to

concrete

hypotheses

about

the

predictive

significance

of

specific

protein

expression

comparisons,

which

can

be

followed

up

for

future

validation.

The

method

has

been

explored

using

clinical

proteomics

data

from

patients

controls,

disease

progressors)

differentiating

between

patients

progressing

to

IPS

versus

controls

that

remain

unaffected

by

IPS

following

the

bone

marrow

transplant

procedure.

Although

sample

size

of

may

appear

small,

it

is

well

accepted

an

appropriate

size

for

the

discovery

and

qualification

phase

of

development

of

biomarker

discovery

The

initial

concept

behind

ProtPair

was

inspired

by

a

rank-based

approach

for

molecular

classification

based

upon

pairwise

mRNA

expression

comparisons.

Geman

et

al

introduced

top-scoring

pair(s)

(TSP)

classifier

for

class

prediction

in

which

the

mRNA

expression

levels

of

genes

are

directly

compared

against

each

other

to

each

other

to

make

classification

The

decision

is

thus

dependent

on

only

the

following

question:

is

the

expression

of

gene

A

higher

than

the

expression

of

gene

B

in

the

sample?

If

so,

the

diagnosis

is

class

If

the

expression

of

gene

B

is

higher

than

for

gene

A,

then

the

diagnosis

is

for

class

Such

decision

rules

implicitly

draw

the

decision

boundary

line

at

y

x.

Although

this

approach

has

been

highly

successful

for

mRNA

expression

experiments,

our

initial

attempts

to

directly

adopt

this

method

were

unsuccessful.

Since

peptides

signals

exhibit

a

wide

dynamic

range,

we

needed

a

more

general

approach

than

y

x

boundary

line

used

in

mRNA

studies.

For

example,

Figure

shows

a

scenario

where

the

two

disease

classes

(represented

by

red

and

blue)

are

clearly

separable,

but

y

x

boundary

line

is

not

an

optimal

separation

line.

Thus,

we

propose

a

more

general

scheme

without

any

implicit

normalization

assumtions.

Figures

and

show

the

scatter

plots

of

peptide

intensities

from

the

top

scoring

proteins

APCS

and

HGFAC.

Note

that

as

in

the

case

of

single

protein/peptide

markers,

of

the

proteins/peptides

is

able

to

provide

good

separation

between

the

two

categories

by

itself.

The

best

way

of

discriminating

among

the

two

cases

is

by

drawing

a

line

(black,

separation

boundary)

that

is

close

to

the

"diagonal"

line

(suggesting

that

both

candidates

contribute

towards

discrimination),

instead

of

being

closer

to

a

horizontal

or

a

vertical

line

(indicating

that

the

separation

could

be

provided

by

a

single

protein).

Hence,

the

combination

of

pairwise

markers

gains

strength

by

combining

the

discrimination

power

achieved

by

both

markers,

which

is

not

possible

by

treating

peptide/protein

markers

in

isolation,

as

is

often

done

in

the

traditional

analysis.

When

all

constituent

peptide

pairs

from

two

proteins

exhibit

consistent

trend

of

significant

differential

expression

change,

all

corresponding

DI

index

values

increase,

leading

to

a

high

median

DI

index

value

across

the

protein

pair..

The

significance

of

the

score

was

assessed

using

permutation

analysis.

Artificial

data

set

was

constructed

by

randomly

rearranging

class

labels,

while

maintaining

the

original

sample

sizes

of

and

for

controls

and

IPS

progressors

respectively.

The

resulting

scores

mirror

those

obtained

by

pure

chance

without

any

meaningful

biological

significance,

while

maintaining

the

overall

statistical

dependency

structure

among

peptides/proteins.

Figure

displays

the

distribution

of

peptide

pair

scores

obtained

from

permutations.

From

this

distribution,

a

pvalue

associated

with

a

given

score

from

unpermuted

"true"

data

can

be

computed

by

taking

the

fraction

of

permuted

data

sets

in

which

a

score

of

at

least

as

large

is

obtained.

This

p-value

is

a

measure

of

probability

of

observing

a

given

score

under

the

hypothesis

that

the

pairs

are

non

informative

for

classification.

Only

among

the

permutations,

with

peptide

pair

scores

in

each

permutation)

scores

came

greater

than

or

equal

to

the

actual

top

score,

yielding

a

p-value

of

−7

Similarly,

Figure

shows

the

distribution

of

top

scores

obtained

using

permutations

from

protein

pairs

by

taking

the

median

of

all

constituting

peptides.

No

score

from

protein

pairs

among

the

permutations

came

near

the

true

top

score

(marked

by

arrow

in

Figure

yielding

a

p-value

of

virtually

The

method

gains

significant

strength

by

combining

multiple

peptide

pairs

originating

from

the

same

set

of

proteins,

since

each

unique

peptide

pair

provides

an

independent

measure

of

discriminatory

power,

which

is

unlikely

to

arrive

by

pure

chance

alone..

The

expected

generalization

error

rate

of

ProtPair

for

the

IPS

data

set

was

performed

using

leave-one-out

cross

validation

(CV).

This

involves

using

both

technical

replicates

from

a

single

patient

as

the

test

data,

and

the

remaining

samples

as

the

training

data,

and

is

repeated

such

that

samples

from

each

patient

is

used

exactly

once

as

the

test

data.

Note

that

in

order

to

obtain

an

unbiased

validation,

leave

one

out

cross

validation

study

refers

to

each

patient

as

opposed

to

each

measurement,

and

all

measurements

from

a

single

patient

were

left

out

during

each

training

step.

In

particular,

both

the

actual

top

scoring

DI

value,

as

well

as

the

set

of

pairs

which

achieve

it,

can

vary

with

the

sample

left

out.

The

estimated

prediction

rate

is

e

N

where

e

is

the

number

of

misclassifications

observed

on

the

test

data

during

the

cross-validation,

and

N

for

the

case

of

IPS

study)

being

the

total

number

of

samples.

For

this

procedure,

there

is

only

a

single

parameter,

threshold

of

peptide

abundance

ratios,

to

select

inside

the

cross

validation

loop.

For

other

procedures

that

do

require

multiple

parameters,

such

as

k-nearest

neighbors,

random

forests

and

support

vector

machines,

the

estimated

prediction

rates

may

be

highly

biased

if

performance

is

sensitive

to

these

parameters

and

they

are

not

properly

cross-validated.

The

prediction

rate

of

the

ProtPair

for

IPS

study

as

measured

by

leave-one-out

CV

is

(corresponding

to

errors

across

samples).

This

is

less

than

the

error

rate

seen

in

Figure

but

is

not

unreasonable.

In

fact,

a

prediction

rate

may

be

highly

beneficial

for

clinical

diagnosis

as

it

may

flag

patients

in

need

of

more

frequent

monitoring.

In

addition,

the

results

obtained

are

consistent

with

recent

studies

by

more

conventional

statistical

methods

This

biomarker

prediction

is

based

on

data

from

the

day

of

SCT,

the

indicated

biomarkers

may

be

altered

further

during

progression

to

IPS.

Note

also

that

the

CV

accuracy

of

a

classifier

is

highly

dependent

upon

the

biological

complexity

of

a

given

disease,

sample

size

and

population

diversity,

and

heterogeneity

of

the

underlying

phenomenon.

In

addition,

the

highest

scoring

pair

may

change

when

the

training

data

is

even

slightly

perturbed

by

adding

or

deleting

a

few

samples

so

that

the

CV

accuracy

is

not

necessarily

reflective

of

global

accuracy

provided

by

the

ultimate

final

pair

(HGFAC

and

APCS

in

this

case).

As

seen

in

Figure

although

there

is

some

amount

of

overlap,

the

final

protein

pair

consistently

shows

strong

evidence

of

discrimination

among

the

two

patient

populations,

illustrating

the

emergent

behavior

of

protein

expression

change

across

the

two

populations.

In

addition

to

being

a

classifier,

ProtPair

has

high

utility

as

a

predictive

and

actionable

tool

to

rank

proteins

in

the

order

of

discrimination

providing

candidates

for

future

validation

testing

specific

hypotheses..

To

understand

if

the

totality

of

the

expressed

protein

list

is

consistent

with

known

pathways

of

IPS,

we

used

Ingenuity

Pathway

Analysis

(IPA)

to

explore

dysregulated

pathways

suggested

by

the

proteins

seen

to

be

significantly

changing.

As

described

in

the

Methods

section,

the

top

proteins

were

imported

into

IPA.

IPA

has

several

algorithms

to

identify

if

the

set

of

proteins

imported

are

associated

with

specific

"canonical"

pathways

or

biological

processes

and

diseases.

Nodes

within

the

network

are

displayed

using

various

shapes

that

represent

the

functional

class

of

the

protein.

All

edges

are

supported

by

at

least

one

reference

from

the

literature,

from

a

textbook,

or

from

canonical

information

stored

in

the

Ingenuity

Knowledge

Base.

In

terms

of

processes

and

diseases

(Figure

the

imported

protein

set

was

most

significantly

associated

with

the

following:

respiratory

disease,

cell

to

cell

signaling

and

interaction,

tissue

development,

cardiovascular

disease,

and

hematological

disease

(all

with

p

values

−6

Inflammatory

diseases

and

inflammatory

response

were

also

highly

significant

(p

values

−5

In

terms

of

canonical

processes,

acute

phase

response

was

top

ranked

(data

not

shown,

p

value

−7

These

results

are

quite

encouraging

and

validate

the

method

as

ProtPair

"top

ranked"

proteins

are

associated

with

the

biological

processes

and

pathways

intimately

associated

with

known

IPS

biology

based

on

both

mouse

models

and

human

studies,

such

as

respiratory

disease,

inflammatory

responses,

and

acute

phase

response

These

pathway

dysregulations

are

consistent

with

recent

studies

In

order

to

more

specifically

examine

the

dysregulated

protein

networks

suggested

by

the

top

proteins,

we

used

the

IPA

database

to

create

a

dense

sub-network

of

targets,

where

the

dysregulated

proteins

(nodes)

are

shaded

and

IPA

inserts

additional

nodes

(colored

white)

and

annotated

interactions

(edges)

in

order

to

connect

as

many

of

the

targets

as

possible

while

restricting

the

total

number

of

nodes

added

(Figure

For

example,

ProtPair

identified

lipopolysaccharide

binding

protein

(LBP),

Fibrinogen,

Hepatocyte

growth

factor

activator

(HGFAC,

a

top

scoring

protein),

fibrinogen

gamma

chain

(FGG),

coagulation

factor

X

and

thrombin

as

changing

in

case

versus

control

samples.

The

changes

seen

for

LBP

have

been

recently

confirmed

by

ELISA

studies

IPA

inserted

extracellular-signal-regulated

kinase

and

nuclear

factor

kappa-light-chain-enhancer

of

activated

B

cells

(NF-KB)

as

they

have

direct

connections

(e.g.

annotations

of

binding

or

regulatory

control)

associated

with

those

proteins.

This

is

consistent

with

the

overall

dysregulatory

themes

of

acute

phase

response

and

cell-cell

signaling

as

NF-KB

is

transcription

factor

controlling

expression

for

many

acute

phase

response

proteins

and

is

regulated

by

TNF-α,

which

is

a

target

for

IPS

therapies

in

clinical

trials

while

proteins

(also

termed

MAP

kinases)

regulate

cell

growth..

In

its

present

form,

ProtPair

makes

predictions

based

entirely

on

the

top

scoring

pairs.

In

the

case

of

IPS

study,

there

is

in

fact

a

unique

top

scoring

pair

-HGFAC

and

APCS,

which

appears

to

have

biological

interest.

However,

there

may

be

many

other

pairs

whose

relative

expression

values

are

informative.

One

possible

direction

of

future

work

is

to

find

a

more

stable,

comparison-based

signature

combining

multiple

high

scoring

pairs.

For

example,

one

may

envision

a

ProtPair

classification

based

on

all

protein

pairs

achieving

the

k

best

scores.

In

this

case,

k

is

a

parameter

whose

optimal

value

can

be

estimated

using

cross-validation..

An

alternative

discovery

based

approach

to

using

ProtPair

would

be

to

use

it

to

detect

variation

in

abundance

patterns

in

comprehensive

peptide

lists,

irrespective

of

their

annotation

status.

For

example,

Rosetta

Elucidator

framework

was

used

to

extract

raw

peak

areas

from

SICs

corresponding

to

all

of

the

isotopic

features

observed

in

regardless

of

the

presence/annotation

of

the

tandem

MS

spectrum

for

the

feature.

Note

that

confident

annotation

MS/MS

is

available

for

only

peptides

of

the

overall

features.

Using

pairwise

comparison

of

these

comprehensive

list

of

global

features

led

to

the

best

discriminating

feature

pair

shown

in

Figure

where

the

axes

mark

the

abundances

of

individual

features.

The

figure

reveals

a

clear

separation

between

the

groups

as

defined

by

the

feature

intensities.

Due

to

the

low

signal

intensity

of

such

features,

it

is

likely

that

they

are

typically

either

not

selected

for

tandem

MS

or

had

extremely

low

signal

in

the

tandem

MS

leading

to

missed/low

confidence

assignment.

However,

the

data

could

be

used

in

an

alternative

targeted

MS

approach,

which

is

specially

useful

to

target

low

abundance

but

important

proteins/peptides

that

may

experience

a

negative

bias

towards

selection

during

the

tandem

MS

sampling.

Such

integration

between

experimental

and

computational

workflows

can

be

very

valuable

for

targeted

MS

in

the

future..

The

results

already

provide

evidence

that

discriminating

comparisons

among

protein

expression

levels

can

be

discovered

even

under

conditions

of

small

sample

size.

Given

the

large

number

of

variables

(peptides/proteins),

a

patient

population

sample

size

of

is

considered

small.

With

number

of

samples

in

the

order

of

hundreds,

more

complex

decision

trees

can

be

learned

from

the

data,

using

only

comparison

questions,

thus

maintaining

easily

interpretable

results

that

do

not

require

any

normalization.

The

corresponding

decision

rules

would

then

be

based

on

more

complex

peptide

abundance

comparisons

involving

more

than

two

proteins.

The

methodology

can

also

be

extended

to

more

complex

and

heterogeneous

data

sets,

for

example

those

combined

from

samples

obtained

from

various

sources

such

as

plasma,

urine,

or

organ

specific

tissue

as

well

as

other

MS

variables

that

reflect

intensity,

such

as

spectral

counts.

With

small

amounts

of

data,

it

may

only

be

possible

to

collect

reliable

estimates

of

pairwise

comparisons

among

expression

levels

so

as

to

avoid

overfitting.

More

data

could

be

used

to

model

the

statistical

dependency

structure

among

families

of

proteins

such

as

metabolic

and

regulatory

pathways

etc.

This

approach

lends

itself

to

a

natural,

hierarchical

family

of

models

which

can

accommodate

various

kinds

and

amounts

of

data.

The

overall

analysis

of

mass

spectrometry

data

using

Mascot

within

Rosetta

Elucidator

framework

identified

peptides

resulting

from

unique

proteins

across

all

patient

samples.

In

order

to

allow

for

confident

identification,

a

false

discovery

rate

of

was

used

as

a

threshold

for

the

identification

of

peptides.

A

total

of

out

of

the

identified

proteins

which

were

found

to

be

identified

by

at

least

or

more

high

scoring

peptides,

were

further

examined

by

ProtPair,

and

the

rest

of

peptides

were

excluded

from

examination.

After

eliminating

single

peptide

hits

for

a

protein,

peptides

were

remaining,

which

were

taken

as

pairs

(leading

to

unique

peptide

pairs)

in

order

to

determine

their

discriminability.

The

resulting

peptide

pairs

were

further

grouped

into

the

corresponding

unique

protein

pairs

(formed

from

proteins,

taken

two

at

a

time),

and

the

discriminabilty

of

the

pairwise

proteins

was

investigated.

Table

shows

the

list

of

protein

pairs

ranked

by

their

DI

score.

If

proteins

in

columns

and

are

denoted

by

and

respectively,

the

ratio

of

intensities

of

is

higher

in

the

case

of

IPS

progressor

patients

than

that

for

controls.

This

implies

that

in

the

case

of

IPS

progressors,

either

(a)

is

upregulated,

or

(b)

is

downregulated,

or

(c)

both

(a)

and

(b)

are

true.

Column

denotes

the

p-value

(probability

that

a

particular

score

would

occur

by

chance)

associated

with

each

DI

score.

Since

multiple

hypotheses

corresponding

to

each

protein

pair

are

being

tested,

false

discovery

rate

(FDR)

was

calculated

(Column

using

empirical

model

from

permutation

tests

in

order

to

correct

for

multiple

hypothesis

testing

For

a

specified

DI

score

threshold,

say,

T,

the

number

S

true

of

observed

scores

T

and

the

number

S

of

scores

T

are

counted.

Assuming

that

the

total

number

of

observed

scores

and

scores

are

equal,

then

the

estimated

FDR

is

simply

S

S

true

Figures

and

show

the

scatter

plot

of

two

unique

peptide

pair

abundances

from

top

scoring

protein

pair

(Table

APCS

(Serum

amyloid

P-component)

and

HGFAC

(Hepatocyte

growth

factor

activator)

indicating

higher

levels

of

HGFAC

in

the

IPS

progressors

(red

dots)

as

opposed

to

controls

(blue

dots),

while

levels

of

APCS

tend

to

be

lower

in

those

cases.

true

distribution

of

patients,

while

the

dotted

lines

are

the

best

fitting

corresponding

Gaussian

distributions.In

both

cases,

the

measurements

independently

suggest

the

overall

trend

that

APCS

is

downregulated

while

HGFAC

is

upregulated

in

patients

that

progress

to

IPS,

and

is

viceversa

for

control

samples.

The

same

trend

of

differential

expression

of

proteins

is

observed

across

all

other

peptide

pair

possibilities

as

indicated

by

their

DI

values

leading

to

the

highest

median

value

across

all

protein

pairs.

The

top

scoring

protein

pair

APCS

and

HGFAC

provided

highest

discriminability

across

all

peptide

pairs,

APCS

is

an

acute

phase

response

protein

whose

concentration

is

known

to

change

significantly

in

response

to

inflammation

Thus,

the

data

suggests

that

patients

with

dysregulation

in

the

concentration

of

specific

acute

phase

response

proteins

at

the

time

of

bone

marrow

transplant

are

highly

likely

to

develop

IPS

within

weeks..

Figures

and

depict

the

significance

of

the

DI

scores

assessed

using

permutation

tests

for

peptide

and

protein

pairs

respectively.

Figure

represents

distribution

of

the

DI

for

peptide

pairs

generated

using

a

total

of

random

permutations

of

class

labels,

while

maintaining

the

original

sample

size

for

individual

clinical

category.

During

each

permutation,

DI

score

for

all

peptide

pairs

is

calculated,

and

Figure

shows

the

distribution

of

DI

from

all

permutations.

The

true

top

peptide

score

obtained

using

"true

labels"

is

indicated

by

black

arrow,

demonstrating

that

the

probability

of

obtaining

the

true

score

from

distribution

is

extremely

low

(p-value<

−7

Similarly,

Figure

represents

distribution

representing

the

median

DI

of

the

all

protein

pairs

during

permutations,

with

arrow

indicating

the

true

top

DI

score

using

"true

labels",

indicating

high

statistical

significance

of

the

true

score

(p-value<

Note

that

combining

multiple

peptide

pairs

from

same

two

protein

results

in

highly

significant

scores,

since

its

is

unlikely

that

multiple

corresponding

peptide

pairs

show

consistent,

high

discrimination

purely

"by

chance",

as

indicated

by

the

low

p-values

and

low

false

discovery

rates

in

Table

Note

that

the

DI

scores

for

most

of

the

peptide

and

protein

pairs

is

centered

around

zero

indicating

that

no

discrimination

is

being

provided

by

the

such

pairs.

This

is

to

be

expected

with

permuted

labels,

since

most

proteins/peptides

should

not

exhibit

any

differential

Blood

specimens

were

obtained

at

the

time

of

stem

cell

transplantation

(Day

Plasma

was

separated

from

the

samples

and

stored

at

until

analysis.

Aliquots

for

individual

plasma

samples

were

thawed

and

depleted

of

the

seven

most

abundant

proteins.

Bovine

trypsin

was

used

for

proteolytic

digestion.

Six

hundred

nanograms

of

each

sample

were

analyzed

by

liquid

chromatography

coupled

with

mass

spectrometry

using

LTQ-FT

as

previously

described

and

the

order

of

sample

injections

was

randomized

over

all

samples.

The

chromatograms

across

all

the

spectra

thus

obtained

were

time

aligned

across

multiple

runs

using

Rosetta

Elucidator

(Rosetta

Biosoftware,

Seattle,

WA)

Proteins

were

identified

using

Mascot

and

protein

teller

within

Rosetta

Elucidator

framework

raw

peak

areas

corresponding

to

each

peptide

were

used

for

peptide

quantification,

and

were

calculated

from

the

selected

ion

chromatograms

(SICs)

using

Rosetta

Elucidator.

No

normalization/scaling

was

performed

and

peak

areas

were

directly

used

for

ProtPair

analysis.

A

total

of

patients

were

investigated

using

technical

replicates

from

each

patient.

Thirteen

of

the

patients

developed

IPS

following

the

transplants,

while

eight

patients

remained

unaffected.

The

outcome

under

study

was

classification

of

the

SCT

patients

into

those

who

subsequently

developed

IPS

or

those

who

remained

unaffected

by

complications.

Protein

pairs

assigned

high

significance

by

ProtPair

were

imported

into

Ingenuity

Pathway

Analysis

(IPA)

(Ingenuity

Systems,

Redwood

City,

CA,

USA)

to

uncover

protein

networks

enriched

in

the

candidate

proteins.

The

software

generates

networks

based

upon

biomedical

literature

and

existing

protein

interaction

databases

to

reveal

biological

networks

associated

with

the

candidate

proteins..

All

patients

(or

their

surrogates)

and

controls

gave

written,

informed

consent

in

accordance

with

the

Declaration

of

Helsinki

and

the

trial

was

approved

by

the

respective

Institutional

Review

Boards

of

the

University

of

Michigan

and

the

Dana-Farber

Cancer

Center.

O O O O O O B I O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A

statistical

approach

for

person

verification

using

human

behavioral

patterns

We

propose

a

person

verification

method

using

behavioral

patterns

of

human

upper

body

motion.

Behavioral

patterns

are

represented

by

three-dimensional

features

obtained

from

a

time-of-flight

camera.

We

take

a

statistical

approach

to

model

the

behavioral

patterns

using

Gaussian

mixture

models

(GMM)

and

support

vector

machines.

We

employ

the

maximum

likelihood

linear

regression

adaptation

method

to

estimate

GMM

parameters

with

a

limited

amount

of

data.

Experimental

results

show

that

it

reduced

by

the

relative

equal

error

rates

from

a

system

using

the

maximum

likelihood

estimation

with

samples

per

subject.

We

also

demonstrate

that

the

proposed

approach

is

robust

against

variations

in

body

motion

over

time.

Identity

verification

systems

are

getting

popular

in

our

daily

life.

They

provide

a

secure

means

of

controlling

access

to

information

or

equipment.

Traditionally,

these

systems

have

required

something

that

one

has

or

something

that

one

knows

(e.g.,

keys

and

password).

However,

these

representations

of

identity

can

be

easily

lost,

manipulated,

or

stolen.

This

problem

can

be

solved

by

a

biometrics

approach,

which

identifies

an

individual

based

on

his/her

characteristic

traits.

Biometrics

can

be

divided

into

two

classes

physiological

and

behavioral..

Physiological

biometrics

uses

physical

traits,

such

as

fingerprint

or

the

iris

This

type

of

biometrics

is

stable

and

accurate

since

it

relies

on

unique

and

permanent

physical

traits.

However,

they

cannot

be

changed

or

reissued

if

the

biometric

data

are

exposed

or

counterfeited

They

are

also

perceived

as

obtrusive

In

behavioral

biometrics,

a

person's

identity

is

verified

through

action

patterns

which

can

be

repeated

in

a

unique

manner

In

comparison

to

physiological

biometrics,

behavioral

biometrics

are

less

stable

since

behavior

may

change

due

to

the

environment

or

the

physical

state

of

the

individual.

On

the

other

hand,

they

are

not

obtrusive

and

are

difficult

to

disguise

or

to

be

imitated

by

*Correspondence:

felipe@ks.cs.titech.ac.jp

Tokyo

Institute

of

Technology,

Ookayama,

Meguro-ku,

Tokyo,

Japan

others

Examples

of

behavioral

biometrics

include

voice,

keystroke

dynamics,

signature,

and

gait.

Voice

can

be

used

for

remote

identity

verification

Keystroke

verifies

the

identity

of

a

user

while

he/she

uses

a

computer

Signature

can

be

used

in

online

transactions

Gait

has

been

proven

to

be

useful

for

surveillance

applications

where

data

can

be

collected

from

a

distance

In

this

paper,

we

focus

on

the

individuality

of

upper

body

motion

as

an

alternative

cue

for

identity

verification

applications

where

the

acquisition

of

other

behavioral

biometrics

is

not

feasible

due

to

visual

and

space

constraints.

An

example

application

is

an

automatic

gatekeeper,

where

a

person

explicitly

requests

permission

to

access

a

secure

area

by

performing

a

body

gesture

in

front

of

a

camera.

Only

a

few

studies

have

been

done

for

this

kind

of

application.

Pratheepan

et

al.

used

arm

waving

motion

and

simple

actions

such

as

sitting

down,

standing

up,

and

walking

away

for

individual

identification

in

surveillance

applications.

These

studies

used

holistic

features

which

represent

characteristics

from

a

region

of

interest

of

a

person

as

a

whole

They

used

these

features

to

create

templates

that

represent

each

person's

behavioral

patterns..

The

holistic

features

used

in

the

above

mentioned

works

are

sensitive

to

visual

variations

such

as

clothing

differences

and

view

and

scale

changes

since

they

rely

on

global

information

about

a

person's

appearance

To

reduce

the

effects

of

visual

variations,

human

body

models

describing

the

kinematic

properties

(skeleton)

or

the

shape

of

the

body

have

been

used

for

feature

extraction

in

gait

recognition

By

doing

this,

it

is

possible

to

extract

features

from

joint

positions

to

alleviate

the

effects

introduced

by

clothing

changes

or

noise.

However,

this

technique

requires

localization

and

tracking

of

specific

human

body

parts,

which

are

often

erroneous..

Accurate

extraction

of

features

also

depends

on

the

correct

segmentation

of

a

person

from

the

background

scene,

which

can

be

affected

by

texture

and

illumination

changes

Recently,

time-of-flight

(ToF)

cameras

have

been

used

to

reduce

such

errors

by

simplifying

background

segmentation

Furthermore,

ToF

cameras

have

a

low

dependency

on

lighting

conditions

and

invariance

to

color

and

texture..

In

previous

approaches

extracted

features

were

utilized

to

create

a

template

characterizing

a

person's

motion.

However,

templates

are

weak

against

the

natural

variations

of

individual

behavioral

patterns

since

they

only

encode

an

average

representation

of

observed

samples

Statistical

models

such

as

Gaussian

mixture

models

(GMM)

and

hidden

Markov

models

(HMM)

have

successfully

handled

variations

in

individual

behavioral

patterns

For

example,

Kale

et

al.

showed

that

HMMs

were

more

robust

than

templates

for

gait

recognition.

Statistical

models

usually

exhibit

higher

accuracy

than

templates

since

they

model

intraindividual

variations

well

and

are

also

able

to

handle

variations

in

the

sample

duration.

One

drawback

is

that

a

relatively

larger

amount

of

data

is

needed

to

estimate

their

parameters.

The

data

sparseness

problem

is

significant

when

the

training

data

are

limited,

which

is

often

solved

by

adaptation

techniques

such

as

maximum

a

posteriori

(MAP)

maximum

likelihood

linear

regression

(MLLR)

and

eigenspace-based

techniques

In

this

paper,

we

propose

a

statistical

method

for

person

verification

based

on

behavioral

patterns

of

human

upper

body

motion,

extending

our

prior

work

published

in

and

later

in

We

use

depth

information

acquired

from

a

ToF

camera

to

extract

characteristic

features

from

specific

parts

of

the

body

in

the

three-dimensional

space.

GMMs

are

used

to

robustly

model

individuals'

behavioral

patterns.

To

cope

with

the

problem

of

data

sparseness,

we

use

the

MLLR

adaptation

technique.

For

identity

verification

tests,

we

combine

GMMs

with

support

vector

machines

(SVM)

and

compare

its

performance

with

the

GMM

log-likelihood

ratio

framework.

Lastly,

we

evaluate

our

method

using

a

data

set

containing

samples

collected

over

different

sessions

to

demonstrate

that

it

is

able

to

verify

the

identity

of

a

person

even

after

a

period

of

time..

The

remainder

of

this

paper

is

organized

as

follows.

Section

gives

a

brief

overview

of

the

adopted

process

for

person

verification.

Section

describes

the

features

that

are

used

for

person

verification

in

our

approach.

Section

describes

the

statistical

modeling

and

adaptation

techniques

used

to

model

a

person's

behavioral

patterns.

Section

describes

the

classifiers

used

for

person

verification

systems.

In

Section

experimental

conditions

are

explained

and

results

are

presented.

Finally,

conclusion

and

future

work

are

described

in

Section

Figure

shows

an

overview

of

our

system,

consisting

of

three

major

phases:

feature

extraction,

model

training,

and

verification.

In

the

feature

extraction

phase,

the

video

input

is

processed

to

extract

features

from

a

human

upper

body

motion.

In

the

model

training

phase,

statistical

models

are

created

for

each

person

using

the

extracted

features.

In

the

verification

phase,

a

sample

input

is

matched

against

the

claimed

person's

model

to

produce

a

score.

If

the

score

is

above

a

threshold,

the

identity

claim

is

accepted

by

the

system,

otherwise

rejected.

In

our

approach,

we

focus

on

capturing

short

image

stream

samples

of

two

kinds

of

upper

body

motion:

a

vertical

motion

of

either

the

left

or

right

arm.

We

have

proposed

a

statistical

approach

for

person

identity

verification

using

behavioral

patterns

observed

on

human

body

motion.

In

particular,

we

used

behavioral

patterns

from

left

arm

and

right

arm

movements.

By

using

a

ToF

camera,

we

simplified

the

segmentation

of

the

human

body

to

track

specific

human

body

parts

in

the

space.

Since

we

extract

static

and

dynamic

features

of

human

motion

directly

from

identified

landmark

points,

the

effects

of

appearance

changes

were

reduced.

By

taking

a

statistical

approach,

we

effectively

modeled

the

natural

variation

in

features

observed

on

behavioral

patterns.

To

deal

with

the

problem

of

data

sparseness,

we

used

the

MLLR

adaptation

method

along

with

a

UBM

to

estimate

parameters

for

person-dependent

GMMs.

In

addition,

we

used

GMM

components

and

MLLR

transformation

parameters

as

features

to

create

supervectors

in

the

context

of

SVMs..

We

have

shown

that

by

using

a

model

adaptation

method

in

the

training

phase,

the

average

EER

of

the

LLR

system

was

reduced

to

a

relative

reduction

of

compared

with

our

SVM

systems.

The

reason

why

the

SVM

systems

did

not

exhibit

better

performance

might

be

because

the

model

adaptation

used

to

derive

GMM

models

for

creating

supervectors

was

not

as

effective

as

The

number

inside

the

parenthesis

is

the

number

of

Gaussian

mixtures

in

the

GMM

used

for

a

given

system

configuration.

First,

we

show

the

accuracy

results

for

a

landmark

point

localization

test.

Then

the

results

for

person

verification

task

are

presented.

O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B I O O O


Inferential MIMO predictive control of the particle size distribution in emulsion polymerization A new inferential 2-step multiple input/multiple output (MIMO) model predictive control (MPC) of the particle size distribution (PSD) in emulsion polymerization processes is proposed. The bulk-like model describing the PSD is used with the material balances of initiator, radicals, monomer and surfactant. The inferential 2-step control strategy uses two measurements available online (without delay): the concentration of surfactant in the aqueous phase by conductimetry, and the concentration of monomer by calorimetry. In a first step, the optimal trajectory of surfactant concentration leading to the target PSD is calculated offline.In a second step, a multivariable model predictive control manipulates online the monomer and surfactant flow rates in order to track the precalculated surfactant concentration trajectory and to maximize the monomer concentration in the polymer particles in a constrained setpoint tracking. Two control strategies are compared (nonlinear MPC and linearized MPC) with and without modelling errors.Model predictive control ; emulsion polymerization ; particle size distribution ; nonlinear distributed parameter system ; inferential control. Polymers are the bases of many products. A big part of these products is produced by emulsion polymerization, such as dispersions (paints and adhesives, such as polyvinyl acetate, acrylic polymers), plastics after particle coagulation (e.g.: polystyrene, polymethyl methacrylate) and rubber (e.g.: styrene butadiene rubber, neoprene rubber). This process has several advantages over other free radical polymerization processes comprising the high conversion rate (close to 100%) and the low viscosity of the medium which enhances temperature control and reaction mixing. Also, the continuous phase in this process is water which reduces the amount of volatile organic compounds. The emulsion medium contains the monomer (usually water insoluble) which is the basis of the polymer, the reaction initiator and the surfactant which allows stabilization of the monomer droplets and polymer particles. The latex produced by emulsion polymerization is characterized by its polymer molecular weight and particle size distribution (PSD). The molecular weight distribution affects the mechanical properties of the final polymer, whereas the PSD affects the rheological and optical properties of the latex and its stability. The PSD also affects film formation properties of the latex (Geurts et al., 1996) . Berend and Richtering (1995) studied the rheology of monodispersed and bidispersed lattices and concluded that the viscosity depends on the volume fraction of small to big particles and on the size of each population (see figure 1 ). Schneider et al. (2002) have shown that bimodal PSD allows increasing the solid content while maintaining a low viscosity (compared to monomodal PSD), which is of interest since high solid contents are usually desired in order to reduce transport costs and drying time. Therefore, it appears that controlling the PSD (of the final latex) is primordial to obtain specific end-use properties for paints and adhesives. It is worthy to note that the process model describing the PSD is nonlinear, distributed and includes an important number of parameters that are not well known or sensitive to impurities. This renders the model-based control of the PSD a difficult task. Moreover, online measurement of bimodal PSD is not available. Existing technologies (such as light scattering) mainly concern the measurement of monodispersed lattices and operate offline. Elizalde et al. (2000) and Schneider and McKenna (2002) realised a comparative study of technologies used to measure the PSD. Separative technologies, such as capillary hydrodynamic fractionation (CHDF), were found to be the most adapted for multi-dispersed lattices. But, the main inconvenience of these techniques is the long analysis time. Therefore, these measurements can hardly be used in an online closed-loop control scheme. An interesting alternative measurement for controlling the PSD is the online measurement of the concentration of free surfactant in the aqueous phase which gives information about the stability of particles and the concentration of micelles. Santos et al. (2007) used the conductimetry and ion-selective electrodes to measure the concentration of free anionic surfactant molecules (sodium dodecyl sulfate) in the aqueous phase in emulsion polymerisation.. Until now, the control of the PSD was treated in different ways in the literature to obtain a bimodal PSD. For the PSD control in such processes, one may distinguish, batch to batch, indirect and direct control strategies (Kiparissides, 2006) :. • Batch to batch control In this method, an offline optimization study is first realised based on the process model to determine the optimal input profiles that give the desired PSD. Then, the obtained input profiles are applied experimentally, therefore assuming perfect process modelling. By collecting measurements after each batch, modelling errors can be corrected and can be assumed to remain unchanged for a given lot of products. By applying this method, Crowley et al. (2000) used the surfactant flow rate to control the PSD. The so-called zero-one model was used to represent the PSD in a semi-batch emulsion polymerization of styrene. Flores- Cerillo and MacGregor (2002) proposed a batch to batch control strategy by adding an online correction of the surfactant concentration in the middle of the reaction using a prediction of the PSD obtained by partial least square models (based on online temperature measurements and offline surface tension and PSD measurements). Immanuel and Doyle III (2002) used an open loop genetic control algorithm to predict the control profiles to obtain the desired PSD of a copolymerization process. Doyle III et al. (2003) proposed a hybrid model-based approach for batch-to-batch control of the PSD. A fundamental population balance model describing the PSD evolution was augmented by a partial least squares model. By this way, only slight batch-to-batch correction was required. Both the surfactant and initiator flow rates were manipulated. Online measurement of the full PSD was assumed available, which is still a challenging issue. The batch to batch iterative feedback PSD control was also used by Immanuel et al. (2008) . They considered correction of modelling errors from batch to batch.. Recently, Alamir et al. (2010) applied an iterative learning control of the PSD by proposing a simplified behavioural model to describe the PSD. Besides offline correction of the model and control inputs, delayed online measurements of the PSD were used in the feedback control. Using a behavioural model represents an advantage due to its simplicity and therefore shorter run time, even though it might increase the batch to batch correction level.. • Indirect (or inferential) control In inferential control, even though the desired property to control is not measured online, the controller uses the available measurements to fulfil its objective. Abedini and Shahrokhi (2008) used the measurement of the surfactant concentration to control the PSD by a single input/single output PID controller that tunes the free surfactant flow rate. They assumed 5% impurities in the surfactant and 10% model mismatch mainly in the coefficient of propagation rate and the diffusion coefficient of monomer, to evaluate the robustness of their strategy. The monomer concentration was not controlled. In this work, a new inferential model-based control strategy of the PSD in emulsion polymerisation was developed. Two online measurements were considered for the control purpose: the monomer conversion by calorimetry and the concentration of surfactant in the aqueous phase by conductimetry. The first limitation may be that conductimetry was validated experimentally only on low solid content systems. The conductimetry still can be applied to high solid content systems if a circulating and dilution loop is added. Meanwhile, it should be reminded that direct online measurement of the PSD is not available and therefore, the conductimetry represents a good alternative. Moreover, the sensors measuring the PSD are much more expensive than a simple conductimeter. This dissuades their installation for online monitoring in the industry, even though it might be possible with a circulation loop. It is to be noted that the papers treating direct control of the PSD (based on online measurement of the PSD) were mainly simulation studies.. Here, a new 2 step MIMO control strategy for emulsion polymerisation control using MPC was constructed as follows. In the first step, the free surfactant concentration trajectory was pre-calculated offline, by model-based optimization, in order to get a predefined bimodal PSD. In the second step, a MIMO MPC was applied online to track the pre-calculated trajectory of free surfactant concentration and also to maximize the concentration of monomer The comparison between the linearized and nonlinear MPC strategies showed that the linearized model approach helps to decrease the calculation time by a factor of 8 while the performance degradation is lower than 7%. The linearized approach could therefore be implemented with a short sampling time (10s), which was not the case for the nonlinear controller. Moreover, optimization using a nonlinear model is not always evident and might not converge correctly. It should be noted however that the development time of a control strategy based on the nonlinear is more straightforward compared to the strategy based on the linearized model. The entry rate coefficient into micelles is: Two simulation cases are discussed:. The first case assumes a perfect model, which helps to validate the method.. The second case assumes a parameter uncertainty, which has an important impact on the open-loop results. This helps to evaluate the performance of the controller in a more realistic case.. In the first case, the proposed linearized MPC strategy is compared to a nonlinear MPC.

O O O O O O B I I O B I O O O O O O O O B I I O O O B I I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I I O B I I I O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


TREE INCLUSION PROBLEMS Given two trees (a target T and a pattern P ) and a natural number w, window embedded subtree problems consist in deciding whether P occurs as an embedded subtree of T and/or finding the number of size (at most) w windows of T which contain pattern P as an embedded subtree.P is an embedded subtree of T if P can be obtained by deleting some nodes from T (if a node v is deleted, all edges adjacent to v are also deleted, and outgoing edges are replaced by edges going from the parent of v (if it exists) to the children of v). Deciding whether P is an embedded subtree of T is known to be NP-complete.Our algorithms run in time O(|T |2 2 |P | ) where |T | (resp. |P |) is the size of T (resp. P ). In the present paper we answered some problems about tree inclusions, namely deciding whether a pattern is an embedded subtree of a target within a w-window, counting the number of windows of height at most w of the target containing the pattern as an embedded subtree, and counting the number of occurrences of the pattern as an embedded subtree of the target within a w-window.. There are many other interesting problems concerning tree inclusions, for instance, counting the number of windows of height exactly w of the target containing the pattern as an embedded subtree, or counting in a slice of the target tree. We traverse T bottom-up (from leaves to root, or in postorder): to scan a node v of T we must first have scanned all its children. With each node v of T we will associate a configuration C v such that for every forest F from C v all trees from F can be simultaneously embedded into T [v]. This leads to the following algorithm. (See Figure 3 (1) for an example.)

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O